[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:10:50.146476 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:44521
I20260709 12:10:50.148113 10011 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:10:50.149226 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:10:50.166915 10018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:10:50.166956 10020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:10:50.168207 10017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:10:51.363297 10019 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 12:10:51.363376 10011 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:10:51.368221 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:10:51.368465 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:10:51.368623 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599051368607 us; error 0 us; skew 500 ppm
I20260709 12:10:51.369401 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:10:51.374732 10011 webserver.cc:533] Webserver started at http://127.9.198.254:35269/ using document root <none> and password file <none>
I20260709 12:10:51.375547 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:10:51.375734 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:10:51.376114 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:10:51.381882 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "4089ee2edc8b44b1b9824070cb041134"
format_stamp: "Formatted at 2026-07-09 12:10:51 on dist-test-slave-glnm"
I20260709 12:10:51.389590 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:10:51.394984 10026 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:51.396340 10011 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:10:51.396726 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "4089ee2edc8b44b1b9824070cb041134"
format_stamp: "Formatted at 2026-07-09 12:10:51 on dist-test-slave-glnm"
I20260709 12:10:51.397087 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:10:51.449231 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:10:51.451010 10011 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:10:51.451515 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:10:51.524564 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:44521
I20260709 12:10:51.524655 10077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:44521 every 8 connection(s)
I20260709 12:10:51.532564 10078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:51.554759 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134: Bootstrap starting.
I20260709 12:10:51.560899 10078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:51.563058 10078 log.cc:826] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134: Log is configured to *not* fsync() on all Append() calls
I20260709 12:10:51.567698 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134: No bootstrap required, opened a new log
I20260709 12:10:51.588642 10078 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4089ee2edc8b44b1b9824070cb041134" member_type: VOTER }
I20260709 12:10:51.589228 10078 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:51.589509 10078 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4089ee2edc8b44b1b9824070cb041134, State: Initialized, Role: FOLLOWER
I20260709 12:10:51.590548 10078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4089ee2edc8b44b1b9824070cb041134" member_type: VOTER }
I20260709 12:10:51.591102 10078 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:10:51.591403 10078 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:10:51.591756 10078 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:51.595913 10078 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4089ee2edc8b44b1b9824070cb041134" member_type: VOTER }
I20260709 12:10:51.596653 10078 leader_election.cc:304] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4089ee2edc8b44b1b9824070cb041134; no voters: 
I20260709 12:10:51.598413 10078 leader_election.cc:290] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:10:51.598788 10081 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:51.601552 10081 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 1 LEADER]: Becoming Leader. State: Replica: 4089ee2edc8b44b1b9824070cb041134, State: Running, Role: LEADER
I20260709 12:10:51.602509 10081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4089ee2edc8b44b1b9824070cb041134" member_type: VOTER }
I20260709 12:10:51.603533 10078 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:10:51.613423 10082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4089ee2edc8b44b1b9824070cb041134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4089ee2edc8b44b1b9824070cb041134" member_type: VOTER } }
I20260709 12:10:51.613690 10083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4089ee2edc8b44b1b9824070cb041134. Latest consensus state: current_term: 1 leader_uuid: "4089ee2edc8b44b1b9824070cb041134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4089ee2edc8b44b1b9824070cb041134" member_type: VOTER } }
I20260709 12:10:51.614261 10082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [sys.catalog]: This master's current role is: LEADER
I20260709 12:10:51.614445 10083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [sys.catalog]: This master's current role is: LEADER
I20260709 12:10:51.618108 10090 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:10:51.631675 10090 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:10:51.634021 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:10:51.649319 10090 catalog_manager.cc:1383] Generated new cluster ID: 6c4d04986e674fe3ae1eee48c84b5bfc
I20260709 12:10:51.649685 10090 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:10:51.672504 10090 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:10:51.674960 10090 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:10:51.692544 10090 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134: Generated new TSK 0
I20260709 12:10:51.693811 10090 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:10:51.700516 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:10:51.709776 10100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:10:51.711673 10101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:10:51.714488 10103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:10:51.714551 10011 server_base.cc:1061] running on GCE node
I20260709 12:10:51.715621 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:10:51.715886 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:10:51.716092 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599051716074 us; error 0 us; skew 500 ppm
I20260709 12:10:51.716897 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:10:51.721174 10011 webserver.cc:533] Webserver started at http://127.9.198.193:43671/ using document root <none> and password file <none>
I20260709 12:10:51.721961 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:10:51.722220 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:10:51.722584 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:10:51.724135 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "2542817e0dd345b3a0fb40a129e92cd4"
format_stamp: "Formatted at 2026-07-09 12:10:51 on dist-test-slave-glnm"
I20260709 12:10:51.731436 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:10:51.736907 10108 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:51.737912 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:10:51.738286 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "2542817e0dd345b3a0fb40a129e92cd4"
format_stamp: "Formatted at 2026-07-09 12:10:51 on dist-test-slave-glnm"
I20260709 12:10:51.738680 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:10:51.760466 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:10:51.762306 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:10:51.765290 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:10:51.770907 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:10:51.771225 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:51.771560 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:10:51.771798 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:51.865396 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:38395
I20260709 12:10:51.865455 10170 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:38395 every 8 connection(s)
I20260709 12:10:51.873275 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:10:51.890499 10176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:10:51.892866 10175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:10:51.899071 10011 server_base.cc:1061] running on GCE node
W20260709 12:10:51.899423 10178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:10:51.900597 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:10:51.900849 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:10:51.901044 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599051901022 us; error 0 us; skew 500 ppm
I20260709 12:10:51.901739 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:10:51.905453 10011 webserver.cc:533] Webserver started at http://127.9.198.194:46527/ using document root <none> and password file <none>
I20260709 12:10:51.906179 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:10:51.906452 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:10:51.906813 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:10:51.908079 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
format_stamp: "Formatted at 2026-07-09 12:10:51 on dist-test-slave-glnm"
I20260709 12:10:51.908625 10171 heartbeater.cc:344] Connected to a master server at 127.9.198.254:44521
I20260709 12:10:51.909113 10171 heartbeater.cc:461] Registering TS with master...
I20260709 12:10:51.910391 10171 heartbeater.cc:507] Master 127.9.198.254:44521 requested a full tablet report, sending...
I20260709 12:10:51.914096 10043 ts_manager.cc:194] Registered new tserver with Master: 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395)
I20260709 12:10:51.917065 10043 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46534
I20260709 12:10:51.918859 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.003s	sys 0.009s
I20260709 12:10:51.925170 10183 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:51.926090 10011 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:10:51.926393 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
format_stamp: "Formatted at 2026-07-09 12:10:51 on dist-test-slave-glnm"
I20260709 12:10:51.926689 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:10:51.954057 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:10:51.955349 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:10:51.957466 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:10:51.960345 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:10:51.960592 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:51.960875 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:10:51.961050 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:52.057960 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:34737
I20260709 12:10:52.058063 10245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:34737 every 8 connection(s)
I20260709 12:10:52.067224 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:10:52.074172 10249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:10:52.079344 10250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:10:52.080235 10246 heartbeater.cc:344] Connected to a master server at 127.9.198.254:44521
I20260709 12:10:52.080755 10246 heartbeater.cc:461] Registering TS with master...
I20260709 12:10:52.081817 10246 heartbeater.cc:507] Master 127.9.198.254:44521 requested a full tablet report, sending...
I20260709 12:10:52.084481 10043 ts_manager.cc:194] Registered new tserver with Master: 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:52.086047 10011 server_base.cc:1061] running on GCE node
I20260709 12:10:52.086617 10043 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46544
W20260709 12:10:52.088025 10252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:10:52.089090 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:10:52.089282 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:10:52.089488 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599052089456 us; error 0 us; skew 500 ppm
I20260709 12:10:52.090067 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:10:52.093190 10011 webserver.cc:533] Webserver started at http://127.9.198.195:43947/ using document root <none> and password file <none>
I20260709 12:10:52.093711 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:10:52.093904 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:10:52.094229 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:10:52.095321 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
format_stamp: "Formatted at 2026-07-09 12:10:52 on dist-test-slave-glnm"
I20260709 12:10:52.099870 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:10:52.103318 10257 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:52.104079 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:10:52.104359 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
format_stamp: "Formatted at 2026-07-09 12:10:52 on dist-test-slave-glnm"
I20260709 12:10:52.104651 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:10:52.116086 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:10:52.117292 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:10:52.118866 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:10:52.120929 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:10:52.121147 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:52.121388 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:10:52.121563 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:10:52.228904 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:32981
I20260709 12:10:52.229043 10319 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:32981 every 8 connection(s)
I20260709 12:10:52.248047 10320 heartbeater.cc:344] Connected to a master server at 127.9.198.254:44521
I20260709 12:10:52.248446 10320 heartbeater.cc:461] Registering TS with master...
I20260709 12:10:52.249307 10320 heartbeater.cc:507] Master 127.9.198.254:44521 requested a full tablet report, sending...
I20260709 12:10:52.251379 10043 ts_manager.cc:194] Registered new tserver with Master: e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:52.252269 10011 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018050027s
I20260709 12:10:52.252714 10043 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46558
I20260709 12:10:52.306627 10043 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46562:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:10:52.310779 10043 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:10:52.421881 10285 tablet_service.cc:1511] Processing CreateTablet for tablet f83ee2731302482d8edfac1392b77842 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:10:52.424360 10285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f83ee2731302482d8edfac1392b77842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.424125 10284 tablet_service.cc:1511] Processing CreateTablet for tablet 62f27d2b65514590b6679d2eb6828fee (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:10:52.424485 10211 tablet_service.cc:1511] Processing CreateTablet for tablet f83ee2731302482d8edfac1392b77842 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:10:52.428925 10284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f27d2b65514590b6679d2eb6828fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.429972 10208 tablet_service.cc:1511] Processing CreateTablet for tablet ee2f944f89c241908a02e20a71442ba9 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:10:52.433213 10282 tablet_service.cc:1511] Processing CreateTablet for tablet ee2f944f89c241908a02e20a71442ba9 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:10:52.434693 10282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2f944f89c241908a02e20a71442ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.439600 10281 tablet_service.cc:1511] Processing CreateTablet for tablet 981e7a9122eb40428f69c523a7cccb73 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:10:52.441128 10281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981e7a9122eb40428f69c523a7cccb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.444675 10280 tablet_service.cc:1511] Processing CreateTablet for tablet 59d069286ea84159b88ed2322a9d2db0 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:10:52.446120 10280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d069286ea84159b88ed2322a9d2db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.450196 10211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f83ee2731302482d8edfac1392b77842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.450723 10283 tablet_service.cc:1511] Processing CreateTablet for tablet 23b4ead3e9474e02810ad3d030508b8e (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:10:52.448876 10209 tablet_service.cc:1511] Processing CreateTablet for tablet 23b4ead3e9474e02810ad3d030508b8e (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:10:52.451256 10210 tablet_service.cc:1511] Processing CreateTablet for tablet 62f27d2b65514590b6679d2eb6828fee (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:10:52.457616 10207 tablet_service.cc:1511] Processing CreateTablet for tablet 981e7a9122eb40428f69c523a7cccb73 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:10:52.455686 10279 tablet_service.cc:1511] Processing CreateTablet for tablet c1bcd0dfb70d4c229823e441456f9291 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:10:52.459530 10208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2f944f89c241908a02e20a71442ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.466763 10209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b4ead3e9474e02810ad3d030508b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.467111 10283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b4ead3e9474e02810ad3d030508b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.466082 10205 tablet_service.cc:1511] Processing CreateTablet for tablet c1bcd0dfb70d4c229823e441456f9291 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:10:52.466121 10206 tablet_service.cc:1511] Processing CreateTablet for tablet 59d069286ea84159b88ed2322a9d2db0 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:10:52.471001 10279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bcd0dfb70d4c229823e441456f9291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.491153 10207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981e7a9122eb40428f69c523a7cccb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.493762 10206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d069286ea84159b88ed2322a9d2db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.500054 10210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f27d2b65514590b6679d2eb6828fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.504854 10338 tablet_bootstrap.cc:492] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:52.507817 10205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bcd0dfb70d4c229823e441456f9291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.514652 10204 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6bb5d638a2405ab5599d59dc4c8d39 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:10:52.517058 10204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6bb5d638a2405ab5599d59dc4c8d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.519688 10282 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6bb5d638a2405ab5599d59dc4c8d39 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:10:52.520939 10282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6bb5d638a2405ab5599d59dc4c8d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.545372 10203 tablet_service.cc:1511] Processing CreateTablet for tablet 79f1ea9709e74f4e950a9becd454ab45 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:10:52.551448 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79f1ea9709e74f4e950a9becd454ab45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.556193 10135 tablet_service.cc:1511] Processing CreateTablet for tablet 62f27d2b65514590b6679d2eb6828fee (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:10:52.561605 10136 tablet_service.cc:1511] Processing CreateTablet for tablet f83ee2731302482d8edfac1392b77842 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:10:52.566412 10134 tablet_service.cc:1511] Processing CreateTablet for tablet 23b4ead3e9474e02810ad3d030508b8e (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:10:52.576818 10135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f27d2b65514590b6679d2eb6828fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.580762 10136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f83ee2731302482d8edfac1392b77842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.584906 10131 tablet_service.cc:1511] Processing CreateTablet for tablet 59d069286ea84159b88ed2322a9d2db0 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:10:52.576894 10133 tablet_service.cc:1511] Processing CreateTablet for tablet ee2f944f89c241908a02e20a71442ba9 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:10:52.587035 10134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b4ead3e9474e02810ad3d030508b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.573897 10130 tablet_service.cc:1511] Processing CreateTablet for tablet c1bcd0dfb70d4c229823e441456f9291 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:10:52.590196 10339 tablet_bootstrap.cc:492] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:52.591033 10131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d069286ea84159b88ed2322a9d2db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.584983 10132 tablet_service.cc:1511] Processing CreateTablet for tablet 981e7a9122eb40428f69c523a7cccb73 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:10:52.582458 10129 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6bb5d638a2405ab5599d59dc4c8d39 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:10:52.588644 10128 tablet_service.cc:1511] Processing CreateTablet for tablet 79f1ea9709e74f4e950a9becd454ab45 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:10:52.599087 10130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bcd0dfb70d4c229823e441456f9291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.601292 10132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981e7a9122eb40428f69c523a7cccb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.630170 10128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79f1ea9709e74f4e950a9becd454ab45. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:10:52.643138 10091 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:10:52.646502 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6bb5d638a2405ab5599d59dc4c8d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.653266 10133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2f944f89c241908a02e20a71442ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.593547 10280 tablet_service.cc:1511] Processing CreateTablet for tablet 79f1ea9709e74f4e950a9becd454ab45 (DEFAULT_TABLE table=test-workload [id=8f9e47cb4e1940a6886e8dde6bfc4d0e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:10:52.660053 10280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79f1ea9709e74f4e950a9becd454ab45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:52.690194 10339 tablet_bootstrap.cc:654] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.708156 10338 tablet_bootstrap.cc:654] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.781836 10341 tablet_bootstrap.cc:492] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:52.791253 10339 tablet_bootstrap.cc:492] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:52.791744 10339 ts_tablet_manager.cc:1403] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.227s	user 0.011s	sys 0.020s
I20260709 12:10:52.796945 10339 raft_consensus.cc:359] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.797752 10339 raft_consensus.cc:385] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.798385 10339 raft_consensus.cc:740] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.799903 10339 consensus_queue.cc:260] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.802285 10341 tablet_bootstrap.cc:654] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.804816 10338 tablet_bootstrap.cc:492] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:52.805260 10338 ts_tablet_manager.cc:1403] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.316s	user 0.003s	sys 0.037s
I20260709 12:10:52.807014 10246 heartbeater.cc:499] Master 127.9.198.254:44521 was elected leader, sending a full tablet report...
I20260709 12:10:52.806478 10339 ts_tablet_manager.cc:1434] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.009s
I20260709 12:10:52.809818 10339 tablet_bootstrap.cc:492] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:52.809897 10338 raft_consensus.cc:359] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.811229 10338 raft_consensus.cc:385] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.811554 10338 raft_consensus.cc:740] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.812266 10338 consensus_queue.cc:260] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
W20260709 12:10:52.818452 10247 tablet.cc:2406] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:10:52.820142 10339 tablet_bootstrap.cc:654] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.821605 10341 tablet_bootstrap.cc:492] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:52.822118 10341 ts_tablet_manager.cc:1403] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.048s	user 0.005s	sys 0.008s
I20260709 12:10:52.825187 10341 raft_consensus.cc:359] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.825985 10341 raft_consensus.cc:385] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.826323 10341 raft_consensus.cc:740] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.827101 10341 consensus_queue.cc:260] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.834498 10339 tablet_bootstrap.cc:492] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:52.835166 10339 ts_tablet_manager.cc:1403] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260709 12:10:52.839064 10339 raft_consensus.cc:359] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:52.839931 10339 raft_consensus.cc:385] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.840274 10339 raft_consensus.cc:740] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.840583 10320 heartbeater.cc:499] Master 127.9.198.254:44521 was elected leader, sending a full tablet report...
I20260709 12:10:52.840898 10338 ts_tablet_manager.cc:1434] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.035s	user 0.024s	sys 0.008s
I20260709 12:10:52.841014 10339 consensus_queue.cc:260] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:52.842049 10338 tablet_bootstrap.cc:492] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:52.843477 10339 ts_tablet_manager.cc:1434] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:10:52.844497 10339 tablet_bootstrap.cc:492] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:52.848790 10338 tablet_bootstrap.cc:654] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.851172 10339 tablet_bootstrap.cc:654] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.861565 10338 tablet_bootstrap.cc:492] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:52.861814 10339 tablet_bootstrap.cc:492] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:52.862087 10338 ts_tablet_manager.cc:1403] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.011s
I20260709 12:10:52.862221 10339 ts_tablet_manager.cc:1403] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.001s
I20260709 12:10:52.863303 10171 heartbeater.cc:499] Master 127.9.198.254:44521 was elected leader, sending a full tablet report...
I20260709 12:10:52.865092 10338 raft_consensus.cc:359] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.865203 10339 raft_consensus.cc:359] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.865936 10339 raft_consensus.cc:385] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.865893 10338 raft_consensus.cc:385] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.866321 10339 raft_consensus.cc:740] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.866359 10338 raft_consensus.cc:740] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.867168 10339 consensus_queue.cc:260] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.868528 10341 ts_tablet_manager.cc:1434] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.046s	user 0.014s	sys 0.008s
I20260709 12:10:52.869486 10341 tablet_bootstrap.cc:492] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:52.874161 10339 ts_tablet_manager.cc:1434] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 12:10:52.875145 10339 tablet_bootstrap.cc:492] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:52.877975 10341 tablet_bootstrap.cc:654] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.884902 10339 tablet_bootstrap.cc:654] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.890272 10338 consensus_queue.cc:260] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.893000 10338 ts_tablet_manager.cc:1434] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 12:10:52.894326 10338 tablet_bootstrap.cc:492] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:52.907451 10341 tablet_bootstrap.cc:492] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:52.908054 10341 ts_tablet_manager.cc:1403] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.012s
I20260709 12:10:52.911255 10338 tablet_bootstrap.cc:654] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.911005 10341 raft_consensus.cc:359] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.911851 10341 raft_consensus.cc:385] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.912216 10341 raft_consensus.cc:740] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.913015 10341 consensus_queue.cc:260] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.915514 10341 ts_tablet_manager.cc:1434] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:10:52.917517 10345 raft_consensus.cc:493] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.918033 10345 raft_consensus.cc:515] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.921139 10345 leader_election.cc:290] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:52.924407 10338 tablet_bootstrap.cc:492] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:52.925347 10341 tablet_bootstrap.cc:492] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:52.924839 10338 ts_tablet_manager.cc:1403] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.004s
I20260709 12:10:52.928580 10338 raft_consensus.cc:359] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.929414 10338 raft_consensus.cc:385] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.929754 10338 raft_consensus.cc:740] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.930485 10338 consensus_queue.cc:260] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.931929 10341 tablet_bootstrap.cc:654] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.933944 10339 tablet_bootstrap.cc:492] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:52.934410 10339 ts_tablet_manager.cc:1403] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.059s	user 0.017s	sys 0.018s
I20260709 12:10:52.937157 10338 ts_tablet_manager.cc:1434] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:10:52.938174 10338 tablet_bootstrap.cc:492] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:52.937748 10339 raft_consensus.cc:359] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.941473 10339 raft_consensus.cc:385] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.941931 10339 raft_consensus.cc:740] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.942890 10339 consensus_queue.cc:260] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.949808 10338 tablet_bootstrap.cc:654] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.954998 10339 ts_tablet_manager.cc:1434] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:10:52.956431 10339 tablet_bootstrap.cc:492] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:52.957724 10343 raft_consensus.cc:493] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.958294 10343 raft_consensus.cc:515] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.961751 10343 leader_election.cc:290] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:52.962375 10343 raft_consensus.cc:493] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.962916 10343 raft_consensus.cc:515] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.965335 10343 leader_election.cc:290] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:52.973227 10339 tablet_bootstrap.cc:654] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:52.982054 10339 tablet_bootstrap.cc:492] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:52.984467 10339 ts_tablet_manager.cc:1403] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20260709 12:10:52.989152 10341 tablet_bootstrap.cc:492] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:52.989789 10341 ts_tablet_manager.cc:1403] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.065s	user 0.038s	sys 0.009s
I20260709 12:10:52.993352 10341 raft_consensus.cc:359] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.994117 10341 raft_consensus.cc:385] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.993801 10339 raft_consensus.cc:359] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.994431 10341 raft_consensus.cc:740] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.995406 10341 consensus_queue.cc:260] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.994540 10339 raft_consensus.cc:385] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:52.997108 10339 raft_consensus.cc:740] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:52.998495 10341 ts_tablet_manager.cc:1434] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:10:52.998826 10339 consensus_queue.cc:260] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:52.999461 10341 tablet_bootstrap.cc:492] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:53.002632 10339 ts_tablet_manager.cc:1434] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 12:10:53.003659 10339 tablet_bootstrap.cc:492] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:53.004848 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.005787 10221 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 0.
I20260709 12:10:53.010445 10338 tablet_bootstrap.cc:492] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:53.010941 10338 ts_tablet_manager.cc:1403] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.016s
I20260709 12:10:53.010824 10260 leader_election.cc:304] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.011830 10341 tablet_bootstrap.cc:654] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.014648 10338 raft_consensus.cc:359] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.015574 10338 raft_consensus.cc:385] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.015969 10338 raft_consensus.cc:740] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.016678 10343 raft_consensus.cc:493] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.017257 10343 raft_consensus.cc:515] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.019807 10343 leader_election.cc:290] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.022200 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.024199 10341 tablet_bootstrap.cc:492] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:53.024694 10341 ts_tablet_manager.cc:1403] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.025s	user 0.001s	sys 0.016s
I20260709 12:10:53.025259 10338 consensus_queue.cc:260] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.028467 10341 raft_consensus.cc:359] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.029174 10341 raft_consensus.cc:385] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.029469 10341 raft_consensus.cc:740] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.030201 10341 consensus_queue.cc:260] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.028357 10339 tablet_bootstrap.cc:654] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.030784 10338 ts_tablet_manager.cc:1434] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:10:53.031963 10338 tablet_bootstrap.cc:492] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:53.032792 10341 ts_tablet_manager.cc:1434] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:10:53.033797 10341 tablet_bootstrap.cc:492] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:53.038730 10338 tablet_bootstrap.cc:654] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.040304 10341 tablet_bootstrap.cc:654] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.041519 10347 raft_consensus.cc:493] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.042065 10347 raft_consensus.cc:515] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.045526 10347 leader_election.cc:290] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
W20260709 12:10:53.058135 10259 leader_election.cc:343] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.058337 10345 raft_consensus.cc:2804] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.059016 10345 raft_consensus.cc:493] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:53.059139 10260 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:34737, user_credentials={real_user=slave}} blocked reactor thread for 48721.8us
I20260709 12:10:53.059559 10345 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.062588 10347 raft_consensus.cc:493] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.063103 10347 raft_consensus.cc:515] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.065183 10347 leader_election.cc:290] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.068955 10345 raft_consensus.cc:515] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.071703 10345 leader_election.cc:290] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:53.073220 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:53.074011 10221 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.083667 10221 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 1.
I20260709 12:10:53.085153 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:53.085939 10341 tablet_bootstrap.cc:492] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:53.085980 10146 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.086578 10341 ts_tablet_manager.cc:1403] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.018s
I20260709 12:10:53.085110 10260 leader_election.cc:304] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.090072 10341 raft_consensus.cc:359] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.090600 10345 raft_consensus.cc:2804] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.090865 10341 raft_consensus.cc:385] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.091301 10341 raft_consensus.cc:740] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.092106 10341 consensus_queue.cc:260] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.099035 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.099778 10220 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 0.
I20260709 12:10:53.102646 10111 leader_election.cc:304] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:53.103519 10347 raft_consensus.cc:2804] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.103991 10347 raft_consensus.cc:493] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.104404 10347 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.102963 10146 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 1.
I20260709 12:10:53.108541 10341 ts_tablet_manager.cc:1434] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:10:53.111703 10341 tablet_bootstrap.cc:492] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:53.112041 10345 raft_consensus.cc:697] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Becoming Leader. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:10:53.112540 10347 raft_consensus.cc:515] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.112993 10345 consensus_queue.cc:237] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.115080 10347 leader_election.cc:290] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.107556 10371 raft_consensus.cc:493] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.115756 10371 raft_consensus.cc:515] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.116549 10338 tablet_bootstrap.cc:492] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:53.117020 10338 ts_tablet_manager.cc:1403] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.034s
I20260709 12:10:53.118336 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:53.118887 10220 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.119089 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.119354 10371 leader_election.cc:290] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:53.119920 10146 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 0.
I20260709 12:10:53.120290 10338 raft_consensus.cc:359] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.121069 10338 raft_consensus.cc:385] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.121367 10338 raft_consensus.cc:740] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.121615 10259 leader_election.cc:304] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.122107 10338 consensus_queue.cc:260] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.122714 10371 raft_consensus.cc:2804] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.123134 10371 raft_consensus.cc:493] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.123497 10371 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.124454 10338 ts_tablet_manager.cc:1434] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:10:53.127125 10220 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 1.
I20260709 12:10:53.128427 10111 leader_election.cc:304] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:53.129211 10347 raft_consensus.cc:2804] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.136483 10338 tablet_bootstrap.cc:492] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:53.142606 10341 tablet_bootstrap.cc:654] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.148290 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.149303 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
W20260709 12:10:53.151288 10111 leader_election.cc:343] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:10:53.151932 10260 leader_election.cc:343] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.162968 10371 raft_consensus.cc:515] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.165616 10338 tablet_bootstrap.cc:654] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.165390 10347 raft_consensus.cc:697] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:53.168241 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981e7a9122eb40428f69c523a7cccb73" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.176506 10347 consensus_queue.cc:237] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.178906 10293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.180778 10291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.181914 10293 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 0.
I20260709 12:10:53.182104 10291 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 0.
I20260709 12:10:53.177858 10294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.177776 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:53.188407 10339 tablet_bootstrap.cc:492] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:53.177821 10295 raft_consensus.cc:2468] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 0.
I20260709 12:10:53.188895 10339 ts_tablet_manager.cc:1403] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.185s	user 0.021s	sys 0.051s
I20260709 12:10:53.179991 10043 catalog_manager.cc:5697] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f reported cstate change: term changed from 0 to 1, leader changed from <none> to e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195). New cstate: current_term: 1 leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:53.186890 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
W20260709 12:10:53.189376 10112 leader_election.cc:343] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.190225 10146 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.190728 10112 leader_election.cc:304] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4; no voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:53.187242 10292 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.192222 10373 raft_consensus.cc:2749] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:10:53.192363 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.190213 10371 leader_election.cc:290] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:53.194371 10187 leader_election.cc:304] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.196128 10357 raft_consensus.cc:2804] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.196547 10357 raft_consensus.cc:493] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.196889 10357 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.200696 10187 leader_election.cc:304] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.202477 10343 raft_consensus.cc:2804] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.202533 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981e7a9122eb40428f69c523a7cccb73" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.202986 10343 raft_consensus.cc:493] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.203459 10343 raft_consensus.cc:3060] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.205824 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:53.209156 10341 tablet_bootstrap.cc:492] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:53.209677 10341 ts_tablet_manager.cc:1403] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.098s	user 0.013s	sys 0.023s
W20260709 12:10:53.208609 10187 leader_election.cc:343] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:10:53.210512 10260 leader_election.cc:343] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.205590 10146 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 1.
I20260709 12:10:53.212352 10259 leader_election.cc:304] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:10:53.213856 10371 raft_consensus.cc:2804] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.214990 10371 raft_consensus.cc:697] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Becoming Leader. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:10:53.220255 10357 raft_consensus.cc:515] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.233949 10292 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 1.
I20260709 12:10:53.218019 10371 consensus_queue.cc:237] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.236235 10343 raft_consensus.cc:515] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.238443 10343 leader_election.cc:290] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.239382 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:53.239684 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981e7a9122eb40428f69c523a7cccb73" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:53.240063 10292 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.240178 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.240450 10295 raft_consensus.cc:3060] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.246706 10341 raft_consensus.cc:359] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.247756 10341 raft_consensus.cc:385] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.248134 10341 raft_consensus.cc:740] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.249763 10357 leader_election.cc:290] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.249163 10341 consensus_queue.cc:260] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.251183 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.252075 10146 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 0.
I20260709 12:10:53.252899 10341 ts_tablet_manager.cc:1434] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.043s	user 0.013s	sys 0.002s
I20260709 12:10:53.253674 10295 raft_consensus.cc:2468] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 1.
I20260709 12:10:53.256883 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981e7a9122eb40428f69c523a7cccb73" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
W20260709 12:10:53.257546 10185 leader_election.cc:343] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.257979 10187 leader_election.cc:304] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.254189 10341 tablet_bootstrap.cc:492] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:53.255338 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:53.255264 10339 raft_consensus.cc:359] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.259322 10357 raft_consensus.cc:2804] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.262001 10292 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 1.
I20260709 12:10:53.262032 10357 raft_consensus.cc:697] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:53.263041 10339 raft_consensus.cc:385] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:10:53.259083 10185 leader_election.cc:343] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.264603 10339 raft_consensus.cc:740] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.265069 10357 consensus_queue.cc:237] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.266019 10339 consensus_queue.cc:260] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.266149 10042 catalog_manager.cc:5697] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:53.269080 10339 ts_tablet_manager.cc:1434] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.003s
I20260709 12:10:53.271363 10339 tablet_bootstrap.cc:492] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:53.271662 10187 leader_election.cc:304] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.267541 10185 leader_election.cc:304] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:53.273363 10343 raft_consensus.cc:2804] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.277232 10338 tablet_bootstrap.cc:492] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:53.278874 10338 ts_tablet_manager.cc:1403] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.154s	user 0.019s	sys 0.001s
I20260709 12:10:53.295921 10343 raft_consensus.cc:697] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:53.295632 10357 raft_consensus.cc:2804] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.301998 10357 raft_consensus.cc:493] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.298760 10343 consensus_queue.cc:237] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.302345 10357 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:10:53.308029 10185 leader_election.cc:343] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.304912 10338 raft_consensus.cc:359] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.317890 10338 raft_consensus.cc:385] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.318231 10338 raft_consensus.cc:740] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.319051 10338 consensus_queue.cc:260] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
W20260709 12:10:53.322561 10185 leader_election.cc:343] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.313184 10357 raft_consensus.cc:515] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.315723 10341 tablet_bootstrap.cc:654] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.316955 10339 tablet_bootstrap.cc:654] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.331720 10043 catalog_manager.cc:5697] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f reported cstate change: term changed from 0 to 1, leader changed from <none> to e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195). New cstate: current_term: 1 leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:53.336323 10357 leader_election.cc:290] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.337558 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:53.338397 10146 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.341781 10338 ts_tablet_manager.cc:1434] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.003s
I20260709 12:10:53.341658 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
W20260709 12:10:53.345062 10187 leader_election.cc:343] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.329438 10040 catalog_manager.cc:5697] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:53.343199 10338 tablet_bootstrap.cc:492] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:53.357280 10146 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 1.
I20260709 12:10:53.359422 10185 leader_election.cc:304] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:53.360532 10357 raft_consensus.cc:2804] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.361002 10357 raft_consensus.cc:697] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:53.361827 10357 consensus_queue.cc:237] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.366524 10339 tablet_bootstrap.cc:492] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:53.367024 10339 ts_tablet_manager.cc:1403] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.096s	user 0.018s	sys 0.013s
I20260709 12:10:53.371140 10338 tablet_bootstrap.cc:654] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.372376 10341 tablet_bootstrap.cc:492] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:53.374300 10341 ts_tablet_manager.cc:1403] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.120s	user 0.030s	sys 0.008s
I20260709 12:10:53.372602 10357 raft_consensus.cc:493] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.374538 10339 raft_consensus.cc:359] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.375540 10339 raft_consensus.cc:385] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.375291 10357 raft_consensus.cc:515] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.375953 10339 raft_consensus.cc:740] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.376936 10339 consensus_queue.cc:260] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.377655 10341 raft_consensus.cc:359] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.378387 10341 raft_consensus.cc:385] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.378717 10341 raft_consensus.cc:740] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.379582 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.380273 10339 ts_tablet_manager.cc:1434] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:10:53.381547 10339 tablet_bootstrap.cc:492] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:53.382407 10341 consensus_queue.cc:260] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.384966 10341 ts_tablet_manager.cc:1434] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:10:53.388605 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.388391 10040 catalog_manager.cc:5697] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:53.390393 10339 tablet_bootstrap.cc:654] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.387692 10357 leader_election.cc:290] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.391750 10341 tablet_bootstrap.cc:492] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
W20260709 12:10:53.391844 10187 leader_election.cc:343] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:10:53.394780 10185 leader_election.cc:343] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.395628 10185 leader_election.cc:304] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:53.398772 10347 raft_consensus.cc:493] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.399309 10347 raft_consensus.cc:515] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.402031 10347 leader_election.cc:290] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.402455 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79f1ea9709e74f4e950a9becd454ab45" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.403725 10341 tablet_bootstrap.cc:654] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
W20260709 12:10:53.404086 10111 leader_election.cc:343] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.404883 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79f1ea9709e74f4e950a9becd454ab45" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
W20260709 12:10:53.407191 10112 leader_election.cc:343] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:10:53.407882 10112 leader_election.cc:304] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4; no voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:53.409469 10347 raft_consensus.cc:2749] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:10:53.409451 10357 raft_consensus.cc:2749] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:10:53.414988 10339 tablet_bootstrap.cc:492] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:53.415482 10339 ts_tablet_manager.cc:1403] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.000s
I20260709 12:10:53.418460 10338 tablet_bootstrap.cc:492] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:53.418370 10339 raft_consensus.cc:359] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.419075 10339 raft_consensus.cc:385] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.419183 10338 ts_tablet_manager.cc:1403] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.019s
I20260709 12:10:53.419409 10339 raft_consensus.cc:740] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.420295 10339 consensus_queue.cc:260] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.422559 10339 ts_tablet_manager.cc:1434] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:10:53.422554 10338 raft_consensus.cc:359] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.423465 10338 raft_consensus.cc:385] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.424022 10339 tablet_bootstrap.cc:492] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0: Bootstrap starting.
I20260709 12:10:53.423946 10338 raft_consensus.cc:740] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.424918 10338 consensus_queue.cc:260] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.427845 10338 ts_tablet_manager.cc:1434] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:10:53.429431 10338 tablet_bootstrap.cc:492] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:53.428761 10040 catalog_manager.cc:5697] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:53.431180 10339 tablet_bootstrap.cc:654] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.438498 10338 tablet_bootstrap.cc:654] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.442780 10339 tablet_bootstrap.cc:492] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0: No bootstrap required, opened a new log
I20260709 12:10:53.443324 10339 ts_tablet_manager.cc:1403] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:10:53.445995 10339 raft_consensus.cc:359] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.446714 10339 raft_consensus.cc:385] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.447013 10339 raft_consensus.cc:740] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.448137 10339 consensus_queue.cc:260] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.451357 10339 ts_tablet_manager.cc:1434] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:10:53.455454 10338 tablet_bootstrap.cc:492] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:53.456017 10338 ts_tablet_manager.cc:1403] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.012s
I20260709 12:10:53.458657 10338 raft_consensus.cc:359] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.459271 10341 tablet_bootstrap.cc:492] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:53.459617 10338 raft_consensus.cc:385] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.459782 10341 ts_tablet_manager.cc:1403] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.073s	user 0.025s	sys 0.016s
I20260709 12:10:53.460058 10338 raft_consensus.cc:740] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.461022 10338 consensus_queue.cc:260] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.462882 10338 ts_tablet_manager.cc:1434] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:10:53.463822 10338 tablet_bootstrap.cc:492] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f: Bootstrap starting.
I20260709 12:10:53.464882 10341 raft_consensus.cc:359] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.465531 10341 raft_consensus.cc:385] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.465829 10341 raft_consensus.cc:740] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.466454 10341 consensus_queue.cc:260] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.470527 10338 tablet_bootstrap.cc:654] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.474000 10341 ts_tablet_manager.cc:1434] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:10:53.475235 10341 tablet_bootstrap.cc:492] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4: Bootstrap starting.
I20260709 12:10:53.481603 10341 tablet_bootstrap.cc:654] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4: Neither blocks nor log segments found. Creating new log.
I20260709 12:10:53.482244 10338 tablet_bootstrap.cc:492] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f: No bootstrap required, opened a new log
I20260709 12:10:53.482704 10338 ts_tablet_manager.cc:1403] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.013s
I20260709 12:10:53.486023 10338 raft_consensus.cc:359] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.486754 10338 raft_consensus.cc:385] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.487017 10338 raft_consensus.cc:740] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.487689 10338 consensus_queue.cc:260] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.489164 10338 ts_tablet_manager.cc:1434] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.001s
I20260709 12:10:53.488394 10341 tablet_bootstrap.cc:492] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4: No bootstrap required, opened a new log
I20260709 12:10:53.494050 10341 ts_tablet_manager.cc:1403] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4: Time spent bootstrapping tablet: real 0.019s	user 0.003s	sys 0.008s
I20260709 12:10:53.495923 10341 raft_consensus.cc:359] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.496421 10341 raft_consensus.cc:385] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:10:53.496640 10341 raft_consensus.cc:740] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Initialized, Role: FOLLOWER
I20260709 12:10:53.497211 10341 consensus_queue.cc:260] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.498912 10341 ts_tablet_manager.cc:1434] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:10:53.521974 10371 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.522184 10357 raft_consensus.cc:493] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.523459 10357 raft_consensus.cc:515] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.525331 10357 leader_election.cc:290] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.526625 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.526788 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.527405 10146 raft_consensus.cc:2393] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8813d89d8ef84ebc98d65964ecbad4f0 in current term 1: Already voted for candidate e75ba39bed984d918b3d9d01a1b37f7f in this term.
I20260709 12:10:53.528820 10185 leader_election.cc:304] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:53.529816 10357 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.532320 10343 raft_consensus.cc:493] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.532759 10343 raft_consensus.cc:515] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.534893 10343 leader_election.cc:290] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.535643 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.536257 10146 raft_consensus.cc:2468] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 0.
I20260709 12:10:53.536984 10357 raft_consensus.cc:2749] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:10:53.536278 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.537674 10292 raft_consensus.cc:2468] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 0.
I20260709 12:10:53.539058 10187 leader_election.cc:304] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.539835 10357 raft_consensus.cc:2804] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.540160 10357 raft_consensus.cc:493] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.540482 10357 raft_consensus.cc:3060] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.546836 10357 raft_consensus.cc:515] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.549582 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:53.550146 10146 raft_consensus.cc:3060] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.553458 10371 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:10:53.556615 10146 raft_consensus.cc:2468] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 1.
I20260709 12:10:53.558413 10185 leader_election.cc:304] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:53.559243 10343 raft_consensus.cc:2804] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.559684 10343 raft_consensus.cc:697] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:53.560453 10343 consensus_queue.cc:237] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.570008 10371 raft_consensus.cc:493] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.570658 10371 raft_consensus.cc:515] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.576318 10371 leader_election.cc:290] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:53.577132 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79f1ea9709e74f4e950a9becd454ab45" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.577524 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79f1ea9709e74f4e950a9becd454ab45" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.577842 10146 raft_consensus.cc:2468] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 0.
I20260709 12:10:53.578156 10220 raft_consensus.cc:2468] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 0.
I20260709 12:10:53.579006 10259 leader_election.cc:304] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.579823 10371 raft_consensus.cc:2804] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.580317 10371 raft_consensus.cc:493] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.580756 10371 raft_consensus.cc:3060] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.582258 10040 catalog_manager.cc:5697] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:53.589864 10347 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.592689 10371 raft_consensus.cc:515] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.596719 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:53.597285 10295 raft_consensus.cc:3060] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.604239 10295 raft_consensus.cc:2468] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 1.
I20260709 12:10:53.605676 10357 leader_election.cc:290] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.612994 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79f1ea9709e74f4e950a9becd454ab45" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:53.613616 10145 raft_consensus.cc:3060] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.624608 10145 raft_consensus.cc:2468] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 1.
I20260709 12:10:53.625645 10371 leader_election.cc:290] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:53.626751 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79f1ea9709e74f4e950a9becd454ab45" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:53.627825 10373 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.628228 10220 raft_consensus.cc:3060] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.630023 10259 leader_election.cc:304] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.631974 10371 raft_consensus.cc:2804] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.633580 10371 raft_consensus.cc:697] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Becoming Leader. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:10:53.634783 10371 consensus_queue.cc:237] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.639151 10220 raft_consensus.cc:2468] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 1.
I20260709 12:10:53.648241 10373 raft_consensus.cc:493] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.648741 10373 raft_consensus.cc:515] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.651203 10357 consensus_queue.cc:1048] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.652318 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.652711 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.657076 10292 raft_consensus.cc:2393] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2542817e0dd345b3a0fb40a129e92cd4 in current term 1: Already voted for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in this term.
I20260709 12:10:53.650915 10373 leader_election.cc:290] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.658267 10112 leader_election.cc:304] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4; no voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:53.659389 10373 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:10:53.661145 10091 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:10:53.659868 10042 catalog_manager.cc:5697] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f reported cstate change: term changed from 0 to 1, leader changed from <none> to e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195). New cstate: current_term: 1 leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:53.679102 10373 raft_consensus.cc:2749] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:10:53.683897 10145 raft_consensus.cc:3060] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.694509 10383 consensus_queue.cc:1048] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:10:53.731873 10383 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.743552 10343 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.753445 10371 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:10:53.769043 10383 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.772574 10370 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:10:53.776803 10373 raft_consensus.cc:493] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.777397 10373 raft_consensus.cc:515] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.780499 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.781219 10220 raft_consensus.cc:2468] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 0.
I20260709 12:10:53.782609 10111 leader_election.cc:304] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:53.783437 10347 raft_consensus.cc:2804] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:10:53.783854 10347 raft_consensus.cc:493] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.783989 10370 raft_consensus.cc:493] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.784260 10347 raft_consensus.cc:3060] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.784482 10370 raft_consensus.cc:515] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:53.787784 10370 leader_election.cc:290] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:53.788903 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" is_pre_election: true
I20260709 12:10:53.792189 10347 raft_consensus.cc:515] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.793069 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4" is_pre_election: true
I20260709 12:10:53.795724 10259 leader_election.cc:304] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e75ba39bed984d918b3d9d01a1b37f7f; no voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:10:53.797479 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:53.798008 10347 leader_election.cc:290] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.798247 10295 raft_consensus.cc:3060] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.802135 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:53.802838 10220 raft_consensus.cc:3060] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.805578 10373 leader_election.cc:290] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:53.807139 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" is_pre_election: true
I20260709 12:10:53.810319 10371 raft_consensus.cc:2749] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:10:53.811107 10292 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:10:53.819290 10357 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:53.823571 10295 raft_consensus.cc:2468] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 1.
I20260709 12:10:53.824934 10112 leader_election.cc:304] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:53.829203 10373 raft_consensus.cc:2804] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:10:53.829983 10373 raft_consensus.cc:697] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:53.830888 10373 consensus_queue.cc:237] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:53.837430 10220 raft_consensus.cc:2468] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 1.
I20260709 12:10:53.846403 10040 catalog_manager.cc:5697] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:53.915540 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f83ee2731302482d8edfac1392b77842"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:44900
I20260709 12:10:53.916344 10221 raft_consensus.cc:606] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Received request to transfer leadership to TS 2542817e0dd345b3a0fb40a129e92cd4
I20260709 12:10:53.930665 10295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
mode: GRACEFUL
new_leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:10:53.931206 10295 raft_consensus.cc:606] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Received request to transfer leadership to TS 2542817e0dd345b3a0fb40a129e92cd4
I20260709 12:10:53.936957 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:44900
I20260709 12:10:53.937513 10221 raft_consensus.cc:606] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Received request to transfer leadership to TS 2542817e0dd345b3a0fb40a129e92cd4
I20260709 12:10:53.942965 10295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
mode: GRACEFUL
new_leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:10:53.943471 10295 raft_consensus.cc:606] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Received request to transfer leadership to TS 2542817e0dd345b3a0fb40a129e92cd4
I20260709 12:10:53.960886 10146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
mode: GRACEFUL
new_leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:60350
I20260709 12:10:53.961567 10146 raft_consensus.cc:606] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Received request to transfer leadership to TS 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:10:53.967650 10146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
mode: GRACEFUL
new_leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:60350
I20260709 12:10:53.968173 10146 raft_consensus.cc:606] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Received request to transfer leadership to TS 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:10:53.994881 10357 consensus_queue.cc:1048] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:54.007099 10357 consensus_queue.cc:1048] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:54.038898 10347 raft_consensus.cc:993] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4: : Instructing follower 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:10:54.039393 10347 raft_consensus.cc:1081] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Signalling peer 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:10:54.041154 10221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44890
I20260709 12:10:54.041750 10221 raft_consensus.cc:493] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:54.042137 10221 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.048724 10221 raft_consensus.cc:515] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.051529 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:54.052071 10221 leader_election.cc:290] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 2 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:54.052153 10146 raft_consensus.cc:3055] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:54.052580 10146 raft_consensus.cc:740] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:54.053337 10146 consensus_queue.cc:260] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.053699 10370 consensus_queue.cc:1048] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:54.053975 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:54.054533 10146 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.054709 10295 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.061420 10146 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 2.
I20260709 12:10:54.062633 10185 leader_election.cc:304] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:54.063149 10345 raft_consensus.cc:993] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f: : Instructing follower 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.063699 10345 raft_consensus.cc:1081] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Signalling peer 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.065485 10146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:60338
I20260709 12:10:54.066078 10146 raft_consensus.cc:493] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:54.066437 10146 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.067138 10357 raft_consensus.cc:2804] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:54.067639 10357 raft_consensus.cc:697] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:54.068467 10357 consensus_queue.cc:237] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.075693 10370 consensus_queue.cc:1048] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:54.078873 10043 catalog_manager.cc:5697] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 1 to 2, leader changed from 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193) to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:54.080009 10295 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 2.
I20260709 12:10:54.084784 10146 raft_consensus.cc:515] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.091636 10146 leader_election.cc:290] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:54.093176 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:54.093796 10221 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.094018 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:54.094574 10295 raft_consensus.cc:3055] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:54.094889 10295 raft_consensus.cc:740] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:10:54.095767 10295 consensus_queue.cc:260] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.098340 10295 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.100383 10221 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.101601 10111 leader_election.cc:304] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:54.102530 10347 raft_consensus.cc:2804] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:54.102977 10347 raft_consensus.cc:697] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:54.103744 10347 consensus_queue.cc:237] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.105470 10295 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.111321 10042 catalog_manager.cc:5697] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 1 to 2, leader changed from e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195) to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:54.242373 10343 raft_consensus.cc:993] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.242851 10357 raft_consensus.cc:1081] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Signalling peer 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.244468 10146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:60344
I20260709 12:10:54.245028 10146 raft_consensus.cc:493] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:54.245414 10146 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.252554 10146 raft_consensus.cc:515] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.254793 10146 leader_election.cc:290] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:54.255585 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:54.255766 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:54.256320 10221 raft_consensus.cc:3055] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:54.256340 10295 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.256901 10221 raft_consensus.cc:740] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:54.257798 10221 consensus_queue.cc:260] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.259323 10221 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.260109 10357 raft_consensus.cc:993] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.260667 10383 raft_consensus.cc:1081] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Signalling peer 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.262418 10146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f83ee2731302482d8edfac1392b77842"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:60344
I20260709 12:10:54.263051 10146 raft_consensus.cc:493] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:54.263489 10146 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.265286 10295 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.265923 10221 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.266734 10112 leader_election.cc:304] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:54.267584 10347 raft_consensus.cc:2804] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:54.268070 10347 raft_consensus.cc:697] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:54.268922 10347 consensus_queue.cc:237] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.269903 10146 raft_consensus.cc:515] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:54.272673 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:54.273203 10221 raft_consensus.cc:3055] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:54.273537 10221 raft_consensus.cc:740] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:54.273761 10146 leader_election.cc:290] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Requested vote from peers e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:54.274361 10221 consensus_queue.cc:260] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:54.275449 10221 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.272354 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:54.276345 10295 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.277778 10042 catalog_manager.cc:5697] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 1 to 2, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:54.281814 10221 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.282507 10295 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.282922 10111 leader_election.cc:304] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:54.284057 10347 raft_consensus.cc:2804] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:54.284467 10347 raft_consensus.cc:697] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:54.285122 10347 consensus_queue.cc:237] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:54.291705 10042 catalog_manager.cc:5697] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 1 to 2, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:54.317426 10347 consensus_queue.cc:1048] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:54.324746 10370 raft_consensus.cc:993] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f: : Instructing follower 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.325183 10370 raft_consensus.cc:1081] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Signalling peer 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:10:54.328876 10146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:60338
I20260709 12:10:54.328940 10420 consensus_queue.cc:1048] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:10:54.329491 10146 raft_consensus.cc:493] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:54.329914 10146 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.340408 10146 raft_consensus.cc:515] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.343619 10146 leader_election.cc:290] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:54.343573 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:54.344684 10220 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.345041 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:54.345749 10295 raft_consensus.cc:3055] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:54.346096 10295 raft_consensus.cc:740] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:10:54.346933 10295 consensus_queue.cc:260] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.348168 10295 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.350828 10420 raft_consensus.cc:993] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4: : Instructing follower 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:10:54.351255 10420 raft_consensus.cc:1081] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Signalling peer 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:10:54.354441 10220 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.354873 10295 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:10:54.355605 10111 leader_election.cc:304] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:10:54.356422 10373 raft_consensus.cc:2804] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:54.356884 10373 raft_consensus.cc:697] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:54.357796 10373 consensus_queue.cc:237] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.356846 10221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44890
I20260709 12:10:54.359922 10221 raft_consensus.cc:493] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:54.360306 10221 raft_consensus.cc:3060] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.366353 10221 raft_consensus.cc:515] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.366672 10043 catalog_manager.cc:5697] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 1 to 2, leader changed from e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195) to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:54.368013 10221 leader_election.cc:290] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 2 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:10:54.369782 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:10:54.369916 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd0dfb70d4c229823e441456f9291" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:54.370368 10295 raft_consensus.cc:3060] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.370632 10146 raft_consensus.cc:3055] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:54.371068 10146 raft_consensus.cc:740] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:54.372982 10146 consensus_queue.cc:260] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.374513 10146 raft_consensus.cc:3060] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:54.377882 10295 raft_consensus.cc:2468] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 2.
I20260709 12:10:54.379074 10187 leader_election.cc:304] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:54.379887 10383 raft_consensus.cc:2804] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:54.380359 10383 raft_consensus.cc:697] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:54.380640 10146 raft_consensus.cc:2468] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 2.
I20260709 12:10:54.381199 10383 consensus_queue.cc:237] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:54.389272 10043 catalog_manager.cc:5697] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 1 to 2, leader changed from 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193) to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:54.496721 10146 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.498499 10383 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.510104 10295 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.512481 10425 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.551337 10295 raft_consensus.cc:1275] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.552873 10373 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.562153 10221 raft_consensus.cc:1275] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.563769 10347 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.706765 10293 raft_consensus.cc:1275] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.708283 10347 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.716843 10221 raft_consensus.cc:1275] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.718458 10420 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.774809 10293 raft_consensus.cc:1275] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.776242 10373 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.785068 10221 raft_consensus.cc:1275] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.786661 10373 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.840374 10293 raft_consensus.cc:1275] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.841325 10221 raft_consensus.cc:1275] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.841915 10425 consensus_queue.cc:1048] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.842694 10373 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.850852 10146 raft_consensus.cc:1275] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.852451 10425 consensus_queue.cc:1048] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.856197 10293 raft_consensus.cc:1275] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:54.858093 10420 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:57.003584 10011 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:10:57.004386 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:57.023834 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:44902
I20260709 12:10:57.024408 10221 raft_consensus.cc:606] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 LEADER]: Received request to transfer leadership to TS e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:57.035748 10146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f83ee2731302482d8edfac1392b77842"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
mode: GRACEFUL
new_leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:10:57.036187 10146 raft_consensus.cc:606] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Received request to transfer leadership to TS e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:10:57.037235 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:10:57.037818 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:57.101533 10431 raft_consensus.cc:993] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4: : Instructing follower e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:10:57.102046 10431 raft_consensus.cc:1081] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Signalling peer e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:10:57.103523 10293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f83ee2731302482d8edfac1392b77842"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:44904
I20260709 12:10:57.103968 10293 raft_consensus.cc:493] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:57.104249 10293 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:57.108978 10293 raft_consensus.cc:515] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:57.110494 10293 leader_election.cc:290] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 3 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:57.111390 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:57.111671 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:57.111948 10146 raft_consensus.cc:3055] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:57.112197 10221 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:57.112265 10146 raft_consensus.cc:740] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:10:57.113035 10146 consensus_queue.cc:260] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:57.114136 10146 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:57.118805 10221 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 3.
I20260709 12:10:57.120131 10146 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 3.
I20260709 12:10:57.120055 10260 leader_election.cc:304] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:57.121013 10430 raft_consensus.cc:2804] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:57.122228 10430 raft_consensus.cc:697] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Becoming Leader. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:10:57.123024 10430 consensus_queue.cc:237] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:10:57.129889 10042 catalog_manager.cc:5697] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f reported cstate change: term changed from 2 to 3, leader changed from 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193) to e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195). New cstate: current_term: 3 leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:57.276500 10443 raft_consensus.cc:993] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:10:57.277026 10343 raft_consensus.cc:1081] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 LEADER]: Signalling peer e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:10:57.278570 10293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:44888
I20260709 12:10:57.279031 10293 raft_consensus.cc:493] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:57.279304 10293 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:57.285821 10293 raft_consensus.cc:515] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:57.287554 10293 leader_election.cc:290] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 3 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:10:57.288491 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:10:57.288617 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:10:57.289007 10146 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:57.289208 10221 raft_consensus.cc:3055] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:57.289551 10221 raft_consensus.cc:740] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:10:57.290308 10221 consensus_queue.cc:260] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:57.291225 10221 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:57.293479 10146 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 3.
I20260709 12:10:57.294523 10259 leader_election.cc:304] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:10:57.295336 10430 raft_consensus.cc:2804] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:57.295748 10430 raft_consensus.cc:697] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Becoming Leader. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:10:57.296365 10221 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 3.
I20260709 12:10:57.296564 10430 consensus_queue.cc:237] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:10:57.307436 10042 catalog_manager.cc:5697] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f reported cstate change: term changed from 2 to 3, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195). New cstate: current_term: 3 leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:57.666170 10221 raft_consensus.cc:1275] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:57.667483 10430 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:57.673719 10146 raft_consensus.cc:1275] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:57.675410 10430 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:57.892614 10146 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:57.893929 10437 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:57.902868 10221 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:57.904623 10452 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:58.038766 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:58.044968 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:10:58.045431 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:59.046383 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:59.051980 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:10:59.052495 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:00.053524 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:00.058737 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:00.059124 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:01.060148 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:01.065132 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:01.065498 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:02.066659 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:02.072016 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:02.072443 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:03.073477 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:03.078606 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:03.079089 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:04.080210 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:04.085826 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:04.086313 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:05.087296 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:05.092514 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:05.092994 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:06.094062 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:06.098721 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:06.099135 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:07.127699 10011 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:11:07.132310 10293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f83ee2731302482d8edfac1392b77842"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
mode: GRACEFUL
new_leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:11:07.133025 10293 raft_consensus.cc:606] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Received request to transfer leadership to TS 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:11:07.138402 10146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
mode: GRACEFUL
new_leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:60350
I20260709 12:11:07.138792 10146 raft_consensus.cc:606] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Received request to transfer leadership to TS 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:11:07.143440 10146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
mode: GRACEFUL
new_leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:60350
I20260709 12:11:07.143838 10146 raft_consensus.cc:606] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Received request to transfer leadership to TS 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:11:07.148231 10146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
mode: GRACEFUL
new_leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:60350
I20260709 12:11:07.148648 10146 raft_consensus.cc:606] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Received request to transfer leadership to TS 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:11:07.155489 10293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
mode: GRACEFUL
new_leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:11:07.155966 10293 raft_consensus.cc:606] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Received request to transfer leadership to TS 8813d89d8ef84ebc98d65964ecbad4f0
I20260709 12:11:07.198149 10465 raft_consensus.cc:993] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4: : Instructing follower 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.198839 10461 raft_consensus.cc:1081] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Signalling peer 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.200565 10221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44890
I20260709 12:11:07.201110 10221 raft_consensus.cc:493] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:07.201524 10221 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.206436 10221 raft_consensus.cc:515] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.208097 10221 leader_election.cc:290] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 3 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:07.208762 10146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:11:07.209080 10293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:07.209482 10146 raft_consensus.cc:3055] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:07.209548 10293 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.209868 10146 raft_consensus.cc:740] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:11:07.210682 10146 consensus_queue.cc:260] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.211848 10146 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.215880 10293 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 3.
I20260709 12:11:07.217133 10187 leader_election.cc:304] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:11:07.218108 10146 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 3.
I20260709 12:11:07.218254 10464 raft_consensus.cc:2804] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:07.219308 10464 raft_consensus.cc:697] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:07.220283 10464 consensus_queue.cc:237] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.226692 10463 raft_consensus.cc:993] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f: : Instructing follower 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.227231 10468 raft_consensus.cc:1081] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Signalling peer 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.228855 10221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f83ee2731302482d8edfac1392b77842"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44886
I20260709 12:11:07.228372 10042 catalog_manager.cc:5697] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 2 to 3, leader changed from 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193) to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:07.229311 10221 raft_consensus.cc:493] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:07.229676 10221 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:07.235613 10221 raft_consensus.cc:515] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:11:07.237247 10221 leader_election.cc:290] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 4 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:07.238229 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:11:07.238595 10293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f83ee2731302482d8edfac1392b77842" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:07.238925 10145 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:07.239187 10293 raft_consensus.cc:3055] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:07.239547 10293 raft_consensus.cc:740] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:11:07.240341 10293 consensus_queue.cc:260] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:11:07.241219 10293 raft_consensus.cc:3060] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:07.245522 10145 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 4.
I20260709 12:11:07.246690 10185 leader_election.cc:304] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:11:07.247274 10293 raft_consensus.cc:2468] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 4.
I20260709 12:11:07.247428 10464 raft_consensus.cc:2804] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:07.247941 10464 raft_consensus.cc:697] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:07.248894 10464 consensus_queue.cc:237] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } }
I20260709 12:11:07.256134 10042 catalog_manager.cc:5697] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 3 to 4, leader changed from e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195) to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 4 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:07.269330 10465 raft_consensus.cc:993] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4: : Instructing follower 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.269811 10461 raft_consensus.cc:1081] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Signalling peer 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.271198 10221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44890
I20260709 12:11:07.271610 10221 raft_consensus.cc:493] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:07.271842 10221 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.275673 10221 raft_consensus.cc:515] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.276949 10221 leader_election.cc:290] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 3 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:07.277899 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:11:07.278002 10293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:07.278472 10145 raft_consensus.cc:3055] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:07.278483 10293 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.278832 10145 raft_consensus.cc:740] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:11:07.279543 10145 consensus_queue.cc:260] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.280486 10145 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.282778 10293 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 3.
I20260709 12:11:07.283810 10187 leader_election.cc:304] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:11:07.284641 10464 raft_consensus.cc:2804] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:07.285066 10464 raft_consensus.cc:697] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:07.285876 10145 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 3.
I20260709 12:11:07.285954 10464 consensus_queue.cc:237] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.293818 10042 catalog_manager.cc:5697] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 2 to 3, leader changed from 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193) to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:07.420940 10463 raft_consensus.cc:993] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f: : Instructing follower 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.421407 10468 raft_consensus.cc:1081] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Signalling peer 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.423074 10221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44886
I20260709 12:11:07.423570 10221 raft_consensus.cc:493] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:07.423892 10221 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:07.428253 10221 raft_consensus.cc:515] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.429860 10221 leader_election.cc:290] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 4 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:07.430791 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:11:07.430955 10293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:07.431394 10145 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:07.431439 10293 raft_consensus.cc:3055] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:07.431895 10293 raft_consensus.cc:740] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:11:07.432662 10293 consensus_queue.cc:260] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.433746 10293 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:07.436318 10145 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 4.
I20260709 12:11:07.437346 10185 leader_election.cc:304] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:11:07.438112 10464 raft_consensus.cc:2804] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:07.438244 10293 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 4.
I20260709 12:11:07.438552 10464 raft_consensus.cc:697] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:07.439404 10464 consensus_queue.cc:237] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.446115 10042 catalog_manager.cc:5697] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 3 to 4, leader changed from e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195) to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 4 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:07.479740 10465 raft_consensus.cc:993] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4: : Instructing follower 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.480453 10461 raft_consensus.cc:1081] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Signalling peer 8813d89d8ef84ebc98d65964ecbad4f0 to start an election
I20260709 12:11:07.482014 10221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
 from {username='slave'} at 127.0.0.1:44890
I20260709 12:11:07.482491 10221 raft_consensus.cc:493] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:07.482764 10221 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.486830 10221 raft_consensus.cc:515] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.488206 10221 leader_election.cc:290] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 3 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:07.489130 10145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:11:07.489130 10293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:07.489910 10293 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.489880 10145 raft_consensus.cc:3055] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:07.490417 10145 raft_consensus.cc:740] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:11:07.491225 10145 consensus_queue.cc:260] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.492408 10145 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:07.496167 10293 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 3.
I20260709 12:11:07.497293 10187 leader_election.cc:304] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8813d89d8ef84ebc98d65964ecbad4f0, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:11:07.498121 10464 raft_consensus.cc:2804] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:07.498562 10464 raft_consensus.cc:697] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Becoming Leader. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:07.498875 10145 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8813d89d8ef84ebc98d65964ecbad4f0 in term 3.
I20260709 12:11:07.499418 10464 consensus_queue.cc:237] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:07.506193 10042 catalog_manager.cc:5697] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 reported cstate change: term changed from 2 to 3, leader changed from 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193) to 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:07.680231 10145 raft_consensus.cc:1275] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:07.681744 10466 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:07.694443 10293 raft_consensus.cc:1275] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:07.696377 10466 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:07.721580 10293 raft_consensus.cc:1275] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:07.722707 10466 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:07.732151 10145 raft_consensus.cc:1275] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:07.733845 10466 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:07.745496 10145 raft_consensus.cc:1275] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:07.746723 10466 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:07.757332 10293 raft_consensus.cc:1275] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:07.758616 10466 consensus_queue.cc:1048] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:08.113355 10295 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:08.113355 10292 raft_consensus.cc:1275] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:08.117059 10488 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:08.117185 10464 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:08.124277 10145 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:08.124716 10143 raft_consensus.cc:1275] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Refusing update from remote peer 8813d89d8ef84ebc98d65964ecbad4f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:08.128294 10464 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:08.130137 10464 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:11:09.091135 10316 debug-util.cc:398] Leaking SignalData structure 0x7b080016d6c0 after lost signal to thread 10014
W20260709 12:11:09.092152 10316 debug-util.cc:398] Leaking SignalData structure 0x7b080016dda0 after lost signal to thread 10077
W20260709 12:11:09.092978 10316 debug-util.cc:398] Leaking SignalData structure 0x7b080009e180 after lost signal to thread 10170
W20260709 12:11:09.093725 10316 debug-util.cc:398] Leaking SignalData structure 0x7b08001af9e0 after lost signal to thread 10245
W20260709 12:11:09.094429 10316 debug-util.cc:398] Leaking SignalData structure 0x7b08000bdc00 after lost signal to thread 10319
I20260709 12:11:10.428048 10011 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:11:10.429358 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:10.442008 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:44902
I20260709 12:11:10.442878 10221 raft_consensus.cc:606] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Received request to transfer leadership to TS 2542817e0dd345b3a0fb40a129e92cd4
I20260709 12:11:10.446245 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:44902
I20260709 12:11:10.446796 10221 raft_consensus.cc:606] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Received request to transfer leadership to TS e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:11:10.449334 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:44902
I20260709 12:11:10.449888 10221 raft_consensus.cc:606] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 LEADER]: Received request to transfer leadership to TS 2542817e0dd345b3a0fb40a129e92cd4
I20260709 12:11:10.452313 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:44902
I20260709 12:11:10.452811 10221 raft_consensus.cc:606] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Received request to transfer leadership to TS e75ba39bed984d918b3d9d01a1b37f7f
I20260709 12:11:10.455240 10221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9"
dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
mode: GRACEFUL
new_leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:44902
I20260709 12:11:10.455668 10221 raft_consensus.cc:606] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Received request to transfer leadership to TS 2542817e0dd345b3a0fb40a129e92cd4
I20260709 12:11:10.456719 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:10.457183 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:10.520875 10496 raft_consensus.cc:993] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:11:10.521504 10466 raft_consensus.cc:1081] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 LEADER]: Signalling peer 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:11:10.523386 10144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:60344
I20260709 12:11:10.524015 10144 raft_consensus.cc:493] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:10.524423 10144 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:10.606144 10144 raft_consensus.cc:515] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.609941 10144 leader_election.cc:290] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 5 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:10.611305 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:11:10.612313 10221 raft_consensus.cc:3055] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:10.612535 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d069286ea84159b88ed2322a9d2db0" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:10.612603 10221 raft_consensus.cc:740] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:10.613248 10295 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:10.613873 10221 consensus_queue.cc:260] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.615072 10221 raft_consensus.cc:3060] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:10.711485 10221 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 5.
I20260709 12:11:10.711485 10295 raft_consensus.cc:2468] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 5.
I20260709 12:11:10.713382 10111 leader_election.cc:304] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, 8813d89d8ef84ebc98d65964ecbad4f0; no voters: 
I20260709 12:11:10.715782 10499 raft_consensus.cc:2804] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:11:10.716995 10499 raft_consensus.cc:697] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 5 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:11:10.718086 10499 consensus_queue.cc:237] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.732080 10466 raft_consensus.cc:993] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:11:10.732699 10496 raft_consensus.cc:1081] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Signalling peer 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:11:10.732043 10043 catalog_manager.cc:5697] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 4 to 5, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 5 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:10.735149 10144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:60344
I20260709 12:11:10.735725 10144 raft_consensus.cc:493] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:10.736099 10144 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:10.746075 10496 raft_consensus.cc:993] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:11:10.746656 10496 raft_consensus.cc:1081] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Signalling peer e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:11:10.748699 10295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:44888
I20260709 12:11:10.749339 10295 raft_consensus.cc:493] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:10.749670 10295 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:10.809118 10144 raft_consensus.cc:515] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.809118 10295 raft_consensus.cc:515] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.811581 10144 leader_election.cc:290] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 4 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:10.813606 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:11:10.814201 10221 raft_consensus.cc:3055] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:10.814296 10295 leader_election.cc:290] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 4 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:11:10.814566 10221 raft_consensus.cc:740] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:10.815390 10221 consensus_queue.cc:260] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.815826 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:11:10.816457 10221 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:10.816480 10220 raft_consensus.cc:3055] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:10.817912 10144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23b4ead3e9474e02810ad3d030508b8e" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:11:10.818159 10220 raft_consensus.cc:740] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:10.818611 10144 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:10.819018 10220 consensus_queue.cc:260] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.819556 10295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee2f944f89c241908a02e20a71442ba9" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:10.820102 10295 raft_consensus.cc:3060] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:10.820492 10220 raft_consensus.cc:3060] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:10.858952 10466 raft_consensus.cc:993] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:11:10.859496 10466 raft_consensus.cc:1081] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Signalling peer 2542817e0dd345b3a0fb40a129e92cd4 to start an election
I20260709 12:11:10.861343 10143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39"
dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
 from {username='slave'} at 127.0.0.1:60344
I20260709 12:11:10.861992 10143 raft_consensus.cc:493] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:10.862389 10143 raft_consensus.cc:3060] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:10.912962 10295 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 4.
I20260709 12:11:10.914634 10112 leader_election.cc:304] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:11:10.915555 10499 raft_consensus.cc:2804] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:10.916035 10499 raft_consensus.cc:697] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:11:10.918978 10221 raft_consensus.cc:2468] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 4.
I20260709 12:11:10.916996 10499 consensus_queue.cc:237] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.919909 10144 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 4.
I20260709 12:11:10.921375 10259 leader_election.cc:304] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:11:10.922413 10497 raft_consensus.cc:2804] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:10.926301 10497 raft_consensus.cc:697] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 4 LEADER]: Becoming Leader. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:11:10.927349 10497 consensus_queue.cc:237] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:10.931807 10043 catalog_manager.cc:5697] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 3 to 4, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 4 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:10.937089 10042 catalog_manager.cc:5697] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f reported cstate change: term changed from 3 to 4, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195). New cstate: current_term: 4 leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:10.974929 10496 raft_consensus.cc:993] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0: : Instructing follower e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:11:10.975435 10466 raft_consensus.cc:1081] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Signalling peer e75ba39bed984d918b3d9d01a1b37f7f to start an election
I20260709 12:11:10.977015 10295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee"
dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
 from {username='slave'} at 127.0.0.1:44888
I20260709 12:11:10.977476 10295 raft_consensus.cc:493] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:10.977913 10295 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:11.015864 10220 raft_consensus.cc:2468] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 4.
I20260709 12:11:11.015969 10143 raft_consensus.cc:515] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:11.017679 10295 raft_consensus.cc:515] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:11.019724 10295 leader_election.cc:290] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 4 election: Requested vote from peers 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395), 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737)
I20260709 12:11:11.018471 10143 leader_election.cc:290] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Requested vote from peers 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194:34737), e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195:32981)
I20260709 12:11:11.020387 10220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:11:11.021418 10292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6bb5d638a2405ab5599d59dc4c8d39" candidate_uuid: "2542817e0dd345b3a0fb40a129e92cd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e75ba39bed984d918b3d9d01a1b37f7f"
I20260709 12:11:11.022024 10292 raft_consensus.cc:3060] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:11.022971 10221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8813d89d8ef84ebc98d65964ecbad4f0"
I20260709 12:11:11.023540 10221 raft_consensus.cc:3055] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:11.023623 10143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62f27d2b65514590b6679d2eb6828fee" candidate_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2542817e0dd345b3a0fb40a129e92cd4"
I20260709 12:11:11.023906 10221 raft_consensus.cc:740] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:11.024327 10143 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:11.024785 10221 consensus_queue.cc:260] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:11.026095 10221 raft_consensus.cc:3060] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:11.021579 10220 raft_consensus.cc:3055] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:11.031477 10220 raft_consensus.cc:740] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8813d89d8ef84ebc98d65964ecbad4f0, State: Running, Role: LEADER
I20260709 12:11:11.032421 10220 consensus_queue.cc:260] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:11.033614 10220 raft_consensus.cc:3060] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:11.105273 10219 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 5 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:11:11.106781 10499 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:11.119117 10143 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 4.
I20260709 12:11:11.120496 10259 leader_election.cc:304] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:11:11.123088 10295 raft_consensus.cc:1275] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 5 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:11:11.123523 10292 raft_consensus.cc:2468] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:11:11.125433 10497 raft_consensus.cc:2804] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:11.125950 10497 raft_consensus.cc:697] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 4 LEADER]: Becoming Leader. State: Replica: e75ba39bed984d918b3d9d01a1b37f7f, State: Running, Role: LEADER
I20260709 12:11:11.126331 10112 leader_election.cc:304] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2542817e0dd345b3a0fb40a129e92cd4, e75ba39bed984d918b3d9d01a1b37f7f; no voters: 
I20260709 12:11:11.126827 10497 consensus_queue.cc:237] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:11.127844 10499 consensus_queue.cc:1048] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:11.129222 10499 raft_consensus.cc:2804] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:11.129673 10499 raft_consensus.cc:697] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Becoming Leader. State: Replica: 2542817e0dd345b3a0fb40a129e92cd4, State: Running, Role: LEADER
I20260709 12:11:11.131160 10221 raft_consensus.cc:2468] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75ba39bed984d918b3d9d01a1b37f7f in term 4.
I20260709 12:11:11.130740 10499 consensus_queue.cc:237] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } }
I20260709 12:11:11.152964 10043 catalog_manager.cc:5697] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 reported cstate change: term changed from 1 to 2, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "2542817e0dd345b3a0fb40a129e92cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:11.161110 10042 catalog_manager.cc:5697] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f reported cstate change: term changed from 3 to 4, leader changed from 8813d89d8ef84ebc98d65964ecbad4f0 (127.9.198.194) to e75ba39bed984d918b3d9d01a1b37f7f (127.9.198.195). New cstate: current_term: 4 leader_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:11.213917 10220 raft_consensus.cc:2468] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2542817e0dd345b3a0fb40a129e92cd4 in term 2.
I20260709 12:11:11.394965 10292 raft_consensus.cc:1275] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:11.396626 10500 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:11.410465 10220 raft_consensus.cc:1275] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:11.412349 10511 consensus_queue.cc:1048] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:11.445567 10143 raft_consensus.cc:1275] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:11.447081 10497 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:11.459136 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:11.469278 10220 raft_consensus.cc:1275] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:11.469851 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:11.470494 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:11.471105 10506 consensus_queue.cc:1048] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:11.561700 10292 raft_consensus.cc:1275] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:11.563196 10511 consensus_queue.cc:1048] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e75ba39bed984d918b3d9d01a1b37f7f" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:11.574667 10220 raft_consensus.cc:1275] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Refusing update from remote peer 2542817e0dd345b3a0fb40a129e92cd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:11.576633 10511 consensus_queue.cc:1048] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:11:11.636318 10143 raft_consensus.cc:1275] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:11.637657 10506 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2542817e0dd345b3a0fb40a129e92cd4" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 38395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:11.647801 10220 raft_consensus.cc:1275] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Refusing update from remote peer e75ba39bed984d918b3d9d01a1b37f7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:11.649438 10497 consensus_queue.cc:1048] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8813d89d8ef84ebc98d65964ecbad4f0" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:12.471812 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:12.476790 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:12.477217 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:13.574342 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:13.585839 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:13.586519 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:14.594115 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:14.598915 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:14.599335 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:11:14.613559 10242 debug-util.cc:398] Leaking SignalData structure 0x7b080012d060 after lost signal to thread 10014
W20260709 12:11:14.614439 10242 debug-util.cc:398] Leaking SignalData structure 0x7b080012d740 after lost signal to thread 10077
W20260709 12:11:14.615248 10242 debug-util.cc:398] Leaking SignalData structure 0x7b08001f4b20 after lost signal to thread 10170
W20260709 12:11:14.615955 10242 debug-util.cc:398] Leaking SignalData structure 0x7b08001bec60 after lost signal to thread 10245
W20260709 12:11:14.616670 10242 debug-util.cc:398] Leaking SignalData structure 0x7b08000a2060 after lost signal to thread 10319
I20260709 12:11:15.600443 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:15.606333 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:15.606830 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:16.608079 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:16.616237 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:16.616802 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:17.618140 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:17.623528 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:17.623946 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:18.624985 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:18.630076 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:18.630510 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:19.631572 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:19.636994 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:19.637482 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:20.673357 10011 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:11:20.674279 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
I20260709 12:11:20.709623 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:11:20.710613 10011 tablet_replica.cc:333] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.711911 10011 raft_consensus.cc:2243] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.712684 10011 raft_consensus.cc:2272] T c1bcd0dfb70d4c229823e441456f9291 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.715119 10011 tablet_replica.cc:333] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.715565 10011 raft_consensus.cc:2243] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.715903 10011 raft_consensus.cc:2272] T 981e7a9122eb40428f69c523a7cccb73 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.717540 10011 tablet_replica.cc:333] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.718044 10011 raft_consensus.cc:2243] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.718449 10011 raft_consensus.cc:2272] T 62f27d2b65514590b6679d2eb6828fee P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.720053 10011 tablet_replica.cc:333] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.720506 10011 raft_consensus.cc:2243] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.720896 10011 raft_consensus.cc:2272] T 23b4ead3e9474e02810ad3d030508b8e P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.722496 10011 tablet_replica.cc:333] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.722949 10011 raft_consensus.cc:2243] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.723354 10011 raft_consensus.cc:2272] T f83ee2731302482d8edfac1392b77842 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.724815 10011 tablet_replica.cc:333] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.725237 10011 raft_consensus.cc:2243] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.725575 10011 raft_consensus.cc:2272] T 79f1ea9709e74f4e950a9becd454ab45 P 2542817e0dd345b3a0fb40a129e92cd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.727664 10011 tablet_replica.cc:333] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.728192 10011 raft_consensus.cc:2243] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 5 LEADER]: Raft consensus shutting down.
W20260709 12:11:20.728622 10185 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:11:20.729398 10011 raft_consensus.cc:2272] T 59d069286ea84159b88ed2322a9d2db0 P 2542817e0dd345b3a0fb40a129e92cd4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.731912 10011 tablet_replica.cc:333] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
W20260709 12:11:20.732066 10185 consensus_peers.cc:597] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 -> Peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Couldn't send request to peer 2542817e0dd345b3a0fb40a129e92cd4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:20.732564 10011 raft_consensus.cc:2243] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:11:20.733317 10011 raft_consensus.cc:2272] T ee2f944f89c241908a02e20a71442ba9 P 2542817e0dd345b3a0fb40a129e92cd4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.734841 10011 tablet_replica.cc:333] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4: stopping tablet replica
I20260709 12:11:20.735227 10011 raft_consensus.cc:2243] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:11:20.735903 10011 raft_consensus.cc:2272] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 2542817e0dd345b3a0fb40a129e92cd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.825817 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
I20260709 12:11:20.848913 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
I20260709 12:11:20.880324 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:11:20.881196 10011 tablet_replica.cc:333] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
W20260709 12:11:20.882133 10259 consensus_peers.cc:597] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f -> Peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Couldn't send request to peer 2542817e0dd345b3a0fb40a129e92cd4. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:38395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:20.882241 10011 raft_consensus.cc:2243] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.882858 10011 raft_consensus.cc:2272] T 62f27d2b65514590b6679d2eb6828fee P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.884828 10011 tablet_replica.cc:333] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.885422 10011 raft_consensus.cc:2243] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.885995 10011 raft_consensus.cc:2272] T 1d6bb5d638a2405ab5599d59dc4c8d39 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.887977 10011 tablet_replica.cc:333] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.888656 10011 raft_consensus.cc:2243] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:11:20.889639 10011 raft_consensus.cc:2272] T c1bcd0dfb70d4c229823e441456f9291 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.891563 10011 tablet_replica.cc:333] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.892243 10011 raft_consensus.cc:2243] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.892848 10011 raft_consensus.cc:2272] T ee2f944f89c241908a02e20a71442ba9 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.895195 10011 tablet_replica.cc:333] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.895857 10011 raft_consensus.cc:2243] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.896286 10011 raft_consensus.cc:2272] T 59d069286ea84159b88ed2322a9d2db0 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:11:20.896610 10185 consensus_peers.cc:597] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 -> Peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Couldn't send request to peer 2542817e0dd345b3a0fb40a129e92cd4. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:38395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:20.898104 10011 tablet_replica.cc:333] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.898679 10011 raft_consensus.cc:2243] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:11:20.899389 10011 raft_consensus.cc:2272] T 981e7a9122eb40428f69c523a7cccb73 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.900979 10011 tablet_replica.cc:333] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.901458 10011 raft_consensus.cc:2243] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.901880 10011 raft_consensus.cc:2272] T 79f1ea9709e74f4e950a9becd454ab45 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.903545 10011 tablet_replica.cc:333] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.903998 10011 raft_consensus.cc:2243] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:11:20.904727 10011 raft_consensus.cc:2272] T f83ee2731302482d8edfac1392b77842 P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:20.906435 10011 tablet_replica.cc:333] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0: stopping tablet replica
I20260709 12:11:20.906919 10011 raft_consensus.cc:2243] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:20.907315 10011 raft_consensus.cc:2272] T 23b4ead3e9474e02810ad3d030508b8e P 8813d89d8ef84ebc98d65964ecbad4f0 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:11:20.956713 10259 consensus_peers.cc:597] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f -> Peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Couldn't send request to peer 2542817e0dd345b3a0fb40a129e92cd4. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:38395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:20.961009 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
W20260709 12:11:20.973304 10259 consensus_peers.cc:597] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f -> Peer 2542817e0dd345b3a0fb40a129e92cd4 (127.9.198.193:38395): Couldn't send request to peer 2542817e0dd345b3a0fb40a129e92cd4. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:38395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:20.982115 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
I20260709 12:11:21.011132 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:11:21.011775 10011 tablet_replica.cc:333] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.012428 10011 raft_consensus.cc:2243] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:21.012941 10011 raft_consensus.cc:2272] T c1bcd0dfb70d4c229823e441456f9291 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.014729 10011 tablet_replica.cc:333] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.015249 10011 raft_consensus.cc:2243] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:21.015647 10011 raft_consensus.cc:2272] T 1d6bb5d638a2405ab5599d59dc4c8d39 P e75ba39bed984d918b3d9d01a1b37f7f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.017113 10011 tablet_replica.cc:333] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.017575 10011 raft_consensus.cc:2243] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:21.018079 10011 raft_consensus.cc:2272] T ee2f944f89c241908a02e20a71442ba9 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.019703 10011 tablet_replica.cc:333] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.020174 10011 raft_consensus.cc:2243] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:21.020574 10011 raft_consensus.cc:2272] T 59d069286ea84159b88ed2322a9d2db0 P e75ba39bed984d918b3d9d01a1b37f7f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.022104 10011 tablet_replica.cc:333] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.022583 10011 raft_consensus.cc:2243] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:21.022955 10011 raft_consensus.cc:2272] T 981e7a9122eb40428f69c523a7cccb73 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.024431 10011 tablet_replica.cc:333] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.024887 10011 raft_consensus.cc:2243] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:11:21.025595 10011 raft_consensus.cc:2272] T 79f1ea9709e74f4e950a9becd454ab45 P e75ba39bed984d918b3d9d01a1b37f7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.027050 10011 tablet_replica.cc:333] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.027518 10011 raft_consensus.cc:2243] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:21.027916 10011 raft_consensus.cc:2272] T f83ee2731302482d8edfac1392b77842 P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.029361 10011 tablet_replica.cc:333] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.029875 10011 raft_consensus.cc:2243] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:11:21.030640 10011 raft_consensus.cc:2272] T 23b4ead3e9474e02810ad3d030508b8e P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.032171 10011 tablet_replica.cc:333] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f: stopping tablet replica
I20260709 12:11:21.032629 10011 raft_consensus.cc:2243] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:11:21.033345 10011 raft_consensus.cc:2272] T 62f27d2b65514590b6679d2eb6828fee P e75ba39bed984d918b3d9d01a1b37f7f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.077332 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:11:21.095566 10011 master.cc:562] Master@127.9.198.254:44521 shutting down...
I20260709 12:11:21.300048 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:11:21.300788 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:21.301138 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4089ee2edc8b44b1b9824070cb041134: stopping tablet replica
I20260709 12:11:21.320924 10011 master.cc:584] Master@127.9.198.254:44521 shutdown complete.
2542817e0dd345b3a0fb40a129e92cd4  4
8813d89d8ef84ebc98d65964ecbad4f0  4
e75ba39bed984d918b3d9d01a1b37f7f  1
2542817e0dd345b3a0fb40a129e92cd4  3
8813d89d8ef84ebc98d65964ecbad4f0  3
e75ba39bed984d918b3d9d01a1b37f7f  3
2542817e0dd345b3a0fb40a129e92cd4  0
8813d89d8ef84ebc98d65964ecbad4f0  8
e75ba39bed984d918b3d9d01a1b37f7f  1
2542817e0dd345b3a0fb40a129e92cd4  3
8813d89d8ef84ebc98d65964ecbad4f0  3
e75ba39bed984d918b3d9d01a1b37f7f  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31219 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:11:21.362488 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:46581
I20260709 12:11:21.363619 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:21.369256 10543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:21.369343 10542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:21.370383 10545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:21.371677 10011 server_base.cc:1061] running on GCE node
I20260709 12:11:21.372656 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:21.372839 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:21.372953 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599081372942 us; error 0 us; skew 500 ppm
I20260709 12:11:21.373400 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:21.375846 10011 webserver.cc:533] Webserver started at http://127.9.198.254:39913/ using document root <none> and password file <none>
I20260709 12:11:21.376293 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:21.376448 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:21.376668 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:21.377854 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "90667bb7730e49e9b2a5e7d6817ac4f3"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.382205 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:11:21.385571 10550 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.386484 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:11:21.386864 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "90667bb7730e49e9b2a5e7d6817ac4f3"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.387185 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:21.400038 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:21.401338 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:21.440650 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:46581
I20260709 12:11:21.440842 10601 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:46581 every 8 connection(s)
I20260709 12:11:21.444623 10602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:21.454926 10602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3: Bootstrap starting.
I20260709 12:11:21.459628 10602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:21.464068 10602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3: No bootstrap required, opened a new log
I20260709 12:11:21.466352 10602 raft_consensus.cc:359] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" member_type: VOTER }
I20260709 12:11:21.466814 10602 raft_consensus.cc:385] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:21.466998 10602 raft_consensus.cc:740] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90667bb7730e49e9b2a5e7d6817ac4f3, State: Initialized, Role: FOLLOWER
I20260709 12:11:21.467589 10602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" member_type: VOTER }
I20260709 12:11:21.468040 10602 raft_consensus.cc:399] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:11:21.468252 10602 raft_consensus.cc:493] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:11:21.468462 10602 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:21.472450 10602 raft_consensus.cc:515] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" member_type: VOTER }
I20260709 12:11:21.472954 10602 leader_election.cc:304] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90667bb7730e49e9b2a5e7d6817ac4f3; no voters: 
I20260709 12:11:21.474090 10602 leader_election.cc:290] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:11:21.474534 10605 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:21.475996 10605 raft_consensus.cc:697] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 1 LEADER]: Becoming Leader. State: Replica: 90667bb7730e49e9b2a5e7d6817ac4f3, State: Running, Role: LEADER
I20260709 12:11:21.476764 10605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" member_type: VOTER }
I20260709 12:11:21.477028 10602 sys_catalog.cc:565] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:11:21.482766 10607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90667bb7730e49e9b2a5e7d6817ac4f3. Latest consensus state: current_term: 1 leader_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" member_type: VOTER } }
I20260709 12:11:21.483399 10607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:21.485140 10606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90667bb7730e49e9b2a5e7d6817ac4f3" member_type: VOTER } }
I20260709 12:11:21.485687 10614 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:11:21.486042 10606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:21.491712 10614 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:11:21.492906 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:11:21.501286 10614 catalog_manager.cc:1383] Generated new cluster ID: c231797d3d9740709cb189f3077d6dc9
I20260709 12:11:21.501600 10614 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:11:21.519119 10614 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:11:21.520397 10614 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:11:21.539162 10614 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3: Generated new TSK 0
I20260709 12:11:21.539719 10614 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:11:21.559453 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:21.566082 10623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:21.567221 10624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:21.570557 10011 server_base.cc:1061] running on GCE node
W20260709 12:11:21.570546 10626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:21.571987 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:21.572178 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:21.572305 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599081572294 us; error 0 us; skew 500 ppm
I20260709 12:11:21.572754 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:21.575034 10011 webserver.cc:533] Webserver started at http://127.9.198.193:35653/ using document root <none> and password file <none>
I20260709 12:11:21.575477 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:21.575630 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:21.575845 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:21.576995 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "5602bb38ed70475da6831345a8d07478"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.581959 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 12:11:21.585121 10631 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.586023 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:11:21.586268 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "5602bb38ed70475da6831345a8d07478"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.586514 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:21.600358 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:21.601552 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:21.603061 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:21.605412 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:21.605588 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.605911 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:21.606069 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.647140 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:39663
I20260709 12:11:21.647238 10693 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:39663 every 8 connection(s)
I20260709 12:11:21.651979 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:21.659273 10698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:21.660282 10699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:21.663497 10701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:21.664252 10011 server_base.cc:1061] running on GCE node
I20260709 12:11:21.665848 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:21.666082 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:21.666266 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599081666249 us; error 0 us; skew 500 ppm
I20260709 12:11:21.666783 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:21.669885 10011 webserver.cc:533] Webserver started at http://127.9.198.194:36023/ using document root <none> and password file <none>
I20260709 12:11:21.670521 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:21.670728 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:21.670729 10694 heartbeater.cc:344] Connected to a master server at 127.9.198.254:46581
I20260709 12:11:21.671154 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:21.671197 10694 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:21.672133 10694 heartbeater.cc:507] Master 127.9.198.254:46581 requested a full tablet report, sending...
I20260709 12:11:21.672792 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "d844c1ce274a4f1cadebcf807f750cca"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.674631 10567 ts_manager.cc:194] Registered new tserver with Master: 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663)
I20260709 12:11:21.677078 10567 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40034
I20260709 12:11:21.678979 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.005s
I20260709 12:11:21.683009 10706 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.683815 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:11:21.684116 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "d844c1ce274a4f1cadebcf807f750cca"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.684409 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:21.706374 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:21.707633 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:21.709147 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:21.711508 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:21.711747 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.711997 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:21.712177 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.752369 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:45737
I20260709 12:11:21.752512 10768 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:45737 every 8 connection(s)
I20260709 12:11:21.757273 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:21.767531 10773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:21.769193 10772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:21.772679 10775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:21.773286 10769 heartbeater.cc:344] Connected to a master server at 127.9.198.254:46581
I20260709 12:11:21.773685 10769 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:21.774163 10011 server_base.cc:1061] running on GCE node
I20260709 12:11:21.774605 10769 heartbeater.cc:507] Master 127.9.198.254:46581 requested a full tablet report, sending...
I20260709 12:11:21.775148 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:21.775406 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:21.775622 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599081775605 us; error 0 us; skew 500 ppm
I20260709 12:11:21.776262 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:21.776968 10567 ts_manager.cc:194] Registered new tserver with Master: d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:21.778874 10567 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40050
I20260709 12:11:21.779161 10011 webserver.cc:533] Webserver started at http://127.9.198.195:45549/ using document root <none> and password file <none>
I20260709 12:11:21.779804 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:21.780038 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:21.780349 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:21.781742 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.786145 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 12:11:21.789542 10780 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.790405 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:11:21.790715 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
format_stamp: "Formatted at 2026-07-09 12:11:21 on dist-test-slave-glnm"
I20260709 12:11:21.791009 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:21.802790 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:21.804008 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:21.805580 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:21.807817 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:21.808041 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.808281 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:21.808461 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:21.848443 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:36337
I20260709 12:11:21.848541 10842 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:36337 every 8 connection(s)
I20260709 12:11:21.862628 10843 heartbeater.cc:344] Connected to a master server at 127.9.198.254:46581
I20260709 12:11:21.863013 10843 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:21.863751 10843 heartbeater.cc:507] Master 127.9.198.254:46581 requested a full tablet report, sending...
I20260709 12:11:21.865831 10567 ts_manager.cc:194] Registered new tserver with Master: bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:21.866338 10011 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014818662s
I20260709 12:11:21.867379 10567 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40056
I20260709 12:11:21.894845 10567 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40066:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:11:21.897790 10567 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:11:21.977562 10659 tablet_service.cc:1511] Processing CreateTablet for tablet c5e45ab4fb5247e48fd8e36318e08482 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:11:21.979053 10659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5e45ab4fb5247e48fd8e36318e08482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:21.984409 10733 tablet_service.cc:1511] Processing CreateTablet for tablet 6dbc30b646ab4fdfae2f48b8ed780522 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:11:21.986284 10733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dbc30b646ab4fdfae2f48b8ed780522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:21.986980 10730 tablet_service.cc:1511] Processing CreateTablet for tablet 0a3b53d3dccf44eab7ff8addb3665c2c (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:11:21.988433 10730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a3b53d3dccf44eab7ff8addb3665c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:21.989476 10658 tablet_service.cc:1511] Processing CreateTablet for tablet 6dbc30b646ab4fdfae2f48b8ed780522 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:11:21.991146 10658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dbc30b646ab4fdfae2f48b8ed780522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:21.990856 10731 tablet_service.cc:1511] Processing CreateTablet for tablet 5499e27815cb4f10830507e5c42f495f (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:11:21.993112 10732 tablet_service.cc:1511] Processing CreateTablet for tablet 596715d7c34e4297b4cf0287d1ee5861 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:11:21.994489 10732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596715d7c34e4297b4cf0287d1ee5861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:21.999869 10657 tablet_service.cc:1511] Processing CreateTablet for tablet 596715d7c34e4297b4cf0287d1ee5861 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:11:22.000118 10734 tablet_service.cc:1511] Processing CreateTablet for tablet c5e45ab4fb5247e48fd8e36318e08482 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:11:22.001286 10657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596715d7c34e4297b4cf0287d1ee5861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.001433 10734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5e45ab4fb5247e48fd8e36318e08482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.005358 10731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5499e27815cb4f10830507e5c42f495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.013325 10729 tablet_service.cc:1511] Processing CreateTablet for tablet 7641bfe7012f419a9329c5d755bf70c5 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:11:22.014860 10729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7641bfe7012f419a9329c5d755bf70c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.015511 10656 tablet_service.cc:1511] Processing CreateTablet for tablet 5499e27815cb4f10830507e5c42f495f (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:11:22.016855 10656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5499e27815cb4f10830507e5c42f495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.018880 10728 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6ab93f5577474fa12fef3ef8f0fd24 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:11:22.020210 10728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6ab93f5577474fa12fef3ef8f0fd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.022886 10727 tablet_service.cc:1511] Processing CreateTablet for tablet 9d162003b66343d791f60ebb8c4f306f (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:11:22.024227 10727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d162003b66343d791f60ebb8c4f306f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.026849 10726 tablet_service.cc:1511] Processing CreateTablet for tablet c5dbb6539f5246788b4eeb3b75494213 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:11:22.028235 10726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5dbb6539f5246788b4eeb3b75494213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.027868 10655 tablet_service.cc:1511] Processing CreateTablet for tablet 0a3b53d3dccf44eab7ff8addb3665c2c (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:11:22.029263 10655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a3b53d3dccf44eab7ff8addb3665c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.030834 10725 tablet_service.cc:1511] Processing CreateTablet for tablet b29c191985ea4b53a1c1e6cde36f44cf (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:11:22.032200 10725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b29c191985ea4b53a1c1e6cde36f44cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.038873 10653 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6ab93f5577474fa12fef3ef8f0fd24 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:11:22.040304 10653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6ab93f5577474fa12fef3ef8f0fd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.042528 10654 tablet_service.cc:1511] Processing CreateTablet for tablet 7641bfe7012f419a9329c5d755bf70c5 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:11:22.044621 10654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7641bfe7012f419a9329c5d755bf70c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.046047 10652 tablet_service.cc:1511] Processing CreateTablet for tablet 9d162003b66343d791f60ebb8c4f306f (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:11:22.047385 10652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d162003b66343d791f60ebb8c4f306f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.046958 10808 tablet_service.cc:1511] Processing CreateTablet for tablet c5e45ab4fb5247e48fd8e36318e08482 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:11:22.048743 10808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5e45ab4fb5247e48fd8e36318e08482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.050905 10651 tablet_service.cc:1511] Processing CreateTablet for tablet c5dbb6539f5246788b4eeb3b75494213 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:11:22.052241 10651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5dbb6539f5246788b4eeb3b75494213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.054847 10650 tablet_service.cc:1511] Processing CreateTablet for tablet b29c191985ea4b53a1c1e6cde36f44cf (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:11:22.056116 10650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b29c191985ea4b53a1c1e6cde36f44cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.059976 10807 tablet_service.cc:1511] Processing CreateTablet for tablet 6dbc30b646ab4fdfae2f48b8ed780522 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:11:22.061805 10807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dbc30b646ab4fdfae2f48b8ed780522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.067584 10806 tablet_service.cc:1511] Processing CreateTablet for tablet 596715d7c34e4297b4cf0287d1ee5861 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:11:22.069061 10806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596715d7c34e4297b4cf0287d1ee5861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.070637 10800 tablet_service.cc:1511] Processing CreateTablet for tablet c5dbb6539f5246788b4eeb3b75494213 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:11:22.071012 10799 tablet_service.cc:1511] Processing CreateTablet for tablet b29c191985ea4b53a1c1e6cde36f44cf (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:11:22.073141 10805 tablet_service.cc:1511] Processing CreateTablet for tablet 5499e27815cb4f10830507e5c42f495f (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:11:22.077943 10803 tablet_service.cc:1511] Processing CreateTablet for tablet 7641bfe7012f419a9329c5d755bf70c5 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:11:22.078358 10802 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6ab93f5577474fa12fef3ef8f0fd24 (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:11:22.079563 10800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5dbb6539f5246788b4eeb3b75494213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.082633 10803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7641bfe7012f419a9329c5d755bf70c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.083648 10801 tablet_service.cc:1511] Processing CreateTablet for tablet 9d162003b66343d791f60ebb8c4f306f (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:11:22.088775 10804 tablet_service.cc:1511] Processing CreateTablet for tablet 0a3b53d3dccf44eab7ff8addb3665c2c (DEFAULT_TABLE table=test-workload [id=6c7f5109a69c4a318d49680b649c25b2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:11:22.097512 10805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5499e27815cb4f10830507e5c42f495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.098332 10799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b29c191985ea4b53a1c1e6cde36f44cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.102258 10801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d162003b66343d791f60ebb8c4f306f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.105500 10802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6ab93f5577474fa12fef3ef8f0fd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.154486 10858 tablet_bootstrap.cc:492] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.161410 10858 tablet_bootstrap.cc:654] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.161739 10804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a3b53d3dccf44eab7ff8addb3665c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:22.219820 10859 tablet_bootstrap.cc:492] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.221565 10860 tablet_bootstrap.cc:492] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.228755 10860 tablet_bootstrap.cc:654] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.233026 10858 tablet_bootstrap.cc:492] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.233578 10858 ts_tablet_manager.cc:1403] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.005s
I20260709 12:11:22.235430 10859 tablet_bootstrap.cc:654] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.236581 10858 raft_consensus.cc:359] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.237358 10858 raft_consensus.cc:385] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.237730 10858 raft_consensus.cc:740] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.238524 10858 consensus_queue.cc:260] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.239445 10860 tablet_bootstrap.cc:492] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.239950 10860 ts_tablet_manager.cc:1403] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.010s
I20260709 12:11:22.240702 10769 heartbeater.cc:499] Master 127.9.198.254:46581 was elected leader, sending a full tablet report...
I20260709 12:11:22.240885 10859 tablet_bootstrap.cc:492] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.241206 10858 ts_tablet_manager.cc:1434] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:11:22.241359 10859 ts_tablet_manager.cc:1403] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 12:11:22.242151 10858 tablet_bootstrap.cc:492] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.242861 10860 raft_consensus.cc:359] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.243543 10860 raft_consensus.cc:385] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.243834 10860 raft_consensus.cc:740] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.244201 10859 raft_consensus.cc:359] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.244550 10860 consensus_queue.cc:260] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.245016 10859 raft_consensus.cc:385] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.245325 10859 raft_consensus.cc:740] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.246337 10859 consensus_queue.cc:260] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.248569 10858 tablet_bootstrap.cc:654] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.253921 10694 heartbeater.cc:499] Master 127.9.198.254:46581 was elected leader, sending a full tablet report...
I20260709 12:11:22.253973 10860 ts_tablet_manager.cc:1434] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 12:11:22.255538 10860 tablet_bootstrap.cc:492] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.264432 10860 tablet_bootstrap.cc:654] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.266389 10858 tablet_bootstrap.cc:492] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.266913 10858 ts_tablet_manager.cc:1403] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.000s
I20260709 12:11:22.269804 10858 raft_consensus.cc:359] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.271616 10843 heartbeater.cc:499] Master 127.9.198.254:46581 was elected leader, sending a full tablet report...
I20260709 12:11:22.271946 10858 raft_consensus.cc:385] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.272336 10858 raft_consensus.cc:740] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.272804 10859 ts_tablet_manager.cc:1434] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.031s	user 0.021s	sys 0.008s
I20260709 12:11:22.273516 10864 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.273801 10859 tablet_bootstrap.cc:492] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.273983 10864 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.276695 10864 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.284792 10858 consensus_queue.cc:260] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.287772 10859 tablet_bootstrap.cc:654] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.290443 10858 ts_tablet_manager.cc:1434] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:11:22.293898 10860 tablet_bootstrap.cc:492] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.294035 10858 tablet_bootstrap.cc:492] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.294534 10860 ts_tablet_manager.cc:1403] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.007s
I20260709 12:11:22.297676 10860 raft_consensus.cc:359] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.298516 10860 raft_consensus.cc:385] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.298866 10860 raft_consensus.cc:740] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.299783 10860 consensus_queue.cc:260] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.302320 10860 ts_tablet_manager.cc:1434] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:11:22.303372 10860 tablet_bootstrap.cc:492] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.305500 10858 tablet_bootstrap.cc:654] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.310094 10860 tablet_bootstrap.cc:654] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.315320 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
W20260709 12:11:22.317231 10710 leader_election.cc:343] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.326056 10860 tablet_bootstrap.cc:492] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.326766 10860 ts_tablet_manager.cc:1403] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.010s
I20260709 12:11:22.330524 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.330796 10860 raft_consensus.cc:359] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.331414 10818 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 0.
I20260709 12:11:22.333098 10710 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 5602bb38ed70475da6831345a8d07478
I20260709 12:11:22.334276 10864 raft_consensus.cc:2804] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.334656 10864 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.334995 10864 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.335953 10858 tablet_bootstrap.cc:492] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.336427 10858 ts_tablet_manager.cc:1403] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.006s
I20260709 12:11:22.339375 10858 raft_consensus.cc:359] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.340112 10858 raft_consensus.cc:385] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.340427 10858 raft_consensus.cc:740] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.341214 10858 consensus_queue.cc:260] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.342609 10864 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.346154 10858 ts_tablet_manager.cc:1434] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.004s
I20260709 12:11:22.346905 10864 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.347175 10858 tablet_bootstrap.cc:492] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.342443 10860 raft_consensus.cc:385] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.346993 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:22.347597 10860 raft_consensus.cc:740] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.348565 10860 consensus_queue.cc:260] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.350821 10860 ts_tablet_manager.cc:1434] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
W20260709 12:11:22.348928 10710 leader_election.cc:343] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.353296 10865 raft_consensus.cc:493] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.353999 10858 tablet_bootstrap.cc:654] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.353766 10865 raft_consensus.cc:515] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.356127 10859 tablet_bootstrap.cc:492] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.356591 10859 ts_tablet_manager.cc:1403] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.083s	user 0.014s	sys 0.007s
I20260709 12:11:22.349565 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:22.358503 10818 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.359892 10860 tablet_bootstrap.cc:492] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.360221 10859 raft_consensus.cc:359] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.360953 10859 raft_consensus.cc:385] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.368680 10859 raft_consensus.cc:740] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.369710 10865 leader_election.cc:290] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.370334 10860 tablet_bootstrap.cc:654] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.370373 10859 consensus_queue.cc:260] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.377405 10866 raft_consensus.cc:493] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.377985 10866 raft_consensus.cc:515] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.380499 10866 leader_election.cc:290] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:22.390390 10859 ts_tablet_manager.cc:1434] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.004s
I20260709 12:11:22.398622 10859 tablet_bootstrap.cc:492] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.406071 10818 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 1.
I20260709 12:11:22.407868 10859 tablet_bootstrap.cc:654] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.407982 10710 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 5602bb38ed70475da6831345a8d07478
I20260709 12:11:22.409132 10864 raft_consensus.cc:2804] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:22.409615 10864 raft_consensus.cc:697] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Becoming Leader. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:22.409799 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b29c191985ea4b53a1c1e6cde36f44cf" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.410533 10864 consensus_queue.cc:237] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
W20260709 12:11:22.411391 10781 leader_election.cc:343] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.430251 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b29c191985ea4b53a1c1e6cde36f44cf" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.430991 10669 raft_consensus.cc:2468] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 0.
I20260709 12:11:22.432118 10860 tablet_bootstrap.cc:492] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.432605 10860 ts_tablet_manager.cc:1403] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.018s
I20260709 12:11:22.432452 10784 leader_election.cc:304] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:22.436641 10866 raft_consensus.cc:2804] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.436744 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.437063 10866 raft_consensus.cc:493] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.437454 10866 raft_consensus.cc:3060] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:22.438326 10635 leader_election.cc:343] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.439152 10858 tablet_bootstrap.cc:492] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.439642 10858 ts_tablet_manager.cc:1403] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.093s	user 0.023s	sys 0.015s
I20260709 12:11:22.440248 10564 catalog_manager.cc:5697] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca reported cstate change: term changed from 0 to 1, leader changed from <none> to d844c1ce274a4f1cadebcf807f750cca (127.9.198.194). New cstate: current_term: 1 leader_uuid: "d844c1ce274a4f1cadebcf807f750cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:22.443107 10858 raft_consensus.cc:359] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.443835 10858 raft_consensus.cc:385] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.444097 10859 tablet_bootstrap.cc:492] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.444162 10858 raft_consensus.cc:740] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.444661 10859 ts_tablet_manager.cc:1403] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.008s
I20260709 12:11:22.446208 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.447023 10860 raft_consensus.cc:359] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.447777 10860 raft_consensus.cc:385] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:11:22.447872 10632 leader_election.cc:343] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.448107 10860 raft_consensus.cc:740] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.448356 10632 leader_election.cc:304] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5602bb38ed70475da6831345a8d07478; no voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:22.448413 10859 raft_consensus.cc:359] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.449144 10859 raft_consensus.cc:385] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.448873 10860 consensus_queue.cc:260] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.445024 10858 consensus_queue.cc:260] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.449447 10859 raft_consensus.cc:740] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.449761 10865 raft_consensus.cc:2749] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:22.450369 10859 consensus_queue.cc:260] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.452016 10858 ts_tablet_manager.cc:1434] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:11:22.453465 10858 tablet_bootstrap.cc:492] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.462808 10866 raft_consensus.cc:515] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.466051 10866 leader_election.cc:290] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:22.466843 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b29c191985ea4b53a1c1e6cde36f44cf" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:22.467175 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b29c191985ea4b53a1c1e6cde36f44cf" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:22.467446 10669 raft_consensus.cc:3060] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.463405 10860 ts_tablet_manager.cc:1434] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.012s
W20260709 12:11:22.468616 10781 leader_election.cc:343] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.473533 10858 tablet_bootstrap.cc:654] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.484072 10669 raft_consensus.cc:2468] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 1.
I20260709 12:11:22.485803 10784 leader_election.cc:304] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:22.486994 10866 raft_consensus.cc:2804] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:22.487623 10866 raft_consensus.cc:697] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
W20260709 12:11:22.489404 10615 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:22.488600 10866 consensus_queue.cc:237] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.492512 10859 ts_tablet_manager.cc:1434] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.028s
I20260709 12:11:22.493757 10859 tablet_bootstrap.cc:492] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.494333 10860 tablet_bootstrap.cc:492] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.500332 10859 tablet_bootstrap.cc:654] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.503873 10860 tablet_bootstrap.cc:654] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.504451 10858 tablet_bootstrap.cc:492] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.505092 10858 ts_tablet_manager.cc:1403] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.004s
I20260709 12:11:22.509331 10858 raft_consensus.cc:359] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.510340 10858 raft_consensus.cc:385] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.510730 10858 raft_consensus.cc:740] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.512346 10858 consensus_queue.cc:260] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.514772 10858 ts_tablet_manager.cc:1434] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:11:22.515796 10858 tablet_bootstrap.cc:492] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.520715 10567 catalog_manager.cc:5697] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:22.522847 10858 tablet_bootstrap.cc:654] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.524219 10859 tablet_bootstrap.cc:492] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.524806 10859 ts_tablet_manager.cc:1403] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.004s
I20260709 12:11:22.528605 10859 raft_consensus.cc:359] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.529744 10859 raft_consensus.cc:385] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.530159 10859 raft_consensus.cc:740] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.531344 10859 consensus_queue.cc:260] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.534897 10859 ts_tablet_manager.cc:1434] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:11:22.536439 10859 tablet_bootstrap.cc:492] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.545059 10859 tablet_bootstrap.cc:654] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.545939 10864 raft_consensus.cc:493] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.546497 10864 raft_consensus.cc:515] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.547812 10860 tablet_bootstrap.cc:492] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.548301 10860 ts_tablet_manager.cc:1403] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.024s
I20260709 12:11:22.548789 10864 leader_election.cc:290] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.551244 10860 raft_consensus.cc:359] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.552011 10860 raft_consensus.cc:385] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.551797 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.552328 10860 raft_consensus.cc:740] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.553418 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
W20260709 12:11:22.555282 10710 leader_election.cc:343] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.553234 10860 consensus_queue.cc:260] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
W20260709 12:11:22.556317 10710 leader_election.cc:343] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.556887 10710 leader_election.cc:304] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d844c1ce274a4f1cadebcf807f750cca; no voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2
I20260709 12:11:22.557865 10860 ts_tablet_manager.cc:1434] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:11:22.558305 10864 raft_consensus.cc:2749] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:22.558848 10860 tablet_bootstrap.cc:492] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.566319 10860 tablet_bootstrap.cc:654] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.567931 10858 tablet_bootstrap.cc:492] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.568480 10858 ts_tablet_manager.cc:1403] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.012s
I20260709 12:11:22.568781 10859 tablet_bootstrap.cc:492] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.569348 10859 ts_tablet_manager.cc:1403] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.015s
I20260709 12:11:22.571728 10858 raft_consensus.cc:359] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.572521 10858 raft_consensus.cc:385] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.572840 10858 raft_consensus.cc:740] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.573621 10859 raft_consensus.cc:359] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.573647 10858 consensus_queue.cc:260] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.574856 10859 raft_consensus.cc:385] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.575434 10859 raft_consensus.cc:740] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.576900 10858 ts_tablet_manager.cc:1434] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:11:22.576607 10859 consensus_queue.cc:260] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.577952 10858 tablet_bootstrap.cc:492] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.580085 10859 ts_tablet_manager.cc:1434] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:11:22.581941 10859 tablet_bootstrap.cc:492] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.583400 10860 tablet_bootstrap.cc:492] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.583941 10860 ts_tablet_manager.cc:1403] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.009s
I20260709 12:11:22.584455 10858 tablet_bootstrap.cc:654] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.587251 10860 raft_consensus.cc:359] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.588090 10860 raft_consensus.cc:385] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.588490 10860 raft_consensus.cc:740] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.588512 10859 tablet_bootstrap.cc:654] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.589605 10860 consensus_queue.cc:260] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.591975 10860 ts_tablet_manager.cc:1434] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:11:22.592299 10884 raft_consensus.cc:493] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.593197 10860 tablet_bootstrap.cc:492] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.592810 10884 raft_consensus.cc:515] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.597149 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.598016 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.598973 10884 leader_election.cc:290] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.599007 10864 raft_consensus.cc:493] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:22.600103 10632 leader_election.cc:343] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.599757 10864 raft_consensus.cc:515] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.600780 10818 raft_consensus.cc:2468] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 0.
I20260709 12:11:22.602417 10864 leader_election.cc:290] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.600263 10860 tablet_bootstrap.cc:654] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.604040 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5499e27815cb4f10830507e5c42f495f" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.604753 10859 tablet_bootstrap.cc:492] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.605261 10859 ts_tablet_manager.cc:1403] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.002s
W20260709 12:11:22.605623 10710 leader_election.cc:343] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.607162 10635 leader_election.cc:304] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:22.607225 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5499e27815cb4f10830507e5c42f495f" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.609092 10859 raft_consensus.cc:359] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
W20260709 12:11:22.609933 10710 leader_election.cc:343] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.610117 10859 raft_consensus.cc:385] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.610438 10710 leader_election.cc:304] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d844c1ce274a4f1cadebcf807f750cca; no voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2
I20260709 12:11:22.610551 10859 raft_consensus.cc:740] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.611726 10864 raft_consensus.cc:2749] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:22.611862 10859 consensus_queue.cc:260] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.615272 10859 ts_tablet_manager.cc:1434] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:11:22.616595 10859 tablet_bootstrap.cc:492] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.617486 10884 raft_consensus.cc:2804] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.617884 10884 raft_consensus.cc:493] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.618223 10884 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.618798 10858 tablet_bootstrap.cc:492] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.619295 10858 ts_tablet_manager.cc:1403] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.016s
I20260709 12:11:22.622324 10858 raft_consensus.cc:359] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.623170 10858 raft_consensus.cc:385] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.623556 10858 raft_consensus.cc:740] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.624464 10858 consensus_queue.cc:260] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.625102 10859 tablet_bootstrap.cc:654] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.627199 10858 ts_tablet_manager.cc:1434] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 12:11:22.628275 10858 tablet_bootstrap.cc:492] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.634646 10858 tablet_bootstrap.cc:654] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.656559 10884 raft_consensus.cc:515] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.660161 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
W20260709 12:11:22.661990 10632 leader_election.cc:343] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:22.663950 10884 leader_election.cc:290] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.665460 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:22.666246 10818 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.667376 10859 tablet_bootstrap.cc:492] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.667865 10860 tablet_bootstrap.cc:492] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.667887 10859 ts_tablet_manager.cc:1403] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.025s
I20260709 12:11:22.668475 10860 ts_tablet_manager.cc:1403] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.075s	user 0.020s	sys 0.044s
I20260709 12:11:22.669607 10858 tablet_bootstrap.cc:492] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.670120 10858 ts_tablet_manager.cc:1403] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.042s	user 0.023s	sys 0.015s
I20260709 12:11:22.671082 10859 raft_consensus.cc:359] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.671335 10860 raft_consensus.cc:359] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.671846 10859 raft_consensus.cc:385] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.671996 10860 raft_consensus.cc:385] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.672180 10859 raft_consensus.cc:740] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.672322 10860 raft_consensus.cc:740] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.672660 10818 raft_consensus.cc:2468] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:22.673558 10864 raft_consensus.cc:493] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.673127 10859 consensus_queue.cc:260] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.673053 10860 consensus_queue.cc:260] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.672952 10858 raft_consensus.cc:359] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.674114 10635 leader_election.cc:304] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:22.674036 10864 raft_consensus.cc:515] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.674362 10858 raft_consensus.cc:385] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.675024 10858 raft_consensus.cc:740] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.675781 10860 ts_tablet_manager.cc:1434] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:11:22.676048 10864 leader_election.cc:290] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.677814 10859 ts_tablet_manager.cc:1434] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:11:22.675724 10858 consensus_queue.cc:260] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.678977 10859 tablet_bootstrap.cc:492] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.679327 10884 raft_consensus.cc:2804] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:22.679852 10884 raft_consensus.cc:697] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:22.679788 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b29c191985ea4b53a1c1e6cde36f44cf" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.680316 10860 tablet_bootstrap.cc:492] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.680684 10669 raft_consensus.cc:2393] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d844c1ce274a4f1cadebcf807f750cca in current term 1: Already voted for candidate bcb0169f1fa94e1096ecb4e038005ab2 in this term.
I20260709 12:11:22.681877 10884 consensus_queue.cc:237] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.682940 10817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b29c191985ea4b53a1c1e6cde36f44cf" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.680692 10858 ts_tablet_manager.cc:1434] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 12:11:22.686640 10858 tablet_bootstrap.cc:492] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.686681 10710 leader_election.cc:304] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d844c1ce274a4f1cadebcf807f750cca; no voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2
I20260709 12:11:22.687918 10864 raft_consensus.cc:3060] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.692638 10858 tablet_bootstrap.cc:654] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.693293 10859 tablet_bootstrap.cc:654] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.693662 10860 tablet_bootstrap.cc:654] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.696906 10864 raft_consensus.cc:2749] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:22.707924 10564 catalog_manager.cc:5697] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:22.709431 10858 tablet_bootstrap.cc:492] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.709909 10858 ts_tablet_manager.cc:1403] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.008s
I20260709 12:11:22.711102 10859 tablet_bootstrap.cc:492] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.711696 10859 ts_tablet_manager.cc:1403] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.004s
I20260709 12:11:22.712415 10858 raft_consensus.cc:359] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.713042 10858 raft_consensus.cc:385] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.713286 10858 raft_consensus.cc:740] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.714020 10858 consensus_queue.cc:260] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.716048 10858 ts_tablet_manager.cc:1434] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:11:22.717005 10858 tablet_bootstrap.cc:492] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca: Bootstrap starting.
I20260709 12:11:22.719321 10859 raft_consensus.cc:359] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.720214 10859 raft_consensus.cc:385] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.720568 10859 raft_consensus.cc:740] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.721336 10859 consensus_queue.cc:260] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.723043 10860 tablet_bootstrap.cc:492] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.723682 10859 ts_tablet_manager.cc:1434] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:11:22.723830 10860 ts_tablet_manager.cc:1403] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.009s
I20260709 12:11:22.724841 10858 tablet_bootstrap.cc:654] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.726743 10860 raft_consensus.cc:359] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.727535 10860 raft_consensus.cc:385] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.727855 10860 raft_consensus.cc:740] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.728674 10860 consensus_queue.cc:260] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.733944 10859 tablet_bootstrap.cc:492] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.736354 10860 ts_tablet_manager.cc:1434] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:11:22.737430 10860 tablet_bootstrap.cc:492] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.737509 10866 raft_consensus.cc:493] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.738703 10866 raft_consensus.cc:515] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.741192 10866 leader_election.cc:290] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:22.740643 10859 tablet_bootstrap.cc:654] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.742359 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.743067 10744 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 0.
I20260709 12:11:22.743793 10860 tablet_bootstrap.cc:654] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.741858 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.744458 10669 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 0.
I20260709 12:11:22.745829 10784 leader_election.cc:304] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:22.746619 10866 raft_consensus.cc:2804] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.746999 10866 raft_consensus.cc:493] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.747285 10866 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.749915 10858 tablet_bootstrap.cc:492] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca: No bootstrap required, opened a new log
I20260709 12:11:22.750515 10858 ts_tablet_manager.cc:1403] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.003s
I20260709 12:11:22.751175 10859 tablet_bootstrap.cc:492] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.751677 10859 ts_tablet_manager.cc:1403] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 12:11:22.753077 10866 raft_consensus.cc:515] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.754014 10858 raft_consensus.cc:359] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.754419 10859 raft_consensus.cc:359] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.754904 10858 raft_consensus.cc:385] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.755244 10859 raft_consensus.cc:385] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.755299 10858 raft_consensus.cc:740] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.755687 10859 raft_consensus.cc:740] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.755949 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:22.756520 10669 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.756592 10859 consensus_queue.cc:260] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.757941 10866 leader_election.cc:290] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:22.758903 10859 ts_tablet_manager.cc:1434] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:11:22.759058 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:22.759639 10744 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.759771 10860 tablet_bootstrap.cc:492] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.760061 10859 tablet_bootstrap.cc:492] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2: Bootstrap starting.
I20260709 12:11:22.760226 10860 ts_tablet_manager.cc:1403] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 12:11:22.763091 10860 raft_consensus.cc:359] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.763749 10860 raft_consensus.cc:385] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.764029 10860 raft_consensus.cc:740] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.764720 10860 consensus_queue.cc:260] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.766768 10860 ts_tablet_manager.cc:1434] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:11:22.767020 10744 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 1.
I20260709 12:11:22.756533 10858 consensus_queue.cc:260] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.768213 10781 leader_election.cc:304] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:22.768388 10866 raft_consensus.cc:493] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.768864 10866 raft_consensus.cc:515] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.769232 10858 ts_tablet_manager.cc:1434] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:11:22.770349 10860 tablet_bootstrap.cc:492] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478: Bootstrap starting.
I20260709 12:11:22.771039 10866 leader_election.cc:290] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:22.771528 10866 raft_consensus.cc:2804] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:22.772804 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.773479 10668 raft_consensus.cc:2468] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 0.
I20260709 12:11:22.774791 10884 raft_consensus.cc:493] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.775235 10884 raft_consensus.cc:515] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.770889 10859 tablet_bootstrap.cc:654] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.777164 10784 leader_election.cc:304] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:22.778195 10885 raft_consensus.cc:2804] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.778537 10885 raft_consensus.cc:493] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.778883 10885 raft_consensus.cc:3060] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.778968 10669 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 1.
I20260709 12:11:22.781307 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.781945 10818 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 0.
I20260709 12:11:22.781843 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.782784 10743 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 0.
I20260709 12:11:22.784202 10635 leader_election.cc:304] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:22.777308 10884 leader_election.cc:290] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.785033 10865 raft_consensus.cc:2804] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.785382 10865 raft_consensus.cc:493] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.785739 10865 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.788904 10860 tablet_bootstrap.cc:654] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:22.772837 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.791983 10744 raft_consensus.cc:2468] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 0.
I20260709 12:11:22.792431 10885 raft_consensus.cc:515] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.795836 10669 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.796343 10885 leader_election.cc:290] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:22.798836 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:22.799409 10743 raft_consensus.cc:3060] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.801608 10865 raft_consensus.cc:515] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.803648 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:22.804268 10668 raft_consensus.cc:3060] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.805413 10859 tablet_bootstrap.cc:492] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2: No bootstrap required, opened a new log
I20260709 12:11:22.805841 10884 raft_consensus.cc:493] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.806216 10859 ts_tablet_manager.cc:1403] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent bootstrapping tablet: real 0.046s	user 0.022s	sys 0.012s
I20260709 12:11:22.806262 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:22.805445 10865 leader_election.cc:290] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.806346 10884 raft_consensus.cc:515] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.806937 10744 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.808463 10884 leader_election.cc:290] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.808910 10859 raft_consensus.cc:359] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.809303 10742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.809558 10859 raft_consensus.cc:385] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.809928 10859 raft_consensus.cc:740] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.810000 10742 raft_consensus.cc:2468] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 0.
I20260709 12:11:22.810617 10859 consensus_queue.cc:260] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.811319 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:22.811671 10817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.811859 10818 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.812225 10817 raft_consensus.cc:2468] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 0.
I20260709 12:11:22.813247 10632 leader_election.cc:304] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:22.814040 10865 raft_consensus.cc:2804] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.814399 10865 raft_consensus.cc:493] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.814750 10865 raft_consensus.cc:3060] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.819988 10818 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:22.821334 10635 leader_election.cc:304] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:22.812561 10859 ts_tablet_manager.cc:1434] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:11:22.822206 10884 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.823622 10884 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.835063 10742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.838230 10884 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.812914 10866 raft_consensus.cc:697] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:22.842670 10866 consensus_queue.cc:237] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.829480 10743 raft_consensus.cc:2468] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 1.
I20260709 12:11:22.850895 10781 leader_election.cc:304] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:22.820175 10744 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:22.854839 10910 raft_consensus.cc:2804] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:22.855510 10910 raft_consensus.cc:697] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:22.856616 10910 consensus_queue.cc:237] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.859226 10668 raft_consensus.cc:2468] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 1.
W20260709 12:11:22.860334 10844 tablet.cc:2406] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:11:22.865309 10884 raft_consensus.cc:2804] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:22.869302 10884 raft_consensus.cc:697] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:22.873090 10884 consensus_queue.cc:237] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.874636 10860 tablet_bootstrap.cc:492] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478: No bootstrap required, opened a new log
I20260709 12:11:22.875129 10860 ts_tablet_manager.cc:1403] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.050s
I20260709 12:11:22.877776 10860 raft_consensus.cc:359] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.878513 10860 raft_consensus.cc:385] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:22.878804 10860 raft_consensus.cc:740] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Initialized, Role: FOLLOWER
I20260709 12:11:22.879508 10860 consensus_queue.cc:260] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
W20260709 12:11:22.879925 10635 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:36337, user_credentials={real_user=slave}} blocked reactor thread for 58681.9us
I20260709 12:11:22.881455 10864 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:22.882485 10860 ts_tablet_manager.cc:1434] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:11:22.883664 10865 raft_consensus.cc:515] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.884254 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.884884 10818 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:22.886199 10865 leader_election.cc:290] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.886155 10635 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:22.886910 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:22.887522 10744 raft_consensus.cc:3060] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.888056 10915 raft_consensus.cc:2804] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:11:22.888573 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:22.889171 10818 raft_consensus.cc:3060] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.894731 10744 raft_consensus.cc:2468] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:22.896121 10632 leader_election.cc:304] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:22.897034 10865 raft_consensus.cc:2804] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:22.897686 10865 raft_consensus.cc:697] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:22.897279 10818 raft_consensus.cc:2468] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:22.888547 10915 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d844c1ce274a4f1cadebcf807f750cca)
I20260709 12:11:22.898766 10865 consensus_queue.cc:237] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.899443 10915 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:22.902283 10885 consensus_queue.cc:1048] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:22.905704 10912 raft_consensus.cc:493] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.906203 10912 raft_consensus.cc:515] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.908545 10912 leader_election.cc:290] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.909793 10885 raft_consensus.cc:493] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.908242 10874 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:11:22.910383 10885 raft_consensus.cc:515] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.915694 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5499e27815cb4f10830507e5c42f495f" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.916395 10818 raft_consensus.cc:2468] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 0.
I20260709 12:11:22.916577 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.917418 10667 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 0.
I20260709 12:11:22.917500 10635 leader_election.cc:304] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:22.918691 10912 raft_consensus.cc:2804] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.919001 10885 leader_election.cc:290] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:22.919175 10912 raft_consensus.cc:493] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.919703 10912 raft_consensus.cc:3060] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.920322 10784 leader_election.cc:304] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:22.921897 10885 raft_consensus.cc:2804] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.922518 10885 raft_consensus.cc:493] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.923034 10885 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:22.926926 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.927852 10743 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 0.
I20260709 12:11:22.930423 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5499e27815cb4f10830507e5c42f495f" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:22.931386 10744 raft_consensus.cc:2468] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 0.
I20260709 12:11:22.930729 10874 raft_consensus.cc:493] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.931857 10915 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.934816 10864 raft_consensus.cc:493] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.935288 10864 raft_consensus.cc:515] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.952291 10864 leader_election.cc:290] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.934546 10874 raft_consensus.cc:515] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.957387 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:22.958221 10817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:22.957839 10564 catalog_manager.cc:5697] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:22.959542 10668 raft_consensus.cc:1240] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Rejecting Update request from peer d844c1ce274a4f1cadebcf807f750cca for earlier term 1. Current term is 2. Ops: []
I20260709 12:11:22.965664 10667 raft_consensus.cc:2468] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 0.
I20260709 12:11:22.969658 10817 raft_consensus.cc:2468] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 0.
I20260709 12:11:22.971583 10710 leader_election.cc:304] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:22.984838 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:22.984676 10912 raft_consensus.cc:515] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:22.994565 10915 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.994874 10918 consensus_queue.cc:1059] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:11:22.995895 10918 raft_consensus.cc:3055] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:22.995967 10912 leader_election.cc:290] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:22.996001 10817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:22.996397 10918 raft_consensus.cc:740] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:22.999433 10919 raft_consensus.cc:2804] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:22.999262 10918 consensus_queue.cc:260] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.000547 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5499e27815cb4f10830507e5c42f495f" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:23.000885 10918 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.001282 10744 raft_consensus.cc:3060] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.000455 10816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5499e27815cb4f10830507e5c42f495f" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:23.001879 10816 raft_consensus.cc:3060] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.013695 10635 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5602bb38ed70475da6831345a8d07478; no voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:23.016244 10885 raft_consensus.cc:515] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.018168 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:23.019694 10912 raft_consensus.cc:2749] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:11:23.021281 10874 leader_election.cc:290] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:23.010221 10919 raft_consensus.cc:493] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:23.022994 10919 raft_consensus.cc:3060] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.022959 10817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" is_pre_election: true
I20260709 12:11:23.024603 10817 raft_consensus.cc:2393] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d844c1ce274a4f1cadebcf807f750cca in current term 1: Already voted for candidate 5602bb38ed70475da6831345a8d07478 in this term.
I20260709 12:11:23.025944 10710 leader_election.cc:304] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d844c1ce274a4f1cadebcf807f750cca; no voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2
I20260709 12:11:23.027176 10874 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.036094 10744 raft_consensus.cc:2468] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:23.036550 10919 raft_consensus.cc:515] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.037670 10632 leader_election.cc:304] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:23.038277 10816 raft_consensus.cc:2468] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 1.
I20260709 12:11:23.038684 10884 raft_consensus.cc:2804] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:23.038060 10564 catalog_manager.cc:5697] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.039211 10884 raft_consensus.cc:697] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:23.043387 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:23.043372 10884 consensus_queue.cc:237] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.044260 10667 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.048238 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:23.049027 10668 raft_consensus.cc:3060] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.051970 10874 raft_consensus.cc:2749] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:23.062191 10885 leader_election.cc:290] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:23.068147 10744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:23.068857 10744 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.069541 10816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:23.070358 10816 raft_consensus.cc:3060] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:23.078711 10744 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 1.
I20260709 12:11:23.079938 10781 leader_election.cc:304] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:23.063700 10919 leader_election.cc:290] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:23.100054 10923 raft_consensus.cc:2804] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:23.104228 10923 raft_consensus.cc:697] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:23.106848 10923 consensus_queue.cc:237] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.109462 10910 consensus_queue.cc:1048] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:23.130189 10816 raft_consensus.cc:2468] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 1.
I20260709 12:11:23.131332 10884 consensus_queue.cc:1048] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:23.133486 10668 raft_consensus.cc:2468] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 1.
I20260709 12:11:23.134521 10710 leader_election.cc:304] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:23.124485 10667 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 1.
I20260709 12:11:23.136682 10918 raft_consensus.cc:2804] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:23.137586 10918 raft_consensus.cc:697] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Becoming Leader. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:23.138454 10918 consensus_queue.cc:237] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.141360 10564 catalog_manager.cc:5697] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:23.161423 10565 catalog_manager.cc:5697] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.164839 10865 consensus_queue.cc:1048] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.167228 10565 catalog_manager.cc:5697] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.180464 10564 catalog_manager.cc:5697] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:23.190697 10565 catalog_manager.cc:5697] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca reported cstate change: term changed from 0 to 1, leader changed from <none> to d844c1ce274a4f1cadebcf807f750cca (127.9.198.194). New cstate: current_term: 1 leader_uuid: "d844c1ce274a4f1cadebcf807f750cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.275115 10865 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.281945 10923 consensus_queue.cc:1048] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.286801 10865 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:23.295466 10910 consensus_queue.cc:1048] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.307650 10865 consensus_queue.cc:1048] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.321151 10865 consensus_queue.cc:1048] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.340283 10923 consensus_queue.cc:1048] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.350557 10910 consensus_queue.cc:1048] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.381423 10743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
mode: GRACEFUL
new_leader_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:54116
I20260709 12:11:23.382054 10743 raft_consensus.cc:606] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Received request to transfer leadership to TS 5602bb38ed70475da6831345a8d07478
I20260709 12:11:23.382371 10743 raft_consensus.cc:614] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:23.400609 10816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
mode: GRACEFUL
new_leader_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:49266
I20260709 12:11:23.401165 10816 raft_consensus.cc:606] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Received request to transfer leadership to TS 5602bb38ed70475da6831345a8d07478
I20260709 12:11:23.415052 10667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
mode: GRACEFUL
new_leader_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:34028
I20260709 12:11:23.415515 10667 raft_consensus.cc:606] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Received request to transfer leadership to TS d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:23.423352 10816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
mode: GRACEFUL
new_leader_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:49266
I20260709 12:11:23.423936 10816 raft_consensus.cc:606] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Received request to transfer leadership to TS d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:23.429513 10816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
mode: GRACEFUL
new_leader_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:49266
I20260709 12:11:23.430079 10816 raft_consensus.cc:606] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Received request to transfer leadership to TS d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:23.509508 10865 consensus_queue.cc:1048] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.520330 10865 consensus_queue.cc:1048] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.555456 10910 consensus_queue.cc:1048] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.560372 10918 consensus_queue.cc:1048] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:23.563450 10866 raft_consensus.cc:993] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2: : Instructing follower d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:23.563902 10923 raft_consensus.cc:1081] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Signalling peer d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:23.569432 10885 consensus_queue.cc:1048] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:23.581764 10874 consensus_queue.cc:1048] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.581830 10743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:54106
I20260709 12:11:23.583268 10743 raft_consensus.cc:493] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:23.583806 10743 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.593360 10743 raft_consensus.cc:515] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.594588 10744 raft_consensus.cc:1240] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Rejecting Update request from peer bcb0169f1fa94e1096ecb4e038005ab2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:11:23.595811 10743 leader_election.cc:290] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 2 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:23.596674 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:23.597431 10667 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.598083 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:23.598686 10910 consensus_queue.cc:1059] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:23.601599 10818 raft_consensus.cc:3055] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:23.601971 10818 raft_consensus.cc:740] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:23.602774 10818 consensus_queue.cc:260] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.603976 10667 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 2.
I20260709 12:11:23.605273 10710 leader_election.cc:304] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:23.604058 10818 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.606105 10874 raft_consensus.cc:2804] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:23.606626 10874 raft_consensus.cc:697] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Becoming Leader. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:23.607606 10874 consensus_queue.cc:237] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.611461 10818 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 2.
W20260709 12:11:23.612001 10866 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:11:23.616514 10565 catalog_manager.cc:5697] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca reported cstate change: term changed from 1 to 2, leader changed from bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195) to d844c1ce274a4f1cadebcf807f750cca (127.9.198.194). New cstate: current_term: 2 leader_uuid: "d844c1ce274a4f1cadebcf807f750cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.641808 10915 raft_consensus.cc:993] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478: : Instructing follower d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:23.642289 10865 raft_consensus.cc:1081] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Signalling peer d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:23.643850 10743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:54100
I20260709 12:11:23.644402 10743 raft_consensus.cc:493] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:23.644711 10743 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.649413 10743 raft_consensus.cc:515] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.651403 10743 leader_election.cc:290] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 2 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:23.652307 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:23.652563 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:23.653046 10667 raft_consensus.cc:3055] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:23.653115 10818 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.653527 10667 raft_consensus.cc:740] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:23.654537 10667 consensus_queue.cc:260] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.656123 10667 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.660564 10818 raft_consensus.cc:2468] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 2.
I20260709 12:11:23.661933 10710 leader_election.cc:304] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:23.662756 10874 raft_consensus.cc:2804] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:23.663056 10667 raft_consensus.cc:2468] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 2.
I20260709 12:11:23.663187 10874 raft_consensus.cc:697] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Becoming Leader. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:23.664117 10874 consensus_queue.cc:237] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.672062 10565 catalog_manager.cc:5697] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca reported cstate change: term changed from 1 to 2, leader changed from 5602bb38ed70475da6831345a8d07478 (127.9.198.193) to d844c1ce274a4f1cadebcf807f750cca (127.9.198.194). New cstate: current_term: 2 leader_uuid: "d844c1ce274a4f1cadebcf807f750cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.756635 10910 raft_consensus.cc:993] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: : Instructing follower 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:23.757244 10866 raft_consensus.cc:1081] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Signalling peer 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:23.759348 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:34024
I20260709 12:11:23.760105 10667 raft_consensus.cc:493] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:23.760562 10667 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.766135 10667 raft_consensus.cc:515] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.767664 10667 leader_election.cc:290] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:23.768550 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:23.768774 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:23.769155 10743 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.769268 10818 raft_consensus.cc:3055] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:23.769622 10818 raft_consensus.cc:740] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:23.770517 10818 consensus_queue.cc:260] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.771731 10818 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.776083 10743 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 2.
I20260709 12:11:23.777478 10632 leader_election.cc:304] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:23.778366 10865 raft_consensus.cc:2804] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:23.778852 10818 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 2.
I20260709 12:11:23.778852 10865 raft_consensus.cc:697] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:23.780049 10865 consensus_queue.cc:237] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.786795 10565 catalog_manager.cc:5697] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 1 to 2, leader changed from bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195) to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.961179 10866 raft_consensus.cc:993] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2: : Instructing follower d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:23.961694 10866 raft_consensus.cc:1081] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Signalling peer d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:23.963200 10743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:54106
I20260709 12:11:23.963702 10743 raft_consensus.cc:493] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:23.964222 10743 raft_consensus.cc:3060] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.969300 10743 raft_consensus.cc:515] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.971159 10743 leader_election.cc:290] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 2 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:23.971967 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:23.972239 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5dbb6539f5246788b4eeb3b75494213" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:23.972581 10667 raft_consensus.cc:3060] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.972849 10818 raft_consensus.cc:3055] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:23.973178 10818 raft_consensus.cc:740] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:23.973901 10818 consensus_queue.cc:260] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.974812 10818 raft_consensus.cc:3060] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:23.977375 10667 raft_consensus.cc:2468] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 2.
I20260709 12:11:23.978643 10710 leader_election.cc:304] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:23.979393 10874 raft_consensus.cc:2804] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:23.979902 10874 raft_consensus.cc:697] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Becoming Leader. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:23.979611 10818 raft_consensus.cc:2468] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 2.
I20260709 12:11:23.980782 10874 consensus_queue.cc:237] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:23.988031 10565 catalog_manager.cc:5697] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca reported cstate change: term changed from 1 to 2, leader changed from bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195) to d844c1ce274a4f1cadebcf807f750cca (127.9.198.194). New cstate: current_term: 2 leader_uuid: "d844c1ce274a4f1cadebcf807f750cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:23.993584 10667 raft_consensus.cc:1275] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:23.995213 10874 consensus_queue.cc:1048] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.003474 10818 raft_consensus.cc:1275] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.005909 10874 consensus_queue.cc:1048] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.049823 10816 raft_consensus.cc:1275] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.050995 10918 consensus_queue.cc:1048] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.059548 10667 raft_consensus.cc:1275] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.061287 10874 consensus_queue.cc:1048] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.261260 10743 raft_consensus.cc:1275] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.262683 10921 consensus_queue.cc:1048] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.271154 10816 raft_consensus.cc:1275] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.272511 10865 consensus_queue.cc:1048] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.481249 10866 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d844c1ce274a4f1cadebcf807f750cca)
I20260709 12:11:24.481864 10866 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:24.483579 10866 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:24.484582 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca" is_pre_election: true
I20260709 12:11:24.484575 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478" is_pre_election: true
I20260709 12:11:24.485219 10743 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 2.
I20260709 12:11:24.485679 10667 raft_consensus.cc:2393] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcb0169f1fa94e1096ecb4e038005ab2 in current term 2: Already voted for candidate 5602bb38ed70475da6831345a8d07478 in this term.
I20260709 12:11:24.486413 10781 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:24.487231 10866 raft_consensus.cc:2804] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:11:24.487582 10866 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d844c1ce274a4f1cadebcf807f750cca)
I20260709 12:11:24.487972 10866 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:24.493306 10866 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:24.494936 10866 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:24.495976 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:24.496126 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:24.496688 10667 raft_consensus.cc:2393] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bcb0169f1fa94e1096ecb4e038005ab2 in current term 2: Already voted for candidate 5602bb38ed70475da6831345a8d07478 in this term.
I20260709 12:11:24.501034 10743 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 2.
I20260709 12:11:24.502091 10781 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 5602bb38ed70475da6831345a8d07478
I20260709 12:11:24.502807 10866 raft_consensus.cc:2804] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:24.503749 10866 raft_consensus.cc:697] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:24.504550 10866 consensus_queue.cc:237] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:24.512670 10565 catalog_manager.cc:5697] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 1 to 2, leader changed from d844c1ce274a4f1cadebcf807f750cca (127.9.198.194) to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:24.515717 10816 raft_consensus.cc:1275] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.520972 10919 consensus_queue.cc:1048] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.531795 10667 raft_consensus.cc:1275] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.533650 10874 consensus_queue.cc:1048] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:24.979296 10667 raft_consensus.cc:1275] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.980196 10743 raft_consensus.cc:1275] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:24.980727 10866 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:24.981528 10953 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:26.468976 10011 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:11:26.469696 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:26.487105 10667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
mode: GRACEFUL
new_leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
 from {username='slave'} at 127.0.0.1:34034
I20260709 12:11:26.487650 10667 raft_consensus.cc:606] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Received request to transfer leadership to TS bcb0169f1fa94e1096ecb4e038005ab2
I20260709 12:11:26.488910 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:11:26.489317 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:26.538897 10970 raft_consensus.cc:993] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: : Instructing follower bcb0169f1fa94e1096ecb4e038005ab2 to start an election
I20260709 12:11:26.539424 10961 raft_consensus.cc:1081] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Signalling peer bcb0169f1fa94e1096ecb4e038005ab2 to start an election
I20260709 12:11:26.540997 10816 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
 from {username='slave'} at 127.0.0.1:49252
I20260709 12:11:26.541666 10816 raft_consensus.cc:493] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:26.542078 10816 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:26.549907 10816 raft_consensus.cc:515] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:26.552342 10816 leader_election.cc:290] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:26.553117 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:26.553506 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:26.553822 10667 raft_consensus.cc:3055] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:26.554184 10667 raft_consensus.cc:740] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:26.554220 10743 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:26.554947 10667 consensus_queue.cc:260] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:26.555876 10667 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:26.562137 10667 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 2.
I20260709 12:11:26.563265 10743 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 2.
I20260709 12:11:26.563562 10784 leader_election.cc:304] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:26.564514 10964 raft_consensus.cc:2804] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:26.565544 10964 raft_consensus.cc:697] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:26.566381 10964 consensus_queue.cc:237] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:26.573195 10565 catalog_manager.cc:5697] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 1 to 2, leader changed from 5602bb38ed70475da6831345a8d07478 (127.9.198.193) to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:27.072414 10667 raft_consensus.cc:1275] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:27.073580 10964 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:27.083979 10743 raft_consensus.cc:1275] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:27.085328 10978 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:27.490159 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:27.496187 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:27.496567 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:28.497684 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:28.504344 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:28.504730 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:29.505738 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:29.510990 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:29.511379 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:30.512696 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:30.519703 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:30.520129 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:31.521282 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:31.527815 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:31.528218 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:32.529125 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:32.535302 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:32.535681 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:33.536482 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:33.542038 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:33.542428 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:34.543282 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:34.549535 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:34.550027 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:35.551014 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:35.556308 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:35.556706 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:36.590637 10011 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:11:36.598603 10818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
mode: GRACEFUL
new_leader_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:49266
I20260709 12:11:36.599362 10818 raft_consensus.cc:606] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Received request to transfer leadership to TS 5602bb38ed70475da6831345a8d07478
I20260709 12:11:36.604592 10818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
mode: GRACEFUL
new_leader_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:49266
I20260709 12:11:36.605125 10818 raft_consensus.cc:606] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Received request to transfer leadership to TS 5602bb38ed70475da6831345a8d07478
I20260709 12:11:36.615814 10743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
mode: GRACEFUL
new_leader_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:54116
I20260709 12:11:36.616294 10743 raft_consensus.cc:606] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Received request to transfer leadership to TS 5602bb38ed70475da6831345a8d07478
I20260709 12:11:36.621824 10743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
mode: GRACEFUL
new_leader_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:54116
I20260709 12:11:36.622318 10743 raft_consensus.cc:606] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Received request to transfer leadership to TS 5602bb38ed70475da6831345a8d07478
I20260709 12:11:36.627460 10743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
mode: GRACEFUL
new_leader_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:54116
I20260709 12:11:36.628096 10743 raft_consensus.cc:606] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Received request to transfer leadership to TS 5602bb38ed70475da6831345a8d07478
I20260709 12:11:36.677865 10990 raft_consensus.cc:993] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca: : Instructing follower 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:36.678519 10993 raft_consensus.cc:1081] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Signalling peer 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:36.680124 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:34010
I20260709 12:11:36.680608 10667 raft_consensus.cc:493] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:36.680922 10667 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:36.686672 10667 raft_consensus.cc:515] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.688550 10667 leader_election.cc:290] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:36.689385 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:36.689601 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d162003b66343d791f60ebb8c4f306f" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:36.690171 10743 raft_consensus.cc:3055] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:36.690246 10818 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:36.690476 10743 raft_consensus.cc:740] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:36.691306 10743 consensus_queue.cc:260] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.692456 10743 raft_consensus.cc:3060] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:36.695797 10818 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:36.697007 10635 leader_election.cc:304] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:36.697359 10743 raft_consensus.cc:2468] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:36.697954 10984 raft_consensus.cc:2804] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:36.698416 10984 raft_consensus.cc:697] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:36.699345 10984 consensus_queue.cc:237] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.706066 10565 catalog_manager.cc:5697] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 2 to 3, leader changed from d844c1ce274a4f1cadebcf807f750cca (127.9.198.194) to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:36.915488 10990 raft_consensus.cc:993] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca: : Instructing follower 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:36.916039 10990 raft_consensus.cc:1081] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Signalling peer 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:36.917735 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:34010
I20260709 12:11:36.918365 10985 raft_consensus.cc:993] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2: : Instructing follower 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:36.918339 10667 raft_consensus.cc:493] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:36.918905 10667 raft_consensus.cc:3060] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:36.919129 10999 raft_consensus.cc:1081] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Signalling peer 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:36.921260 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:34024
I20260709 12:11:36.921833 10668 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:36.922190 10668 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:36.925261 10667 raft_consensus.cc:515] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.927372 10667 leader_election.cc:290] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:36.927953 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:36.927894 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6ab93f5577474fa12fef3ef8f0fd24" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:36.928635 10818 raft_consensus.cc:3060] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:36.928689 10743 raft_consensus.cc:3055] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:36.929117 10743 raft_consensus.cc:740] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:36.929179 10668 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.929968 10743 consensus_queue.cc:260] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.931201 10743 raft_consensus.cc:3060] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:36.932403 10742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:36.932657 10816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:36.933043 10742 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:36.933297 10816 raft_consensus.cc:3055] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:36.933738 10816 raft_consensus.cc:740] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:36.934547 10816 consensus_queue.cc:260] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.935596 10816 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:36.936345 10818 raft_consensus.cc:2468] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 2.
I20260709 12:11:36.937691 10668 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:36.938676 10743 raft_consensus.cc:2468] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 2.
I20260709 12:11:36.937678 10635 leader_election.cc:304] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:36.940399 10984 raft_consensus.cc:2804] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:36.940876 10984 raft_consensus.cc:697] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 2 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:36.941581 10742 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:36.941948 10984 consensus_queue.cc:237] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.942880 10632 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:36.943228 10816 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:36.943828 10992 raft_consensus.cc:2804] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:36.945597 10992 raft_consensus.cc:697] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:36.946672 10992 consensus_queue.cc:237] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:36.951099 10565 catalog_manager.cc:5697] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 1 to 2, leader changed from d844c1ce274a4f1cadebcf807f750cca (127.9.198.194) to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:36.963165 10565 catalog_manager.cc:5697] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 2 to 3, leader changed from bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195) to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:37.089614 10742 raft_consensus.cc:1275] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.091009 10984 consensus_queue.cc:1048] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:37.094754 10985 raft_consensus.cc:993] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2: : Instructing follower 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:37.095376 10999 raft_consensus.cc:1081] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Signalling peer 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:37.097837 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:34024
I20260709 12:11:37.098369 10668 raft_consensus.cc:493] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:37.098819 10668 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:37.108328 10816 raft_consensus.cc:1275] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.109647 10992 consensus_queue.cc:1048] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:37.109876 10668 raft_consensus.cc:515] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:37.112006 10668 leader_election.cc:290] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:37.112630 10742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:37.113260 10742 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:37.115195 10816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:37.115758 10816 raft_consensus.cc:3055] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:37.116031 10816 raft_consensus.cc:740] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:37.116770 10816 consensus_queue.cc:260] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:37.118072 10816 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:37.120582 10742 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:37.122574 10632 leader_election.cc:304] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:37.123406 10992 raft_consensus.cc:2804] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:37.125823 10816 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:37.127218 10992 raft_consensus.cc:697] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:37.128495 10992 consensus_queue.cc:237] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:37.137773 10565 catalog_manager.cc:5697] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 2 to 3, leader changed from bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195) to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:37.213864 10990 raft_consensus.cc:993] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca: : Instructing follower 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:37.214347 10993 raft_consensus.cc:1081] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Signalling peer 5602bb38ed70475da6831345a8d07478 to start an election
I20260709 12:11:37.215665 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
 from {username='slave'} at 127.0.0.1:34010
I20260709 12:11:37.216163 10668 raft_consensus.cc:493] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:37.216481 10668 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:37.221031 10668 raft_consensus.cc:515] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:37.222551 10668 leader_election.cc:290] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Requested vote from peers d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:37.223425 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:37.223713 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7641bfe7012f419a9329c5d755bf70c5" candidate_uuid: "5602bb38ed70475da6831345a8d07478" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:37.223996 10743 raft_consensus.cc:3055] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:37.224220 10818 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:37.224325 10743 raft_consensus.cc:740] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:37.225111 10743 consensus_queue.cc:260] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:37.226217 10743 raft_consensus.cc:3060] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:37.230774 10818 raft_consensus.cc:2468] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:37.231902 10635 leader_election.cc:304] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5602bb38ed70475da6831345a8d07478, bcb0169f1fa94e1096ecb4e038005ab2; no voters: 
I20260709 12:11:37.232388 10743 raft_consensus.cc:2468] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5602bb38ed70475da6831345a8d07478 in term 3.
I20260709 12:11:37.232553 10992 raft_consensus.cc:2804] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:37.232997 10992 raft_consensus.cc:697] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Becoming Leader. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:37.233938 10992 consensus_queue.cc:237] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:37.241258 10565 catalog_manager.cc:5697] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 reported cstate change: term changed from 2 to 3, leader changed from d844c1ce274a4f1cadebcf807f750cca (127.9.198.194) to 5602bb38ed70475da6831345a8d07478 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "5602bb38ed70475da6831345a8d07478" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:37.353749 10743 raft_consensus.cc:1275] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:37.355158 10992 consensus_queue.cc:1048] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:37.366954 10818 raft_consensus.cc:1275] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:37.368366 10992 consensus_queue.cc:1048] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:37.387847 10743 raft_consensus.cc:1275] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.389266 10992 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:37.400295 10818 raft_consensus.cc:1275] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.402091 11013 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:37.537019 10818 raft_consensus.cc:1275] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.538375 11013 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:37.547150 10743 raft_consensus.cc:1275] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.549170 11000 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:37.642238 10818 raft_consensus.cc:1275] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.643496 11000 consensus_queue.cc:1048] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:37.656374 10743 raft_consensus.cc:1275] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Refusing update from remote peer 5602bb38ed70475da6831345a8d07478: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:37.658411 11019 consensus_queue.cc:1048] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:39.667269 10011 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:11:39.668066 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:39.676080 10668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
mode: GRACEFUL
new_leader_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:34034
I20260709 12:11:39.676822 10668 raft_consensus.cc:606] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 LEADER]: Received request to transfer leadership to TS d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:39.679546 10668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
mode: GRACEFUL
new_leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
 from {username='slave'} at 127.0.0.1:34034
I20260709 12:11:39.680009 10668 raft_consensus.cc:606] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Received request to transfer leadership to TS bcb0169f1fa94e1096ecb4e038005ab2
I20260709 12:11:39.682242 10668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
mode: GRACEFUL
new_leader_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:34034
I20260709 12:11:39.682716 10668 raft_consensus.cc:606] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Received request to transfer leadership to TS d844c1ce274a4f1cadebcf807f750cca
I20260709 12:11:39.685115 10668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482"
dest_uuid: "5602bb38ed70475da6831345a8d07478"
mode: GRACEFUL
new_leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
 from {username='slave'} at 127.0.0.1:34034
I20260709 12:11:39.685570 10668 raft_consensus.cc:606] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Received request to transfer leadership to TS bcb0169f1fa94e1096ecb4e038005ab2
I20260709 12:11:39.686795 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:11:39.687372 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:39.779161 11019 raft_consensus.cc:993] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478: : Instructing follower d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:39.779656 11021 raft_consensus.cc:1081] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 LEADER]: Signalling peer d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:39.781199 10743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:54100
I20260709 12:11:39.781831 10743 raft_consensus.cc:493] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:39.782152 10743 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:39.786885 10743 raft_consensus.cc:515] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:39.789268 10743 leader_election.cc:290] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 3 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:39.790180 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:39.790478 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a3b53d3dccf44eab7ff8addb3665c2c" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:39.790824 10668 raft_consensus.cc:3055] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:39.791054 10818 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:39.791183 10668 raft_consensus.cc:740] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:39.792094 10668 consensus_queue.cc:260] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:39.793612 10668 raft_consensus.cc:3060] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:39.796317 10818 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 3.
I20260709 12:11:39.797580 10710 leader_election.cc:304] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:39.798509 10993 raft_consensus.cc:2804] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:39.798897 10668 raft_consensus.cc:2468] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 3.
I20260709 12:11:39.799523 10993 raft_consensus.cc:697] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 3 LEADER]: Becoming Leader. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:39.800544 10993 consensus_queue.cc:237] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:39.808245 10565 catalog_manager.cc:5697] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca reported cstate change: term changed from 2 to 3, leader changed from 5602bb38ed70475da6831345a8d07478 (127.9.198.193) to d844c1ce274a4f1cadebcf807f750cca (127.9.198.194). New cstate: current_term: 3 leader_uuid: "d844c1ce274a4f1cadebcf807f750cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:39.963534 11019 raft_consensus.cc:993] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478: : Instructing follower d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:39.963968 11021 raft_consensus.cc:1081] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Signalling peer d844c1ce274a4f1cadebcf807f750cca to start an election
I20260709 12:11:39.965477 10743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522"
dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
 from {username='slave'} at 127.0.0.1:54100
I20260709 12:11:39.966107 10743 raft_consensus.cc:493] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:39.966406 10743 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:39.972016 10743 raft_consensus.cc:515] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:39.973522 10743 leader_election.cc:290] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 4 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195:36337)
I20260709 12:11:39.974745 10818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
I20260709 12:11:39.974592 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dbc30b646ab4fdfae2f48b8ed780522" candidate_uuid: "d844c1ce274a4f1cadebcf807f750cca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:39.975382 10818 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:39.975485 10668 raft_consensus.cc:3055] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:39.975903 10668 raft_consensus.cc:740] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:39.976769 10668 consensus_queue.cc:260] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:39.978117 10668 raft_consensus.cc:3060] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:39.982698 10818 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 4.
I20260709 12:11:39.984052 10710 leader_election.cc:304] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:39.984795 10993 raft_consensus.cc:2804] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:39.985381 10993 raft_consensus.cc:697] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 4 LEADER]: Becoming Leader. State: Replica: d844c1ce274a4f1cadebcf807f750cca, State: Running, Role: LEADER
I20260709 12:11:39.985320 10668 raft_consensus.cc:2468] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844c1ce274a4f1cadebcf807f750cca in term 4.
I20260709 12:11:39.986683 10993 consensus_queue.cc:237] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:39.994196 10565 catalog_manager.cc:5697] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca reported cstate change: term changed from 3 to 4, leader changed from 5602bb38ed70475da6831345a8d07478 (127.9.198.193) to d844c1ce274a4f1cadebcf807f750cca (127.9.198.194). New cstate: current_term: 4 leader_uuid: "d844c1ce274a4f1cadebcf807f750cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:40.147743 11019 raft_consensus.cc:993] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: : Instructing follower bcb0169f1fa94e1096ecb4e038005ab2 to start an election
I20260709 12:11:40.148244 11021 raft_consensus.cc:1081] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Signalling peer bcb0169f1fa94e1096ecb4e038005ab2 to start an election
I20260709 12:11:40.149731 10818 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
 from {username='slave'} at 127.0.0.1:49252
I20260709 12:11:40.150206 10818 raft_consensus.cc:493] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:40.150498 10818 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:40.154764 10818 raft_consensus.cc:515] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:40.156883 10818 leader_election.cc:290] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:40.157670 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:40.157850 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596715d7c34e4297b4cf0287d1ee5861" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:40.158270 10667 raft_consensus.cc:3055] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:40.158455 10743 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:40.158631 10667 raft_consensus.cc:740] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:40.159459 10667 consensus_queue.cc:260] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:40.160527 10667 raft_consensus.cc:3060] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:40.163947 10743 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 4.
I20260709 12:11:40.165099 10781 leader_election.cc:304] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:40.165199 10667 raft_consensus.cc:2468] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 4.
I20260709 12:11:40.165970 10999 raft_consensus.cc:2804] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:40.167135 10999 raft_consensus.cc:697] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:40.168112 10999 consensus_queue.cc:237] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:40.175900 10565 catalog_manager.cc:5697] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 3 to 4, leader changed from 5602bb38ed70475da6831345a8d07478 (127.9.198.193) to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 4 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.254819 11019 raft_consensus.cc:993] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478: : Instructing follower bcb0169f1fa94e1096ecb4e038005ab2 to start an election
I20260709 12:11:40.255280 11021 raft_consensus.cc:1081] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Signalling peer bcb0169f1fa94e1096ecb4e038005ab2 to start an election
I20260709 12:11:40.256912 10818 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482"
dest_uuid: "bcb0169f1fa94e1096ecb4e038005ab2"
 from {username='slave'} at 127.0.0.1:49252
I20260709 12:11:40.257304 10818 raft_consensus.cc:493] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:40.257537 10818 raft_consensus.cc:3060] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:40.261495 10818 raft_consensus.cc:515] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:40.262866 10818 leader_election.cc:290] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663), d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737)
I20260709 12:11:40.263634 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5602bb38ed70475da6831345a8d07478"
I20260709 12:11:40.263836 10743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e45ab4fb5247e48fd8e36318e08482" candidate_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d844c1ce274a4f1cadebcf807f750cca"
I20260709 12:11:40.264200 10667 raft_consensus.cc:3055] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:40.264277 10743 raft_consensus.cc:3060] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:40.264547 10667 raft_consensus.cc:740] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5602bb38ed70475da6831345a8d07478, State: Running, Role: LEADER
I20260709 12:11:40.265300 10667 consensus_queue.cc:260] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:40.266367 10667 raft_consensus.cc:3060] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:40.270745 10743 raft_consensus.cc:2468] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 2.
I20260709 12:11:40.271739 10781 leader_election.cc:304] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcb0169f1fa94e1096ecb4e038005ab2, d844c1ce274a4f1cadebcf807f750cca; no voters: 
I20260709 12:11:40.272418 10999 raft_consensus.cc:2804] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:40.272383 10667 raft_consensus.cc:2468] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcb0169f1fa94e1096ecb4e038005ab2 in term 2.
I20260709 12:11:40.272979 10999 raft_consensus.cc:697] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Becoming Leader. State: Replica: bcb0169f1fa94e1096ecb4e038005ab2, State: Running, Role: LEADER
I20260709 12:11:40.273880 10999 consensus_queue.cc:237] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } }
I20260709 12:11:40.280174 10565 catalog_manager.cc:5697] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 reported cstate change: term changed from 1 to 2, leader changed from 5602bb38ed70475da6831345a8d07478 (127.9.198.193) to bcb0169f1fa94e1096ecb4e038005ab2 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.323362 10818 raft_consensus.cc:1275] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:40.324515 10993 consensus_queue.cc:1048] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:40.336767 10667 raft_consensus.cc:1275] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:40.338357 11034 consensus_queue.cc:1048] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:40.505348 10667 raft_consensus.cc:1275] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:40.505300 10818 raft_consensus.cc:1275] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 FOLLOWER]: Refusing update from remote peer d844c1ce274a4f1cadebcf807f750cca: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:40.507879 10993 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:40.508985 11034 consensus_queue.cc:1048] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb0169f1fa94e1096ecb4e038005ab2" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:40.688863 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:40.691658 10667 raft_consensus.cc:1275] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:40.693364 10999 consensus_queue.cc:1048] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:11:40.702699 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:40.703329 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:40.709450 10742 raft_consensus.cc:1275] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:40.713548 10999 consensus_queue.cc:1048] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:40.755203 10667 raft_consensus.cc:1275] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:40.756716 11042 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5602bb38ed70475da6831345a8d07478" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 39663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:40.768172 10742 raft_consensus.cc:1275] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 4 FOLLOWER]: Refusing update from remote peer bcb0169f1fa94e1096ecb4e038005ab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:40.770088 11029 consensus_queue.cc:1048] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844c1ce274a4f1cadebcf807f750cca" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 45737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:41.704643 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:41.710263 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:41.710713 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:42.711730 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:42.717154 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:42.717667 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:43.718640 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:43.724141 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:43.724678 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:44.725668 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:44.731801 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:44.732261 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:45.733193 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:45.739064 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:45.739552 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:46.740571 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:46.747066 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:46.747570 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:47.748696 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:47.755077 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:47.755579 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:48.756723 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:48.762743 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:48.763283 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:49.796875 10011 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:11:49.797695 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
W20260709 12:11:49.801569 10784 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 12:11:49.805773 10784 consensus_peers.cc:597] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 -> Peer 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663): Couldn't send request to peer 5602bb38ed70475da6831345a8d07478. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:49.834777 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:11:49.835600 10011 tablet_replica.cc:333] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.836431 10011 raft_consensus.cc:2243] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:11:49.837433 10011 raft_consensus.cc:2272] T 9d162003b66343d791f60ebb8c4f306f P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.839639 10011 tablet_replica.cc:333] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.840287 10011 raft_consensus.cc:2243] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:11:49.841130 10011 raft_consensus.cc:2272] T 5499e27815cb4f10830507e5c42f495f P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.843269 10011 tablet_replica.cc:333] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.843957 10011 raft_consensus.cc:2243] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.844435 10011 raft_consensus.cc:2272] T 0a3b53d3dccf44eab7ff8addb3665c2c P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.846386 10011 tablet_replica.cc:333] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.846946 10011 raft_consensus.cc:2243] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.847409 10011 raft_consensus.cc:2272] T c5dbb6539f5246788b4eeb3b75494213 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.849198 10011 tablet_replica.cc:333] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.849807 10011 raft_consensus.cc:2243] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.850273 10011 raft_consensus.cc:2272] T c5e45ab4fb5247e48fd8e36318e08482 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.851941 10011 tablet_replica.cc:333] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.852480 10011 raft_consensus.cc:2243] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.852919 10011 raft_consensus.cc:2272] T 6dbc30b646ab4fdfae2f48b8ed780522 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.854809 10011 tablet_replica.cc:333] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.855373 10011 raft_consensus.cc:2243] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.855930 10011 raft_consensus.cc:2272] T b29c191985ea4b53a1c1e6cde36f44cf P 5602bb38ed70475da6831345a8d07478 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.858659 10011 tablet_replica.cc:333] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.859227 10011 raft_consensus.cc:2243] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.859611 10011 raft_consensus.cc:2272] T 596715d7c34e4297b4cf0287d1ee5861 P 5602bb38ed70475da6831345a8d07478 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.861227 10011 tablet_replica.cc:333] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.861703 10011 raft_consensus.cc:2243] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:11:49.862521 10011 raft_consensus.cc:2272] T 6e6ab93f5577474fa12fef3ef8f0fd24 P 5602bb38ed70475da6831345a8d07478 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.864306 10011 tablet_replica.cc:333] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478: stopping tablet replica
I20260709 12:11:49.864791 10011 raft_consensus.cc:2243] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:11:49.865494 10011 raft_consensus.cc:2272] T 7641bfe7012f419a9329c5d755bf70c5 P 5602bb38ed70475da6831345a8d07478 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:11:49.872725 10784 consensus_peers.cc:597] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 -> Peer 5602bb38ed70475da6831345a8d07478 (127.9.198.193:39663): Couldn't send request to peer 5602bb38ed70475da6831345a8d07478. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:49.898316 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
I20260709 12:11:49.920501 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
I20260709 12:11:49.950114 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:11:49.950867 10011 tablet_replica.cc:333] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.951923 10011 raft_consensus.cc:2243] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:11:49.953013 10011 raft_consensus.cc:2272] T c5dbb6539f5246788b4eeb3b75494213 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.954936 10011 tablet_replica.cc:333] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.955430 10011 raft_consensus.cc:2243] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.955802 10011 raft_consensus.cc:2272] T 7641bfe7012f419a9329c5d755bf70c5 P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.957448 10011 tablet_replica.cc:333] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.957952 10011 raft_consensus.cc:2243] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.958312 10011 raft_consensus.cc:2272] T 6e6ab93f5577474fa12fef3ef8f0fd24 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.959893 10011 tablet_replica.cc:333] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.960311 10011 raft_consensus.cc:2243] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.960670 10011 raft_consensus.cc:2272] T 9d162003b66343d791f60ebb8c4f306f P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.962436 10011 tablet_replica.cc:333] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.962910 10011 raft_consensus.cc:2243] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:11:49.963584 10011 raft_consensus.cc:2272] T 6dbc30b646ab4fdfae2f48b8ed780522 P d844c1ce274a4f1cadebcf807f750cca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.965116 10011 tablet_replica.cc:333] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.965562 10011 raft_consensus.cc:2243] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.965979 10011 raft_consensus.cc:2272] T c5e45ab4fb5247e48fd8e36318e08482 P d844c1ce274a4f1cadebcf807f750cca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.967609 10011 tablet_replica.cc:333] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.968027 10011 raft_consensus.cc:2243] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.968389 10011 raft_consensus.cc:2272] T 5499e27815cb4f10830507e5c42f495f P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.969998 10011 tablet_replica.cc:333] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.970595 10011 raft_consensus.cc:2243] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:11:49.971303 10011 raft_consensus.cc:2272] T 0a3b53d3dccf44eab7ff8addb3665c2c P d844c1ce274a4f1cadebcf807f750cca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.972937 10011 tablet_replica.cc:333] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.973400 10011 raft_consensus.cc:2243] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.973834 10011 raft_consensus.cc:2272] T b29c191985ea4b53a1c1e6cde36f44cf P d844c1ce274a4f1cadebcf807f750cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:49.975502 10011 tablet_replica.cc:333] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca: stopping tablet replica
I20260709 12:11:49.975939 10011 raft_consensus.cc:2243] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:49.976322 10011 raft_consensus.cc:2272] T 596715d7c34e4297b4cf0287d1ee5861 P d844c1ce274a4f1cadebcf807f750cca [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:11:49.999152 10781 consensus_peers.cc:597] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 -> Peer d844c1ce274a4f1cadebcf807f750cca (127.9.198.194:45737): Couldn't send request to peer d844c1ce274a4f1cadebcf807f750cca. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:45737: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:50.005915 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
I20260709 12:11:50.024549 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
I20260709 12:11:50.050377 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:11:50.051112 10011 tablet_replica.cc:333] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.051796 10011 raft_consensus.cc:2243] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:50.052328 10011 raft_consensus.cc:2272] T 6e6ab93f5577474fa12fef3ef8f0fd24 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.054152 10011 tablet_replica.cc:333] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.054657 10011 raft_consensus.cc:2243] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:50.055059 10011 raft_consensus.cc:2272] T 5499e27815cb4f10830507e5c42f495f P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.056696 10011 tablet_replica.cc:333] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.057209 10011 raft_consensus.cc:2243] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:50.057952 10011 raft_consensus.cc:2272] T 0a3b53d3dccf44eab7ff8addb3665c2c P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.059667 10011 tablet_replica.cc:333] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.060156 10011 raft_consensus.cc:2243] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:50.060580 10011 raft_consensus.cc:2272] T 7641bfe7012f419a9329c5d755bf70c5 P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.062224 10011 tablet_replica.cc:333] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.062712 10011 raft_consensus.cc:2243] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:50.063112 10011 raft_consensus.cc:2272] T c5dbb6539f5246788b4eeb3b75494213 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.064677 10011 tablet_replica.cc:333] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.065152 10011 raft_consensus.cc:2243] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:11:50.065956 10011 raft_consensus.cc:2272] T 596715d7c34e4297b4cf0287d1ee5861 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.067589 10011 tablet_replica.cc:333] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.068091 10011 raft_consensus.cc:2243] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:11:50.068821 10011 raft_consensus.cc:2272] T b29c191985ea4b53a1c1e6cde36f44cf P bcb0169f1fa94e1096ecb4e038005ab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.070706 10011 tablet_replica.cc:333] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.071214 10011 raft_consensus.cc:2243] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:50.071628 10011 raft_consensus.cc:2272] T 9d162003b66343d791f60ebb8c4f306f P bcb0169f1fa94e1096ecb4e038005ab2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.073558 10011 tablet_replica.cc:333] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.074585 10011 raft_consensus.cc:2243] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:11:50.075429 10011 raft_consensus.cc:2272] T c5e45ab4fb5247e48fd8e36318e08482 P bcb0169f1fa94e1096ecb4e038005ab2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.077042 10011 tablet_replica.cc:333] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2: stopping tablet replica
I20260709 12:11:50.077498 10011 raft_consensus.cc:2243] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:50.077937 10011 raft_consensus.cc:2272] T 6dbc30b646ab4fdfae2f48b8ed780522 P bcb0169f1fa94e1096ecb4e038005ab2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.106861 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:11:50.126906 10011 master.cc:562] Master@127.9.198.254:46581 shutting down...
I20260709 12:11:50.150003 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:11:50.150601 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:50.150993 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P 90667bb7730e49e9b2a5e7d6817ac4f3: stopping tablet replica
I20260709 12:11:50.172291 10011 master.cc:584] Master@127.9.198.254:46581 shutdown complete.
5602bb38ed70475da6831345a8d07478  4
bcb0169f1fa94e1096ecb4e038005ab2  2
d844c1ce274a4f1cadebcf807f750cca  4
5602bb38ed70475da6831345a8d07478  3
bcb0169f1fa94e1096ecb4e038005ab2  3
d844c1ce274a4f1cadebcf807f750cca  4
5602bb38ed70475da6831345a8d07478  8
bcb0169f1fa94e1096ecb4e038005ab2  1
d844c1ce274a4f1cadebcf807f750cca  1
5602bb38ed70475da6831345a8d07478  4
bcb0169f1fa94e1096ecb4e038005ab2  3
d844c1ce274a4f1cadebcf807f750cca  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28851 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:11:50.214638 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:36219
I20260709 12:11:50.215845 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:50.222218 11055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:50.222244 11054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:50.223146 11057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:50.223383 10011 server_base.cc:1061] running on GCE node
I20260709 12:11:50.224364 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:50.224543 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:50.224653 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599110224642 us; error 0 us; skew 500 ppm
I20260709 12:11:50.225178 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:50.227897 10011 webserver.cc:533] Webserver started at http://127.9.198.254:40577/ using document root <none> and password file <none>
I20260709 12:11:50.228345 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:50.228499 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:50.228749 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:50.230232 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "bbfb3415e272477dac9ba599cc40a89d"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.234819 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:11:50.238515 11062 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.239322 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:11:50.239578 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "bbfb3415e272477dac9ba599cc40a89d"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.239848 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:50.259495 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:50.260800 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:50.300199 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:36219
I20260709 12:11:50.300294 11113 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:36219 every 8 connection(s)
I20260709 12:11:50.304198 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:50.314350 11114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Bootstrap starting.
I20260709 12:11:50.319025 11114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:50.323668 11114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: No bootstrap required, opened a new log
I20260709 12:11:50.325850 11114 raft_consensus.cc:359] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbfb3415e272477dac9ba599cc40a89d" member_type: VOTER }
I20260709 12:11:50.326224 11114 raft_consensus.cc:385] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:50.326403 11114 raft_consensus.cc:740] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbfb3415e272477dac9ba599cc40a89d, State: Initialized, Role: FOLLOWER
I20260709 12:11:50.326903 11114 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbfb3415e272477dac9ba599cc40a89d" member_type: VOTER }
I20260709 12:11:50.327322 11114 raft_consensus.cc:399] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:11:50.327545 11114 raft_consensus.cc:493] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:11:50.327752 11114 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:50.331921 11114 raft_consensus.cc:515] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbfb3415e272477dac9ba599cc40a89d" member_type: VOTER }
I20260709 12:11:50.332402 11114 leader_election.cc:304] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbfb3415e272477dac9ba599cc40a89d; no voters: 
I20260709 12:11:50.333504 11114 leader_election.cc:290] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:11:50.333922 11117 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:50.335268 11117 raft_consensus.cc:697] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 1 LEADER]: Becoming Leader. State: Replica: bbfb3415e272477dac9ba599cc40a89d, State: Running, Role: LEADER
I20260709 12:11:50.335997 11117 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbfb3415e272477dac9ba599cc40a89d" member_type: VOTER }
I20260709 12:11:50.336601 11114 sys_catalog.cc:565] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:11:50.340768 11118 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbfb3415e272477dac9ba599cc40a89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbfb3415e272477dac9ba599cc40a89d" member_type: VOTER } }
I20260709 12:11:50.341276 11119 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbfb3415e272477dac9ba599cc40a89d. Latest consensus state: current_term: 1 leader_uuid: "bbfb3415e272477dac9ba599cc40a89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbfb3415e272477dac9ba599cc40a89d" member_type: VOTER } }
I20260709 12:11:50.341651 11118 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:50.342073 11119 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:50.345043 11124 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:11:50.350117 11124 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:11:50.353930 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:11:50.359524 11124 catalog_manager.cc:1383] Generated new cluster ID: 668ff0dc0b8445cf9c22b5ca487f5409
I20260709 12:11:50.359838 11124 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:11:50.400753 11124 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:11:50.402458 11124 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:11:50.421434 11124 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Generated new TSK 0
I20260709 12:11:50.422171 11124 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:11:50.491282 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:50.499202 11136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:50.500452 11137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:50.503391 11139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:50.503448 10011 server_base.cc:1061] running on GCE node
I20260709 12:11:50.504922 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:50.505185 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:50.505388 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599110505361 us; error 0 us; skew 500 ppm
I20260709 12:11:50.506042 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:50.509148 10011 webserver.cc:533] Webserver started at http://127.9.198.193:35863/ using document root <none> and password file <none>
I20260709 12:11:50.509855 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:50.510093 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:50.510442 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:50.511912 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "016411114c6d42039bd04025378dfc98"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.518057 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 12:11:50.522109 11144 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.523011 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:11:50.523391 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "016411114c6d42039bd04025378dfc98"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.523777 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:50.565986 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:50.567328 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:50.568961 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:50.571686 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:50.571938 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.572198 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:50.572398 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.616762 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:45115
I20260709 12:11:50.616879 11206 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:45115 every 8 connection(s)
I20260709 12:11:50.622411 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:50.630187 11211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:50.631837 11212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:50.637943 11214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:50.641393 10011 server_base.cc:1061] running on GCE node
I20260709 12:11:50.642726 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:50.642994 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:50.643019 11207 heartbeater.cc:344] Connected to a master server at 127.9.198.254:36219
I20260709 12:11:50.643283 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599110643254 us; error 0 us; skew 500 ppm
I20260709 12:11:50.643528 11207 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:50.644212 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:50.644444 11207 heartbeater.cc:507] Master 127.9.198.254:36219 requested a full tablet report, sending...
I20260709 12:11:50.647243 10011 webserver.cc:533] Webserver started at http://127.9.198.194:43503/ using document root <none> and password file <none>
I20260709 12:11:50.647228 11079 ts_manager.cc:194] Registered new tserver with Master: 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:11:50.648021 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:50.648311 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:50.648605 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:50.649799 11079 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50196
I20260709 12:11:50.650007 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "fecfc03de52a4308bc9b045cad5aac1e"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.655110 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:11:50.658857 11219 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.659705 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:11:50.660063 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "fecfc03de52a4308bc9b045cad5aac1e"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.660360 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:50.671736 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:50.673049 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:50.674813 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:50.677232 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:50.677470 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.677740 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:50.677944 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.719499 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:37243
I20260709 12:11:50.719626 11281 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:37243 every 8 connection(s)
I20260709 12:11:50.724341 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:50.732693 11285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:50.734962 11286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:50.738997 10011 server_base.cc:1061] running on GCE node
W20260709 12:11:50.741132 11289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:50.741858 11282 heartbeater.cc:344] Connected to a master server at 127.9.198.254:36219
I20260709 12:11:50.742269 11282 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:50.742414 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:50.742666 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:50.742867 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599110742849 us; error 0 us; skew 500 ppm
I20260709 12:11:50.743163 11282 heartbeater.cc:507] Master 127.9.198.254:36219 requested a full tablet report, sending...
I20260709 12:11:50.743646 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:50.745529 11079 ts_manager.cc:194] Registered new tserver with Master: fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:50.746733 10011 webserver.cc:533] Webserver started at http://127.9.198.195:40077/ using document root <none> and password file <none>
I20260709 12:11:50.747139 11079 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50208
I20260709 12:11:50.747298 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:50.747637 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:50.747959 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:50.749251 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.753814 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:11:50.757370 11293 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.758343 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:11:50.758665 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
format_stamp: "Formatted at 2026-07-09 12:11:50 on dist-test-slave-glnm"
I20260709 12:11:50.759027 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:50.793568 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:50.795094 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:50.796734 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:50.799227 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:50.799506 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.799794 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:50.799993 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:50.847587 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:42967
I20260709 12:11:50.847811 11355 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:42967 every 8 connection(s)
I20260709 12:11:50.862672 11356 heartbeater.cc:344] Connected to a master server at 127.9.198.254:36219
I20260709 12:11:50.863214 11356 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:50.864125 11356 heartbeater.cc:507] Master 127.9.198.254:36219 requested a full tablet report, sending...
I20260709 12:11:50.866230 11079 ts_manager.cc:194] Registered new tserver with Master: bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:50.866508 10011 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015829879s
I20260709 12:11:50.868108 11079 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50220
I20260709 12:11:50.894980 11079 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50226:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:11:50.901306 11079 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:11:51.109019 11246 tablet_service.cc:1511] Processing CreateTablet for tablet 25983a67e23a4c708ae496061b992207 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:11:51.109395 11247 tablet_service.cc:1511] Processing CreateTablet for tablet 01e75f61a948459a884630690e51c686 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:11:51.110703 11246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25983a67e23a4c708ae496061b992207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.111796 11247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e75f61a948459a884630690e51c686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.112666 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 9424adcaacdf40f3bbd2f639ef2a48c2 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:11:51.114176 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9424adcaacdf40f3bbd2f639ef2a48c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.115625 11171 tablet_service.cc:1511] Processing CreateTablet for tablet 25983a67e23a4c708ae496061b992207 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:11:51.117077 11171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25983a67e23a4c708ae496061b992207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.117115 11243 tablet_service.cc:1511] Processing CreateTablet for tablet 6d225f8a873c4abe992018eb9353664f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:11:51.118643 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d225f8a873c4abe992018eb9353664f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.118871 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 9424adcaacdf40f3bbd2f639ef2a48c2 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:11:51.120216 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9424adcaacdf40f3bbd2f639ef2a48c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.122861 11169 tablet_service.cc:1511] Processing CreateTablet for tablet 28d2d114b03e403c83f7001bc117434e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:11:51.124413 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28d2d114b03e403c83f7001bc117434e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.124264 11244 tablet_service.cc:1511] Processing CreateTablet for tablet 28d2d114b03e403c83f7001bc117434e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:11:51.125622 11244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28d2d114b03e403c83f7001bc117434e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.126139 11168 tablet_service.cc:1511] Processing CreateTablet for tablet 6d225f8a873c4abe992018eb9353664f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:11:51.127526 11168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d225f8a873c4abe992018eb9353664f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.130897 11167 tablet_service.cc:1511] Processing CreateTablet for tablet d1db2221cc614cac91fdaa49f8d6cb61 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:11:51.132339 11167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1db2221cc614cac91fdaa49f8d6cb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.134099 11240 tablet_service.cc:1511] Processing CreateTablet for tablet e26c5b0a79b0482cb39b479148fba412 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:11:51.135697 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26c5b0a79b0482cb39b479148fba412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.136543 11165 tablet_service.cc:1511] Processing CreateTablet for tablet e26c5b0a79b0482cb39b479148fba412 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:11:51.137768 11164 tablet_service.cc:1511] Processing CreateTablet for tablet d97e23379bda4a1b9ad1e2eabbf4d82f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:11:51.139128 11164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97e23379bda4a1b9ad1e2eabbf4d82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.138954 11241 tablet_service.cc:1511] Processing CreateTablet for tablet 99332ff05c3e46579ef9ff825e42a628 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:11:51.113785 11172 tablet_service.cc:1511] Processing CreateTablet for tablet 01e75f61a948459a884630690e51c686 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:11:51.122864 11242 tablet_service.cc:1511] Processing CreateTablet for tablet d1db2221cc614cac91fdaa49f8d6cb61 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:11:51.145610 11239 tablet_service.cc:1511] Processing CreateTablet for tablet d97e23379bda4a1b9ad1e2eabbf4d82f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:11:51.146261 11163 tablet_service.cc:1511] Processing CreateTablet for tablet c21b703f1a4a445a8505bb01530a8439 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:11:51.148013 11238 tablet_service.cc:1511] Processing CreateTablet for tablet c21b703f1a4a445a8505bb01530a8439 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:11:51.150265 11166 tablet_service.cc:1511] Processing CreateTablet for tablet 99332ff05c3e46579ef9ff825e42a628 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:11:51.151924 11165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26c5b0a79b0482cb39b479148fba412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.152699 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21b703f1a4a445a8505bb01530a8439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.153417 11241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99332ff05c3e46579ef9ff825e42a628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.153677 11166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99332ff05c3e46579ef9ff825e42a628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.154181 11238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21b703f1a4a445a8505bb01530a8439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.154901 11239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97e23379bda4a1b9ad1e2eabbf4d82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.167927 11316 tablet_service.cc:1511] Processing CreateTablet for tablet d1db2221cc614cac91fdaa49f8d6cb61 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:11:51.169133 11321 tablet_service.cc:1511] Processing CreateTablet for tablet 01e75f61a948459a884630690e51c686 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:11:51.170974 11315 tablet_service.cc:1511] Processing CreateTablet for tablet 99332ff05c3e46579ef9ff825e42a628 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:11:51.171607 11320 tablet_service.cc:1511] Processing CreateTablet for tablet 25983a67e23a4c708ae496061b992207 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:11:51.174467 11242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1db2221cc614cac91fdaa49f8d6cb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.174572 11319 tablet_service.cc:1511] Processing CreateTablet for tablet 9424adcaacdf40f3bbd2f639ef2a48c2 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:11:51.176949 11318 tablet_service.cc:1511] Processing CreateTablet for tablet 28d2d114b03e403c83f7001bc117434e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:11:51.179396 11317 tablet_service.cc:1511] Processing CreateTablet for tablet 6d225f8a873c4abe992018eb9353664f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:11:51.190055 11314 tablet_service.cc:1511] Processing CreateTablet for tablet e26c5b0a79b0482cb39b479148fba412 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:11:51.190222 11313 tablet_service.cc:1511] Processing CreateTablet for tablet d97e23379bda4a1b9ad1e2eabbf4d82f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:11:51.169521 11316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1db2221cc614cac91fdaa49f8d6cb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.198318 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26c5b0a79b0482cb39b479148fba412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.202102 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25983a67e23a4c708ae496061b992207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.202840 11313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97e23379bda4a1b9ad1e2eabbf4d82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.206044 11321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e75f61a948459a884630690e51c686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.206234 11172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01e75f61a948459a884630690e51c686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.215157 11312 tablet_service.cc:1511] Processing CreateTablet for tablet c21b703f1a4a445a8505bb01530a8439 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:11:51.217973 11315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99332ff05c3e46579ef9ff825e42a628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.222056 11312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21b703f1a4a445a8505bb01530a8439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.226016 11319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9424adcaacdf40f3bbd2f639ef2a48c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.230000 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28d2d114b03e403c83f7001bc117434e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.234035 11317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d225f8a873c4abe992018eb9353664f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.251104 11168 tablet_service.cc:1511] Processing CreateTablet for tablet 73b50321123c433f91c5d0972791cf37 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:11:51.253046 11168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73b50321123c433f91c5d0972791cf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.286434 11164 tablet_service.cc:1511] Processing CreateTablet for tablet e5f804d45235415fad0f1442ea04aa47 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:11:51.287919 11164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f804d45235415fad0f1442ea04aa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.297300 11375 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:51.303109 11166 tablet_service.cc:1511] Processing CreateTablet for tablet a629d78b61074c2cbd007926bd78fa29 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:11:51.304581 11166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a629d78b61074c2cbd007926bd78fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.310708 11374 tablet_bootstrap.cc:492] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:51.310787 11247 tablet_service.cc:1511] Processing CreateTablet for tablet 73b50321123c433f91c5d0972791cf37 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:11:51.312690 11170 tablet_service.cc:1511] Processing CreateTablet for tablet d5a029ce312749bdaf2113d2823e993e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:11:51.313385 11247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73b50321123c433f91c5d0972791cf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.314011 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5a029ce312749bdaf2113d2823e993e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.324594 11246 tablet_service.cc:1511] Processing CreateTablet for tablet e5f804d45235415fad0f1442ea04aa47 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:11:51.339020 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 88ddcb89ab6d47cd97e8c19734f0e365 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:11:51.340571 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ddcb89ab6d47cd97e8c19734f0e365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.341509 11244 tablet_service.cc:1511] Processing CreateTablet for tablet a629d78b61074c2cbd007926bd78fa29 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
W20260709 12:11:51.349471 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:51.350353 11246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f804d45235415fad0f1442ea04aa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.353745 11244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a629d78b61074c2cbd007926bd78fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.366951 11374 tablet_bootstrap.cc:654] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:51.367113 11247 tablet_service.cc:1511] Processing CreateTablet for tablet 88ddcb89ab6d47cd97e8c19734f0e365 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:11:51.368440 11375 tablet_bootstrap.cc:654] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:51.368898 11247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ddcb89ab6d47cd97e8c19734f0e365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.373566 11317 tablet_service.cc:1511] Processing CreateTablet for tablet 73b50321123c433f91c5d0972791cf37 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:11:51.379746 11241 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9c5be972b3443eb3211fc2eb29d575 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:11:51.382611 11243 tablet_service.cc:1511] Processing CreateTablet for tablet 00b8260131ab49ecb2df9631e5704096 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:11:51.383708 11240 tablet_service.cc:1511] Processing CreateTablet for tablet bbf57c68621b4e90a77c76eb3974b80e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:11:51.389076 11317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73b50321123c433f91c5d0972791cf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.401919 11170 tablet_service.cc:1511] Processing CreateTablet for tablet bbf57c68621b4e90a77c76eb3974b80e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:11:51.389991 11320 tablet_service.cc:1511] Processing CreateTablet for tablet e5f804d45235415fad0f1442ea04aa47 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:11:51.406589 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f804d45235415fad0f1442ea04aa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.408452 11244 tablet_service.cc:1511] Processing CreateTablet for tablet dedbfb6dda024d1e81b2740f108080af (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:11:51.399839 11312 tablet_service.cc:1511] Processing CreateTablet for tablet a629d78b61074c2cbd007926bd78fa29 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:11:51.409922 11312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a629d78b61074c2cbd007926bd78fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.392714 11171 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9c5be972b3443eb3211fc2eb29d575 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:11:51.410717 11238 tablet_service.cc:1511] Processing CreateTablet for tablet 35c7899b1a314ae1b67824dfee6ff091 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:11:51.411407 11171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9c5be972b3443eb3211fc2eb29d575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.409932 11244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dedbfb6dda024d1e81b2740f108080af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.402262 11245 tablet_service.cc:1511] Processing CreateTablet for tablet d5a029ce312749bdaf2113d2823e993e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:11:51.414536 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5a029ce312749bdaf2113d2823e993e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.420409 11165 tablet_service.cc:1511] Processing CreateTablet for tablet dedbfb6dda024d1e81b2740f108080af (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:11:51.423403 11169 tablet_service.cc:1511] Processing CreateTablet for tablet 35c7899b1a314ae1b67824dfee6ff091 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:11:51.430078 11319 tablet_service.cc:1511] Processing CreateTablet for tablet d5a029ce312749bdaf2113d2823e993e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:11:51.431370 11319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5a029ce312749bdaf2113d2823e993e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.433468 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 88ddcb89ab6d47cd97e8c19734f0e365 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:11:51.434257 11163 tablet_service.cc:1511] Processing CreateTablet for tablet 00b8260131ab49ecb2df9631e5704096 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:11:51.434803 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ddcb89ab6d47cd97e8c19734f0e365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.435055 11172 tablet_service.cc:1511] Processing CreateTablet for tablet 38d626794db04be2b9c36cc76fcf2183 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:11:51.435945 11238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35c7899b1a314ae1b67824dfee6ff091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.437500 11316 tablet_service.cc:1511] Processing CreateTablet for tablet 1f9c5be972b3443eb3211fc2eb29d575 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:11:51.438798 11316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9c5be972b3443eb3211fc2eb29d575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.441777 11321 tablet_service.cc:1511] Processing CreateTablet for tablet 00b8260131ab49ecb2df9631e5704096 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:11:51.428531 11165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dedbfb6dda024d1e81b2740f108080af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.447991 11315 tablet_service.cc:1511] Processing CreateTablet for tablet bbf57c68621b4e90a77c76eb3974b80e (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:11:51.452425 11242 tablet_service.cc:1511] Processing CreateTablet for tablet 38d626794db04be2b9c36cc76fcf2183 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:11:51.455899 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b8260131ab49ecb2df9631e5704096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.456622 11242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d626794db04be2b9c36cc76fcf2183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.461253 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf57c68621b4e90a77c76eb3974b80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.470968 11376 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:51.474054 11374 tablet_bootstrap.cc:492] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:51.474611 11374 ts_tablet_manager.cc:1403] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.164s	user 0.009s	sys 0.005s
I20260709 12:11:51.478009 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35c7899b1a314ae1b67824dfee6ff091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.478497 11374 raft_consensus.cc:359] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.479460 11374 raft_consensus.cc:385] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:51.479785 11374 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:51.480847 11374 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.482000 11241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9c5be972b3443eb3211fc2eb29d575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.486182 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf57c68621b4e90a77c76eb3974b80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.487578 11168 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6a39d516e943a9b2c8c4213ac11902 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:11:51.493104 11168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6a39d516e943a9b2c8c4213ac11902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.492977 11313 tablet_service.cc:1511] Processing CreateTablet for tablet dedbfb6dda024d1e81b2740f108080af (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:11:51.493809 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b8260131ab49ecb2df9631e5704096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.494426 11313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dedbfb6dda024d1e81b2740f108080af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.498754 11315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf57c68621b4e90a77c76eb3974b80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.498721 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6a39d516e943a9b2c8c4213ac11902 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:11:51.500411 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6a39d516e943a9b2c8c4213ac11902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.501971 11172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d626794db04be2b9c36cc76fcf2183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.507962 11312 tablet_service.cc:1511] Processing CreateTablet for tablet 35c7899b1a314ae1b67824dfee6ff091 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:11:51.509375 11312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35c7899b1a314ae1b67824dfee6ff091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.532630 11375 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:51.533202 11375 ts_tablet_manager.cc:1403] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.236s	user 0.012s	sys 0.004s
I20260709 12:11:51.536248 11376 tablet_bootstrap.cc:654] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:51.537096 11375 raft_consensus.cc:359] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.541270 11282 heartbeater.cc:499] Master 127.9.198.254:36219 was elected leader, sending a full tablet report...
I20260709 12:11:51.554347 11374 ts_tablet_manager.cc:1434] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.079s	user 0.019s	sys 0.010s
I20260709 12:11:51.555418 11374 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:51.556345 11321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b8260131ab49ecb2df9631e5704096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.558112 11317 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6a39d516e943a9b2c8c4213ac11902 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:11:51.538758 11375 raft_consensus.cc:385] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:51.559226 11375 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:51.562875 11317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6a39d516e943a9b2c8c4213ac11902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.584295 11374 tablet_bootstrap.cc:654] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:51.596088 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd765747f9648049b905c70696b206d (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:11:51.596868 11313 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd765747f9648049b905c70696b206d (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:11:51.597752 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd765747f9648049b905c70696b206d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.598176 11313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd765747f9648049b905c70696b206d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.601610 11240 tablet_service.cc:1511] Processing CreateTablet for tablet 02e478aac72b4a62a8ed9124187f2dec (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:11:51.602682 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 02e478aac72b4a62a8ed9124187f2dec (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:11:51.603885 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e478aac72b4a62a8ed9124187f2dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.560097 11375 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.615407 11172 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb796253655415ba47e989105d7c807 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:11:51.610697 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e478aac72b4a62a8ed9124187f2dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.620394 11318 tablet_service.cc:1511] Processing CreateTablet for tablet 38d626794db04be2b9c36cc76fcf2183 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:11:51.620858 11172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb796253655415ba47e989105d7c807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.620777 11165 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd765747f9648049b905c70696b206d (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:11:51.627245 11241 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb796253655415ba47e989105d7c807 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:11:51.628672 11241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb796253655415ba47e989105d7c807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.629902 11315 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb796253655415ba47e989105d7c807 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:11:51.628314 11163 tablet_service.cc:1511] Processing CreateTablet for tablet 80d10b7c9e4644e9a03edf10767df625 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:11:51.632505 11376 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:51.633035 11376 ts_tablet_manager.cc:1403] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.162s	user 0.012s	sys 0.000s
I20260709 12:11:51.635993 11376 raft_consensus.cc:359] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.636719 11376 raft_consensus.cc:385] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:51.637044 11376 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:51.637921 11315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb796253655415ba47e989105d7c807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.639691 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d626794db04be2b9c36cc76fcf2183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.639798 11168 tablet_service.cc:1511] Processing CreateTablet for tablet 02e478aac72b4a62a8ed9124187f2dec (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:11:51.652695 11207 heartbeater.cc:499] Master 127.9.198.254:36219 was elected leader, sending a full tablet report...
I20260709 12:11:51.655223 11243 tablet_service.cc:1511] Processing CreateTablet for tablet 80d10b7c9e4644e9a03edf10767df625 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:11:51.656553 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d10b7c9e4644e9a03edf10767df625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.659713 11166 tablet_service.cc:1511] Processing CreateTablet for tablet 35bce44584bf45b68e6ff125c8f30655 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:11:51.661337 11247 tablet_service.cc:1511] Processing CreateTablet for tablet 35bce44584bf45b68e6ff125c8f30655 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:11:51.662631 11247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35bce44584bf45b68e6ff125c8f30655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.668443 11165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd765747f9648049b905c70696b206d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.669932 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d10b7c9e4644e9a03edf10767df625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.671110 11168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e478aac72b4a62a8ed9124187f2dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.673969 11166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35bce44584bf45b68e6ff125c8f30655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.670338 11376 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.687608 11320 tablet_service.cc:1511] Processing CreateTablet for tablet 35bce44584bf45b68e6ff125c8f30655 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:11:51.688985 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35bce44584bf45b68e6ff125c8f30655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.697271 11375 ts_tablet_manager.cc:1434] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.164s	user 0.007s	sys 0.006s
I20260709 12:11:51.698506 11375 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:51.708770 11316 tablet_service.cc:1511] Processing CreateTablet for tablet 80d10b7c9e4644e9a03edf10767df625 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:11:51.710294 11316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d10b7c9e4644e9a03edf10767df625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.719339 11374 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:51.720292 11374 ts_tablet_manager.cc:1403] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.165s	user 0.020s	sys 0.021s
I20260709 12:11:51.727867 11374 raft_consensus.cc:359] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.728598 11374 raft_consensus.cc:385] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:51.728914 11374 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:51.733588 11375 tablet_bootstrap.cc:654] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:51.751793 11066 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64685.9us
I20260709 12:11:51.752686 11376 ts_tablet_manager.cc:1434] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.119s	user 0.018s	sys 0.022s
I20260709 12:11:51.754237 11356 heartbeater.cc:499] Master 127.9.198.254:36219 was elected leader, sending a full tablet report...
W20260709 12:11:51.750895 11065 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57798.5us
I20260709 12:11:51.758351 11168 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad54acf641349bc8119ce4c0eae446b (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:11:51.759697 11168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad54acf641349bc8119ce4c0eae446b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.751207 11374 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.770555 11376 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:51.779326 11374 ts_tablet_manager.cc:1434] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 12:11:51.780479 11374 tablet_bootstrap.cc:492] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:51.800616 11318 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad54acf641349bc8119ce4c0eae446b (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:11:51.802104 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad54acf641349bc8119ce4c0eae446b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.801893 11163 tablet_service.cc:1511] Processing CreateTablet for tablet 4958b19b61094e1db763fdb94852f994 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:11:51.803160 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4958b19b61094e1db763fdb94852f994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.803318 11374 tablet_bootstrap.cc:654] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:51.804622 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad54acf641349bc8119ce4c0eae446b (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:11:51.805938 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad54acf641349bc8119ce4c0eae446b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.811882 11376 tablet_bootstrap.cc:654] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:51.824904 11315 tablet_service.cc:1511] Processing CreateTablet for tablet 4958b19b61094e1db763fdb94852f994 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:11:51.826416 11315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4958b19b61094e1db763fdb94852f994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.826893 11240 tablet_service.cc:1511] Processing CreateTablet for tablet 4958b19b61094e1db763fdb94852f994 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:11:51.828277 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4958b19b61094e1db763fdb94852f994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.831056 11243 tablet_service.cc:1511] Processing CreateTablet for tablet aa8154ca19364187a13db1be3e0b4f53 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:11:51.831115 11165 tablet_service.cc:1511] Processing CreateTablet for tablet 5df07764623640ce8ec29ad695812b7c (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:11:51.832546 11165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df07764623640ce8ec29ad695812b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.838106 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa8154ca19364187a13db1be3e0b4f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.841150 11166 tablet_service.cc:1511] Processing CreateTablet for tablet aa8154ca19364187a13db1be3e0b4f53 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:11:51.842613 11166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa8154ca19364187a13db1be3e0b4f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.853741 11379 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.854417 11379 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.858572 11172 tablet_service.cc:1511] Processing CreateTablet for tablet 7f736e38db6b45bfa87e35af564bffc1 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:11:51.858904 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 4a634acd9ff543cca551c8a28e72d6c4 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:11:51.859902 11172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f736e38db6b45bfa87e35af564bffc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.860572 11241 tablet_service.cc:1511] Processing CreateTablet for tablet 4a634acd9ff543cca551c8a28e72d6c4 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:11:51.853766 11247 tablet_service.cc:1511] Processing CreateTablet for tablet 5df07764623640ce8ec29ad695812b7c (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:11:51.862880 11241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a634acd9ff543cca551c8a28e72d6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.864970 11313 tablet_service.cc:1511] Processing CreateTablet for tablet 4a634acd9ff543cca551c8a28e72d6c4 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:11:51.866333 11313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a634acd9ff543cca551c8a28e72d6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.870071 11247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df07764623640ce8ec29ad695812b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.870265 11316 tablet_service.cc:1511] Processing CreateTablet for tablet aa8154ca19364187a13db1be3e0b4f53 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:11:51.871625 11316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa8154ca19364187a13db1be3e0b4f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.872846 11244 tablet_service.cc:1511] Processing CreateTablet for tablet 7f736e38db6b45bfa87e35af564bffc1 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:11:51.873915 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a634acd9ff543cca551c8a28e72d6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.874179 11244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f736e38db6b45bfa87e35af564bffc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.888264 11312 tablet_service.cc:1511] Processing CreateTablet for tablet 5df07764623640ce8ec29ad695812b7c (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:11:51.881206 11379 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:51.894315 11312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df07764623640ce8ec29ad695812b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.895159 11375 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:51.895709 11375 ts_tablet_manager.cc:1403] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.197s	user 0.008s	sys 0.015s
I20260709 12:11:51.898394 11376 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:51.898952 11376 ts_tablet_manager.cc:1403] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.144s	user 0.019s	sys 0.006s
I20260709 12:11:51.923094 11374 tablet_bootstrap.cc:492] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:51.923173 11375 raft_consensus.cc:359] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.923210 11246 tablet_service.cc:1511] Processing CreateTablet for tablet f62903fc48c140aca4aa949625fa88f6 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:11:51.923902 11374 ts_tablet_manager.cc:1403] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.144s	user 0.012s	sys 0.007s
I20260709 12:11:51.924093 11375 raft_consensus.cc:385] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:51.924515 11375 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:51.924829 11246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62903fc48c140aca4aa949625fa88f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.927268 11374 raft_consensus.cc:359] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.928115 11374 raft_consensus.cc:385] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:51.928467 11374 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:51.929725 11374 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.929839 11375 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.933578 11315 tablet_service.cc:1511] Processing CreateTablet for tablet 7f736e38db6b45bfa87e35af564bffc1 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:11:51.935014 11315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f736e38db6b45bfa87e35af564bffc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.935091 11383 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.935648 11383 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.936350 11375 ts_tablet_manager.cc:1434] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.002s
I20260709 12:11:51.937350 11375 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:51.919448 11376 raft_consensus.cc:359] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.938427 11376 raft_consensus.cc:385] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:51.938766 11376 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:51.940845 11242 tablet_service.cc:1511] Processing CreateTablet for tablet 3e235bf9f3514f629a5c26c1eac58682 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:11:51.942266 11242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e235bf9f3514f629a5c26c1eac58682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.942312 11383 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:51.944793 11375 tablet_bootstrap.cc:654] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:51.939672 11376 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:51.956262 11320 tablet_service.cc:1511] Processing CreateTablet for tablet 3e235bf9f3514f629a5c26c1eac58682 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:11:51.956983 11376 ts_tablet_manager.cc:1434] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 12:11:51.957803 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e235bf9f3514f629a5c26c1eac58682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.958065 11376 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:51.961293 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 4c40179116e14aa2aff8e95298cbcd4f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:11:51.962648 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c40179116e14aa2aff8e95298cbcd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.978914 11170 tablet_service.cc:1511] Processing CreateTablet for tablet f62903fc48c140aca4aa949625fa88f6 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:11:51.980309 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62903fc48c140aca4aa949625fa88f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.983090 11163 tablet_service.cc:1511] Processing CreateTablet for tablet 3e235bf9f3514f629a5c26c1eac58682 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:11:51.985934 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e235bf9f3514f629a5c26c1eac58682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.982528 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 06a9ec0400af47eeaffc85f3f1d38de4 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:11:51.983392 11168 tablet_service.cc:1511] Processing CreateTablet for tablet 4c40179116e14aa2aff8e95298cbcd4f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:11:51.990716 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06a9ec0400af47eeaffc85f3f1d38de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.990945 11168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c40179116e14aa2aff8e95298cbcd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.995266 11165 tablet_service.cc:1511] Processing CreateTablet for tablet 06a9ec0400af47eeaffc85f3f1d38de4 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:11:51.996613 11165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06a9ec0400af47eeaffc85f3f1d38de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:51.998976 11379 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.999485 11379 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.001572 11379 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.009801 11313 tablet_service.cc:1511] Processing CreateTablet for tablet f62903fc48c140aca4aa949625fa88f6 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:11:52.011133 11313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62903fc48c140aca4aa949625fa88f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.017529 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.021150 11321 tablet_service.cc:1511] Processing CreateTablet for tablet 06a9ec0400af47eeaffc85f3f1d38de4 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:11:52.022540 11321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06a9ec0400af47eeaffc85f3f1d38de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.022883 11374 ts_tablet_manager.cc:1434] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.000s
I20260709 12:11:52.029649 11382 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.030262 11382 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.038295 11240 tablet_service.cc:1511] Processing CreateTablet for tablet 4c40179116e14aa2aff8e95298cbcd4f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:11:52.039705 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c40179116e14aa2aff8e95298cbcd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.046147 11382 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:52.055548 11222 leader_election.cc:343] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.055606 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.047657 11376 tablet_bootstrap.cc:654] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.058267 11374 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
W20260709 12:11:52.066183 11222 leader_election.cc:343] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.066368 11379 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.071503 11379 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.072153 11172 tablet_service.cc:1511] Processing CreateTablet for tablet 16e4ceea03b94eb2803569189ab5fcca (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:11:52.073450 11172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e4ceea03b94eb2803569189ab5fcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.083456 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.083258 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 53e6b498179b4c96b8b0be6240a36efb (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:11:52.084673 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e6b498179b4c96b8b0be6240a36efb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:11:52.084945 11222 leader_election.cc:343] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.093678 11379 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.102339 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:52.104333 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:52.117825 11223 leader_election.cc:343] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.118459 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:11:52.119840 11223 leader_election.cc:343] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.120304 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:52.120651 11379 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.123301 11379 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.123831 11242 tablet_service.cc:1511] Processing CreateTablet for tablet 16e4ceea03b94eb2803569189ab5fcca (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:11:52.125159 11242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e4ceea03b94eb2803569189ab5fcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.114717 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:52.126084 11330 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:52.129464 11169 tablet_service.cc:1511] Processing CreateTablet for tablet 6f62270698314885bc0e03e0de4507ae (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:11:52.130826 11316 tablet_service.cc:1511] Processing CreateTablet for tablet 6f62270698314885bc0e03e0de4507ae (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:11:52.132122 11316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f62270698314885bc0e03e0de4507ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.132274 11171 tablet_service.cc:1511] Processing CreateTablet for tablet d49a939bed404bfcb2538b5331071d3b (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:11:52.130820 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f62270698314885bc0e03e0de4507ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.134498 11374 tablet_bootstrap.cc:654] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.141759 11171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d49a939bed404bfcb2538b5331071d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.143471 11243 tablet_service.cc:1511] Processing CreateTablet for tablet 6f62270698314885bc0e03e0de4507ae (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:11:52.144776 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f62270698314885bc0e03e0de4507ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.145555 11247 tablet_service.cc:1511] Processing CreateTablet for tablet 53e6b498179b4c96b8b0be6240a36efb (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:11:52.146839 11247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e6b498179b4c96b8b0be6240a36efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.147042 11317 tablet_service.cc:1511] Processing CreateTablet for tablet d49a939bed404bfcb2538b5331071d3b (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:11:52.148326 11317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d49a939bed404bfcb2538b5331071d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.149623 11241 tablet_service.cc:1511] Processing CreateTablet for tablet d49a939bed404bfcb2538b5331071d3b (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:11:52.151065 11241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d49a939bed404bfcb2538b5331071d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.158228 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:52.159947 11147 leader_election.cc:343] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.164961 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.178341 11170 tablet_service.cc:1511] Processing CreateTablet for tablet fab915731e0347ae8353a7f300c44401 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:11:52.179857 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab915731e0347ae8353a7f300c44401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.194475 11320 tablet_service.cc:1511] Processing CreateTablet for tablet 53e6b498179b4c96b8b0be6240a36efb (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:11:52.196877 11314 tablet_service.cc:1511] Processing CreateTablet for tablet fab915731e0347ae8353a7f300c44401 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:11:52.198324 11318 tablet_service.cc:1511] Processing CreateTablet for tablet 16e4ceea03b94eb2803569189ab5fcca (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:11:52.200335 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:52.204634 11148 leader_election.cc:343] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.196913 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e6b498179b4c96b8b0be6240a36efb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:11:52.206144 11296 leader_election.cc:343] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.207370 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e4ceea03b94eb2803569189ab5fcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.210095 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:52.210920 11382 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.211130 11245 tablet_service.cc:1511] Processing CreateTablet for tablet fab915731e0347ae8353a7f300c44401 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:11:52.211423 11382 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.212440 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab915731e0347ae8353a7f300c44401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.211591 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab915731e0347ae8353a7f300c44401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.214881 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:52.216286 11147 leader_election.cc:343] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:52.187090 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 67268us
I20260709 12:11:52.222998 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:52.225467 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:52.226430 11148 leader_election.cc:343] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.213523 11382 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.226248 11163 tablet_service.cc:1511] Processing CreateTablet for tablet ddf8cca10e1244d8882abd13fa81113a (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:11:52.227031 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:52.227815 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddf8cca10e1244d8882abd13fa81113a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.230170 11376 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:52.230643 11376 ts_tablet_manager.cc:1403] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.273s	user 0.015s	sys 0.002s
I20260709 12:11:52.233748 11382 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.233601 11376 raft_consensus.cc:359] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.234409 11376 raft_consensus.cc:385] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.234691 11376 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.235399 11376 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.236052 11244 tablet_service.cc:1511] Processing CreateTablet for tablet ddf8cca10e1244d8882abd13fa81113a (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:11:52.237382 11244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddf8cca10e1244d8882abd13fa81113a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.240301 11403 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.241088 11223 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:52.242539 11399 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:52.242750 11376 ts_tablet_manager.cc:1434] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.012s	user 0.000s	sys 0.007s
I20260709 12:11:52.242956 11399 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.243360 11399 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.243880 11376 tablet_bootstrap.cc:492] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:52.250615 11399 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.251509 11375 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:52.252023 11375 ts_tablet_manager.cc:1403] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.315s	user 0.014s	sys 0.005s
W20260709 12:11:52.253399 11296 leader_election.cc:343] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.253571 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:52.253961 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:52.254890 11383 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:11:52.255038 11222 leader_election.cc:343] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.255880 11375 raft_consensus.cc:359] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.256599 11375 raft_consensus.cc:385] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.256929 11375 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.257266 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:52.257870 11330 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.257683 11375 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.261988 11375 ts_tablet_manager.cc:1434] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:11:52.263029 11375 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:52.268596 11376 tablet_bootstrap.cc:654] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.276635 11374 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:52.277164 11374 ts_tablet_manager.cc:1403] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.233s	user 0.009s	sys 0.006s
I20260709 12:11:52.271366 11399 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.280179 11374 raft_consensus.cc:359] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.280886 11374 raft_consensus.cc:385] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.281188 11374 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.282125 11330 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:52.283437 11223 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:52.284061 11375 tablet_bootstrap.cc:654] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.285377 11399 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:52.286083 11374 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.286617 11399 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:52.287611 11399 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.291888 11374 ts_tablet_manager.cc:1434] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:11:52.292232 11316 tablet_service.cc:1511] Processing CreateTablet for tablet ddf8cca10e1244d8882abd13fa81113a (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:11:52.292924 11374 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:52.293556 11316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddf8cca10e1244d8882abd13fa81113a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.294646 11376 tablet_bootstrap.cc:492] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:52.295161 11376 ts_tablet_manager.cc:1403] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.052s	user 0.006s	sys 0.006s
I20260709 12:11:52.298202 11376 raft_consensus.cc:359] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.300817 11376 raft_consensus.cc:385] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.301613 11376 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.302469 11376 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.302802 11172 tablet_service.cc:1511] Processing CreateTablet for tablet 86c5246d860d4d559ea665671cc46b5c (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:11:52.312326 11312 tablet_service.cc:1511] Processing CreateTablet for tablet 86c5246d860d4d559ea665671cc46b5c (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:11:52.314957 11172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c5246d860d4d559ea665671cc46b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.316619 11376 ts_tablet_manager.cc:1434] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:11:52.314685 11243 tablet_service.cc:1511] Processing CreateTablet for tablet 86c5246d860d4d559ea665671cc46b5c (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:11:52.318701 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c5246d860d4d559ea665671cc46b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.325352 11312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c5246d860d4d559ea665671cc46b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.331904 11376 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:52.342000 11244 tablet_service.cc:1511] Processing CreateTablet for tablet f2d37a48723c42228c9717f514174704 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:11:52.343082 11376 tablet_bootstrap.cc:654] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.343411 11244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d37a48723c42228c9717f514174704. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:11:52.351948 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:52.354395 11316 tablet_service.cc:1511] Processing CreateTablet for tablet f2d37a48723c42228c9717f514174704 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:11:52.355607 11169 tablet_service.cc:1511] Processing CreateTablet for tablet b9dab54d96124869a4a52f9cc08e288a (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:11:52.356292 11316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d37a48723c42228c9717f514174704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.356884 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9dab54d96124869a4a52f9cc08e288a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.362759 11163 tablet_service.cc:1511] Processing CreateTablet for tablet f2d37a48723c42228c9717f514174704 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:11:52.364259 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d37a48723c42228c9717f514174704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.383044 11320 tablet_service.cc:1511] Processing CreateTablet for tablet b9dab54d96124869a4a52f9cc08e288a (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:11:52.384987 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9dab54d96124869a4a52f9cc08e288a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.386025 11375 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:52.386590 11375 ts_tablet_manager.cc:1403] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.124s	user 0.042s	sys 0.025s
I20260709 12:11:52.387094 11245 tablet_service.cc:1511] Processing CreateTablet for tablet b9dab54d96124869a4a52f9cc08e288a (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:11:52.388965 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9dab54d96124869a4a52f9cc08e288a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.389961 11375 raft_consensus.cc:359] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.390769 11375 raft_consensus.cc:385] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.391132 11375 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.392149 11375 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.401309 11169 tablet_service.cc:1511] Processing CreateTablet for tablet e03893c6062f45ea972113cfa57ba362 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:11:52.313000 11374 tablet_bootstrap.cc:654] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.403263 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e03893c6062f45ea972113cfa57ba362. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.408244 11242 tablet_service.cc:1511] Processing CreateTablet for tablet e03893c6062f45ea972113cfa57ba362 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:11:52.409534 11242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e03893c6062f45ea972113cfa57ba362. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.410629 11079 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:52.412588 11375 ts_tablet_manager.cc:1434] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:11:52.420917 11375 tablet_bootstrap.cc:492] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:52.423957 11314 tablet_service.cc:1511] Processing CreateTablet for tablet e03893c6062f45ea972113cfa57ba362 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:11:52.425292 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e03893c6062f45ea972113cfa57ba362. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.429864 11374 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:52.430377 11375 tablet_bootstrap.cc:654] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.429924 11165 tablet_service.cc:1511] Processing CreateTablet for tablet ab7767a1cf4e4b5493ca9acc8daeabab (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:11:52.431469 11165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7767a1cf4e4b5493ca9acc8daeabab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.434064 11240 tablet_service.cc:1511] Processing CreateTablet for tablet ab7767a1cf4e4b5493ca9acc8daeabab (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:11:52.435429 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7767a1cf4e4b5493ca9acc8daeabab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.439443 11403 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.439963 11403 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.442312 11403 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.443812 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:52.444434 11318 tablet_service.cc:1511] Processing CreateTablet for tablet ab7767a1cf4e4b5493ca9acc8daeabab (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:11:52.445722 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7767a1cf4e4b5493ca9acc8daeabab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.445982 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:52.447600 11148 leader_election.cc:343] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:52.448752 11147 leader_election.cc:343] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.449210 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:52.452759 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 8eaf18bb7a144d35b42cf5548b769c0f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:11:52.453570 11403 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.454072 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eaf18bb7a144d35b42cf5548b769c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.459275 11376 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:52.459895 11376 ts_tablet_manager.cc:1403] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.128s	user 0.020s	sys 0.049s
I20260709 12:11:52.430521 11374 ts_tablet_manager.cc:1403] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.138s	user 0.011s	sys 0.001s
I20260709 12:11:52.462958 11376 raft_consensus.cc:359] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.463711 11376 raft_consensus.cc:385] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.464025 11376 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.465468 11374 raft_consensus.cc:359] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.466214 11374 raft_consensus.cc:385] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.466519 11374 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.469022 11376 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.471338 11244 tablet_service.cc:1511] Processing CreateTablet for tablet 8eaf18bb7a144d35b42cf5548b769c0f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:11:52.473111 11244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eaf18bb7a144d35b42cf5548b769c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.474922 11376 ts_tablet_manager.cc:1434] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 12:11:52.475944 11376 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:52.477876 11374 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.480525 11374 ts_tablet_manager.cc:1434] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:11:52.481622 11374 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:52.488226 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 8eaf18bb7a144d35b42cf5548b769c0f (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:11:52.489707 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eaf18bb7a144d35b42cf5548b769c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.493657 11374 tablet_bootstrap.cc:654] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.493460 11169 tablet_service.cc:1511] Processing CreateTablet for tablet 39571de4248f4cfd85f9b1062a776cc8 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:11:52.494791 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39571de4248f4cfd85f9b1062a776cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.503798 11243 tablet_service.cc:1511] Processing CreateTablet for tablet 39571de4248f4cfd85f9b1062a776cc8 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:11:52.504940 11376 tablet_bootstrap.cc:654] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.505183 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39571de4248f4cfd85f9b1062a776cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.507488 11375 tablet_bootstrap.cc:492] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:52.508069 11375 ts_tablet_manager.cc:1403] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.087s	user 0.012s	sys 0.022s
I20260709 12:11:52.528033 11399 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.528363 11376 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:52.528882 11376 ts_tablet_manager.cc:1403] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.001s
I20260709 12:11:52.528566 11399 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.530910 11399 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.524806 11375 raft_consensus.cc:359] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.533596 11375 raft_consensus.cc:385] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.533337 11318 tablet_service.cc:1511] Processing CreateTablet for tablet 39571de4248f4cfd85f9b1062a776cc8 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:11:52.533936 11375 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.534252 11374 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:52.534700 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39571de4248f4cfd85f9b1062a776cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.535087 11374 ts_tablet_manager.cc:1403] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.009s
I20260709 12:11:52.534811 11375 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.538017 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:52.538636 11330 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:52.540060 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:52.545487 11399 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:52.546051 11399 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.546491 11399 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.552448 11375 ts_tablet_manager.cc:1434] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:11:52.553236 11383 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.553929 11383 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.559144 11375 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:52.563295 11374 raft_consensus.cc:359] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.564159 11374 raft_consensus.cc:385] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.564474 11374 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.567066 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.567827 11181 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:52.569085 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:52.531963 11376 raft_consensus.cc:359] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.532678 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.570106 11182 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:52.571291 11376 raft_consensus.cc:385] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.571671 11376 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.571810 11416 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:52.572386 11416 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.572232 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:52.572825 11383 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:52.572460 11376 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.573081 11416 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.573544 11374 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.574548 11257 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:52.575935 11374 ts_tablet_manager.cc:1434] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.040s	user 0.019s	sys 0.003s
I20260709 12:11:52.577821 11374 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:52.579303 11169 tablet_service.cc:1511] Processing CreateTablet for tablet 5fc47889c90446dd927b2ac3010d58fd (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:11:52.580814 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc47889c90446dd927b2ac3010d58fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.584877 11374 tablet_bootstrap.cc:654] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.586580 11416 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.590267 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:52.590883 11182 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.591823 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:52.592417 11257 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.592229 11399 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.595973 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:52.596544 11330 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.598059 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 2fdc8cd3021d4b678786056187782c16 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:11:52.587184 11376 ts_tablet_manager.cc:1434] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.002s
I20260709 12:11:52.599426 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fdc8cd3021d4b678786056187782c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.601025 11240 tablet_service.cc:1511] Processing CreateTablet for tablet 5fc47889c90446dd927b2ac3010d58fd (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:11:52.602463 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc47889c90446dd927b2ac3010d58fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.606098 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:52.606758 11181 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.607374 11330 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:52.609513 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:52.610951 11379 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:52.612394 11376 tablet_bootstrap.cc:492] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:52.612289 11242 tablet_service.cc:1511] Processing CreateTablet for tablet 2fdc8cd3021d4b678786056187782c16 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:11:52.598060 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 5fc47889c90446dd927b2ac3010d58fd (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:11:52.613761 11242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fdc8cd3021d4b678786056187782c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.615839 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fc47889c90446dd927b2ac3010d58fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.620905 11312 tablet_service.cc:1511] Processing CreateTablet for tablet 2fdc8cd3021d4b678786056187782c16 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:11:52.622447 11376 tablet_bootstrap.cc:654] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.623874 11312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fdc8cd3021d4b678786056187782c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.631865 11165 tablet_service.cc:1511] Processing CreateTablet for tablet 458129fddd7e4b33b63d818f6af21f23 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:11:52.634159 11399 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.638461 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 458129fddd7e4b33b63d818f6af21f23 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:11:52.639801 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 458129fddd7e4b33b63d818f6af21f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.640671 11181 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:52.646286 11375 tablet_bootstrap.cc:654] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.649926 11165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 458129fddd7e4b33b63d818f6af21f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.651921 11182 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:52.653138 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:52.654136 11383 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:52.660943 11320 tablet_service.cc:1511] Processing CreateTablet for tablet 458129fddd7e4b33b63d818f6af21f23 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:11:52.662953 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 458129fddd7e4b33b63d818f6af21f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.703682 11416 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:52.704843 11416 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.705557 11416 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.731055 11379 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:52.743170 11379 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.744580 11376 tablet_bootstrap.cc:492] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:52.745139 11376 ts_tablet_manager.cc:1403] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.133s	user 0.019s	sys 0.058s
I20260709 12:11:52.746228 11375 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:52.746829 11375 ts_tablet_manager.cc:1403] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.188s	user 0.020s	sys 0.006s
I20260709 12:11:52.748147 11376 raft_consensus.cc:359] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.748867 11376 raft_consensus.cc:385] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.749154 11376 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.744417 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.750172 11375 raft_consensus.cc:359] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.750603 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 0a6ec18bb8c54ece99a794e109201789 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:11:52.750972 11375 raft_consensus.cc:385] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.751405 11375 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.751830 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a6ec18bb8c54ece99a794e109201789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.751220 11376 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.752465 11375 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.737910 11383 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:52.754957 11376 ts_tablet_manager.cc:1434] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:11:52.756508 11376 tablet_bootstrap.cc:492] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:52.758414 11163 tablet_service.cc:1511] Processing CreateTablet for tablet 7cba0650205b4eb0b416158a057a90b9 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:11:52.759784 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cba0650205b4eb0b416158a057a90b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.760000 11243 tablet_service.cc:1511] Processing CreateTablet for tablet 7cba0650205b4eb0b416158a057a90b9 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:11:52.761487 11243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cba0650205b4eb0b416158a057a90b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.766361 11376 tablet_bootstrap.cc:654] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.768185 11375 ts_tablet_manager.cc:1434] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:11:52.771447 11416 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:52.755421 11383 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.784075 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 0a6ec18bb8c54ece99a794e109201789 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:11:52.785773 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a6ec18bb8c54ece99a794e109201789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.788810 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 0a6ec18bb8c54ece99a794e109201789 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:11:52.789855 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a6ec18bb8c54ece99a794e109201789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.795594 11374 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:52.796275 11374 ts_tablet_manager.cc:1403] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.219s	user 0.023s	sys 0.015s
I20260709 12:11:52.803373 11257 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
W20260709 12:11:52.810477 11296 leader_election.cc:343] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.781188 11421 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.811834 11421 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.812963 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:52.815702 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 7cba0650205b4eb0b416158a057a90b9 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:11:52.816953 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cba0650205b4eb0b416158a057a90b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.782332 11375 tablet_bootstrap.cc:492] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:52.815533 11403 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.823197 11403 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.826651 11403 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.815660 11416 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.826756 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:52.827786 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:52.830044 11416 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.836553 11416 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:11:52.837082 11148 leader_election.cc:343] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:52.838030 11147 leader_election.cc:343] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.838543 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:52.839064 11182 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.839725 11418 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.840214 11418 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.843901 11418 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:52.844713 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:52.846169 11223 leader_election.cc:343] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.846558 11421 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:11:52.847290 11296 leader_election.cc:343] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.847483 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.847788 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:52.848630 11418 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:52.849857 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:52.850250 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:52.851491 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:52.851723 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:11:52.850615 11222 leader_election.cc:343] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.852391 11182 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:52.852490 11222 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:11:52.852053 11296 leader_election.cc:343] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:52.860205 11296 leader_election.cc:343] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:52.860921 11376 tablet_bootstrap.cc:492] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:52.866271 11376 ts_tablet_manager.cc:1403] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.110s	user 0.021s	sys 0.004s
I20260709 12:11:52.860703 11418 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:11:52.867070 11357 tablet.cc:2406] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:11:52.870368 11170 tablet_service.cc:1511] Processing CreateTablet for tablet 9fdf86dcb01f4d81913c37b45cfc3879 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:11:52.871909 11170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fdf86dcb01f4d81913c37b45cfc3879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.871687 11242 tablet_service.cc:1511] Processing CreateTablet for tablet 9fdf86dcb01f4d81913c37b45cfc3879 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:11:52.872246 11376 raft_consensus.cc:359] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.872996 11376 raft_consensus.cc:385] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.878684 11374 raft_consensus.cc:359] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.879478 11163 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa73285733a4a67a3e705a20155f69d (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:11:52.879912 11374 raft_consensus.cc:385] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:52.880337 11374 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.880651 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa73285733a4a67a3e705a20155f69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.873337 11376 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:52.882824 11242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fdf86dcb01f4d81913c37b45cfc3879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.885519 11376 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.865497 11421 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.881742 11403 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.925031 11320 tablet_service.cc:1511] Processing CreateTablet for tablet 9fdf86dcb01f4d81913c37b45cfc3879 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:11:52.937186 11314 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa73285733a4a67a3e705a20155f69d (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:11:52.938696 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa73285733a4a67a3e705a20155f69d (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:11:52.886000 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:52.939845 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa73285733a4a67a3e705a20155f69d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:11:52.940217 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 80061.6us
I20260709 12:11:52.940376 11379 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:52.941428 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:52.942617 11421 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:52.943405 11421 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:52.943811 11421 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.944240 11421 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:52.947134 11376 ts_tablet_manager.cc:1434] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 12:11:52.881232 11374 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:52.955668 11376 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:52.958089 11257 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:52.959643 11320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fdf86dcb01f4d81913c37b45cfc3879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.964131 11314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa73285733a4a67a3e705a20155f69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.964761 11169 tablet_service.cc:1511] Processing CreateTablet for tablet 89068ffedf554d89af62a8b1fe0700c0 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:11:52.966009 11169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89068ffedf554d89af62a8b1fe0700c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.952301 11077 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:52.973384 11318 tablet_service.cc:1511] Processing CreateTablet for tablet 89068ffedf554d89af62a8b1fe0700c0 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:11:52.974887 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89068ffedf554d89af62a8b1fe0700c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.976135 11375 tablet_bootstrap.cc:654] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.961452 11374 ts_tablet_manager.cc:1434] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.165s	user 0.011s	sys 0.000s
I20260709 12:11:52.982882 11374 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
W20260709 12:11:52.958283 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 120275us
I20260709 12:11:52.989338 11374 tablet_bootstrap.cc:654] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.996613 11376 tablet_bootstrap.cc:654] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:52.996840 11163 tablet_service.cc:1511] Processing CreateTablet for tablet c519deb8010e411995230a6685947b97 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:11:52.998098 11163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c519deb8010e411995230a6685947b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:52.998858 11245 tablet_service.cc:1511] Processing CreateTablet for tablet 89068ffedf554d89af62a8b1fe0700c0 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:11:52.999758 11245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89068ffedf554d89af62a8b1fe0700c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:53.019383 11079 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:53.012084 11383 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.003048 11421 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.025456 11383 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.039213 11383 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.026634 11240 tablet_service.cc:1511] Processing CreateTablet for tablet c519deb8010e411995230a6685947b97 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:11:53.040429 11240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c519deb8010e411995230a6685947b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:53.040979 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:53.042614 11296 leader_election.cc:343] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.043617 11421 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.044611 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:53.045310 11180 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.040187 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:11:53.051664 11296 leader_election.cc:343] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.052340 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:53.054379 11375 tablet_bootstrap.cc:492] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.054255 11318 tablet_service.cc:1511] Processing CreateTablet for tablet c519deb8010e411995230a6685947b97 (DEFAULT_TABLE table=test-workload [id=3e3271307e694b2281edeb86f889d4e9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:11:53.054952 11375 ts_tablet_manager.cc:1403] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.283s	user 0.036s	sys 0.015s
I20260709 12:11:53.054316 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.056272 11257 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.067049 11257 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.055595 11318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c519deb8010e411995230a6685947b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:53.075291 11376 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.075855 11376 ts_tablet_manager.cc:1403] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.121s	user 0.011s	sys 0.005s
I20260709 12:11:53.075421 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.076722 11383 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.077289 11383 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:53.078965 11420 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.079252 11383 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.079536 11420 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.081785 11420 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:11:53.068467 11421 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.078991 11375 raft_consensus.cc:359] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.086254 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.088583 11257 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:53.089529 11374 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.090296 11374 ts_tablet_manager.cc:1403] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.013s
I20260709 12:11:53.094312 11374 raft_consensus.cc:359] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.095351 11374 raft_consensus.cc:385] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.095821 11374 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.097481 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.098400 11420 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.097515 11374 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.098789 11420 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.099150 11420 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.101078 11374 ts_tablet_manager.cc:1434] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:11:53.104226 11376 raft_consensus.cc:359] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.104921 11376 raft_consensus.cc:385] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.105192 11376 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.105888 11376 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.107185 11180 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.107214 11420 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.087407 11375 raft_consensus.cc:385] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.108477 11375 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.098201 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.109208 11375 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:53.110488 11296 leader_election.cc:343] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.111564 11375 ts_tablet_manager.cc:1434] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.002s
I20260709 12:11:53.112998 11374 tablet_bootstrap.cc:492] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.117457 11376 ts_tablet_manager.cc:1434] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:11:53.118409 11376 tablet_bootstrap.cc:492] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.118562 11420 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:11:53.119242 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.119846 11257 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.122296 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:53.123847 11374 tablet_bootstrap.cc:654] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:53.123834 11296 leader_election.cc:343] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.125245 11375 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.127391 11257 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.128965 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:53.130172 11420 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.130713 11420 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:53.131847 11420 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.132755 11375 tablet_bootstrap.cc:654] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.147454 11078 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:53.155959 11376 tablet_bootstrap.cc:654] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.169190 11379 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:53.178051 11426 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.178544 11426 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.180751 11426 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.182507 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.183214 11330 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:53.184618 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:53.186456 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:53.188730 11147 leader_election.cc:343] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.190177 11426 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.190593 11426 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.190932 11426 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.191000 11077 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:53.197719 11399 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:53.202598 11426 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.205711 11420 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:53.205255 11376 tablet_bootstrap.cc:492] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.211536 11376 ts_tablet_manager.cc:1403] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.093s	user 0.017s	sys 0.000s
I20260709 12:11:53.214763 11376 raft_consensus.cc:359] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.215447 11376 raft_consensus.cc:385] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.215719 11376 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.215912 11374 tablet_bootstrap.cc:492] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.216503 11374 ts_tablet_manager.cc:1403] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.104s	user 0.039s	sys 0.044s
I20260709 12:11:53.218814 11426 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.219717 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.220383 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:53.220938 11330 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.221283 11375 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.221745 11375 ts_tablet_manager.cc:1403] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.097s	user 0.018s	sys 0.030s
I20260709 12:11:53.224395 11375 raft_consensus.cc:359] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.224669 11374 raft_consensus.cc:359] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.225070 11375 raft_consensus.cc:385] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.225600 11375 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
W20260709 12:11:53.225606 11147 leader_election.cc:343] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.226411 11375 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.228683 11375 ts_tablet_manager.cc:1434] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:11:53.225575 11374 raft_consensus.cc:385] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.229316 11374 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.229615 11375 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.230363 11374 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.232417 11374 ts_tablet_manager.cc:1434] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.000s
I20260709 12:11:53.233309 11374 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.234977 11416 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:53.235960 11375 tablet_bootstrap.cc:654] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.238261 11376 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.239766 11421 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.240458 11376 ts_tablet_manager.cc:1434] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.001s
I20260709 12:11:53.240222 11421 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.241984 11374 tablet_bootstrap.cc:654] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.229918 11330 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:53.245823 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:53.247875 11426 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.248515 11426 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:53.249598 11426 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.251955 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.254518 11421 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.254771 11376 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.264219 11376 tablet_bootstrap.cc:654] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.253736 11374 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
W20260709 12:11:53.254943 11296 leader_election.cc:343] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.267393 11374 ts_tablet_manager.cc:1403] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.002s
I20260709 12:11:53.269835 11375 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.270517 11375 ts_tablet_manager.cc:1403] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.019s
I20260709 12:11:53.270331 11374 raft_consensus.cc:359] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.271039 11374 raft_consensus.cc:385] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.271299 11374 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.272142 11374 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.273980 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.273942 11374 ts_tablet_manager.cc:1434] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:11:53.274645 11257 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:53.273890 11375 raft_consensus.cc:359] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.275429 11375 raft_consensus.cc:385] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.275554 11374 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.275703 11375 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.276466 11375 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.278604 11375 ts_tablet_manager.cc:1434] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:11:53.279543 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:53.282202 11421 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.280738 11375 tablet_bootstrap.cc:492] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.280894 11376 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.283921 11376 ts_tablet_manager.cc:1403] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.008s
I20260709 12:11:53.286502 11376 raft_consensus.cc:359] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.287144 11376 raft_consensus.cc:385] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.287408 11376 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.281610 11374 tablet_bootstrap.cc:654] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.282560 11421 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.288244 11375 tablet_bootstrap.cc:654] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.288156 11421 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.288583 11076 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:53.290829 11376 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.293823 11376 ts_tablet_manager.cc:1434] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:11:53.295382 11376 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.303634 11421 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.306658 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:11:53.308130 11296 leader_election.cc:343] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.309020 11376 tablet_bootstrap.cc:654] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.310448 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.311060 11257 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.316126 11374 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.316603 11374 ts_tablet_manager.cc:1403] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.006s
I20260709 12:11:53.318435 11257 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.319341 11374 raft_consensus.cc:359] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.319579 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:53.320003 11374 raft_consensus.cc:385] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.320266 11374 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.320434 11420 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.320883 11420 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:53.321333 11374 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.321731 11420 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.323801 11374 ts_tablet_manager.cc:1434] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:11:53.327104 11421 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.328575 11374 tablet_bootstrap.cc:492] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.337623 11421 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.338994 11421 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.344918 11375 tablet_bootstrap.cc:492] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.345609 11375 ts_tablet_manager.cc:1403] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.065s	user 0.024s	sys 0.034s
I20260709 12:11:53.346164 11374 tablet_bootstrap.cc:654] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.349246 11375 raft_consensus.cc:359] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.350637 11375 raft_consensus.cc:385] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.351006 11375 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.351266 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.351354 11079 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:11:53.353253 11296 leader_election.cc:343] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.353742 11426 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.352166 11375 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.354228 11426 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.356359 11426 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:53.357688 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:53.358054 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.358750 11421 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:11:53.359433 11148 leader_election.cc:343] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.359848 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.360658 11256 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:53.361168 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.362087 11426 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.362574 11426 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:53.363137 11147 leader_election.cc:343] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.363726 11147 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:53.364584 11375 ts_tablet_manager.cc:1434] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:11:53.362761 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:53.365528 11375 tablet_bootstrap.cc:492] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.365567 11403 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.366281 11426 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.365566 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.366901 11421 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.367275 11421 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.367630 11421 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.368525 11374 tablet_bootstrap.cc:492] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.368860 11379 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.368988 11374 ts_tablet_manager.cc:1403] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.002s
I20260709 12:11:53.369323 11379 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.371395 11379 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.371852 11379 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.372167 11374 raft_consensus.cc:359] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.372614 11375 tablet_bootstrap.cc:654] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.372257 11379 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.372916 11374 raft_consensus.cc:385] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.373281 11374 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.374002 11374 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.374392 11379 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.374997 11421 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.376140 11374 ts_tablet_manager.cc:1434] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:11:53.377019 11374 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.377985 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.379176 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.379827 11180 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:53.379900 11420 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.379971 11376 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.380488 11420 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.380726 11376 ts_tablet_manager.cc:1403] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.086s	user 0.031s	sys 0.025s
I20260709 12:11:53.382726 11420 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.373164 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.387529 11257 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:53.387681 11376 raft_consensus.cc:359] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.384984 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.385268 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.389119 11256 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:53.389475 11331 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:53.380805 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:53.388365 11376 raft_consensus.cc:385] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.389235 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.390638 11421 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.390993 11255 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.391907 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:53.393030 11426 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.394131 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.394281 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.395049 11399 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:11:53.395159 11296 leader_election.cc:343] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.395447 11399 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.395814 11399 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.396478 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.397234 11330 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:53.398638 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.399268 11180 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:53.402019 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:53.403473 11379 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.406996 11255 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.408887 11399 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.390507 11376 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.410604 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:53.410869 11376 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.412554 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:53.413225 11421 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.413794 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:53.415858 11330 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.416697 11426 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.413939 11420 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.418220 11420 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:53.419101 11420 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.413316 11180 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.413687 11399 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.414594 11421 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.415158 11376 ts_tablet_manager.cc:1434] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.003s
I20260709 12:11:53.423194 11421 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.424224 11376 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.424858 11330 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:53.429003 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.429913 11399 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.430701 11379 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.431105 11399 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:53.433426 11375 tablet_bootstrap.cc:492] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.434180 11375 ts_tablet_manager.cc:1403] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.019s
I20260709 12:11:53.432045 11399 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.434960 11180 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:53.439181 11375 raft_consensus.cc:359] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.439847 11375 raft_consensus.cc:385] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.440121 11375 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.440828 11375 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.443420 11375 ts_tablet_manager.cc:1434] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:11:53.444813 11375 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.444914 11421 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.447552 11374 tablet_bootstrap.cc:654] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.449703 11076 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:53.455996 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:53.460175 11376 tablet_bootstrap.cc:654] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.459834 11180 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.466065 11421 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.467221 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.467873 11255 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.470753 11375 tablet_bootstrap.cc:654] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.471812 11079 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:53.475225 11180 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.475957 11255 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.476538 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:53.477487 11421 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.478058 11421 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:53.479256 11421 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.483090 11374 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.483541 11374 ts_tablet_manager.cc:1403] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.107s	user 0.008s	sys 0.011s
I20260709 12:11:53.487321 11374 raft_consensus.cc:359] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.487993 11374 raft_consensus.cc:385] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.488267 11374 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.488919 11374 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.493541 11375 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.494065 11375 ts_tablet_manager.cc:1403] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.013s
I20260709 12:11:53.494788 11376 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.495227 11376 ts_tablet_manager.cc:1403] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.015s
I20260709 12:11:53.495317 11374 ts_tablet_manager.cc:1434] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:11:53.496248 11374 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.497033 11375 raft_consensus.cc:359] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.497763 11375 raft_consensus.cc:385] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.497735 11376 raft_consensus.cc:359] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.498256 11375 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.498535 11376 raft_consensus.cc:385] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.498824 11376 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.499274 11375 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.499465 11376 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.501957 11376 ts_tablet_manager.cc:1434] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:11:53.502911 11376 tablet_bootstrap.cc:492] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.508931 11375 ts_tablet_manager.cc:1434] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 12:11:53.511586 11374 tablet_bootstrap.cc:654] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.511260 11076 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:53.513540 11421 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:53.519676 11375 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.522292 11376 tablet_bootstrap.cc:654] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.533954 11421 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.534514 11421 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.536700 11421 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.538604 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.539319 11180 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:53.539553 11375 tablet_bootstrap.cc:654] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.541182 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:53.541282 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.542024 11255 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:53.542403 11416 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.542770 11416 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.543099 11416 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.545284 11374 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.546089 11374 ts_tablet_manager.cc:1403] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.009s
I20260709 12:11:53.549082 11374 raft_consensus.cc:359] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.549841 11374 raft_consensus.cc:385] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.550211 11374 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.550068 11421 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:53.551188 11374 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.555948 11374 ts_tablet_manager.cc:1434] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:11:53.556836 11374 tablet_bootstrap.cc:492] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.557614 11416 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.562852 11416 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.563040 11374 tablet_bootstrap.cc:654] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.564497 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.564567 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:53.565200 11256 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.565357 11180 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.574844 11376 tablet_bootstrap.cc:492] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.575332 11376 ts_tablet_manager.cc:1403] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.011s
I20260709 12:11:53.575791 11256 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.577416 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.577961 11376 raft_consensus.cc:359] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.578318 11416 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.578671 11376 raft_consensus.cc:385] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.579136 11376 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.579538 11416 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:53.579813 11376 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.580608 11375 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.580415 11416 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.581425 11375 ts_tablet_manager.cc:1403] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.006s
I20260709 12:11:53.582145 11376 ts_tablet_manager.cc:1434] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:11:53.583057 11376 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.585140 11375 raft_consensus.cc:359] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.589123 11375 raft_consensus.cc:385] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.591455 11375 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.588819 11376 tablet_bootstrap.cc:654] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.593369 11375 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.593801 11383 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:53.596443 11375 ts_tablet_manager.cc:1434] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:11:53.598203 11375 tablet_bootstrap.cc:492] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.590229 11180 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:53.620790 11375 tablet_bootstrap.cc:654] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.626935 11181 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.628116 11376 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.628780 11376 ts_tablet_manager.cc:1403] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.005s
I20260709 12:11:53.621497 11076 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:53.632254 11376 raft_consensus.cc:359] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.632961 11376 raft_consensus.cc:385] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.633280 11376 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.634027 11376 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.636164 11376 ts_tablet_manager.cc:1434] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:11:53.637118 11376 tablet_bootstrap.cc:492] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.639281 11375 tablet_bootstrap.cc:492] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.639758 11375 ts_tablet_manager.cc:1403] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.000s
I20260709 12:11:53.642529 11375 raft_consensus.cc:359] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.643261 11376 tablet_bootstrap.cc:654] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.643218 11375 raft_consensus.cc:385] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.643752 11375 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.644573 11375 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.645761 11374 tablet_bootstrap.cc:492] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.646222 11374 ts_tablet_manager.cc:1403] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.090s	user 0.037s	sys 0.035s
I20260709 12:11:53.649672 11374 raft_consensus.cc:359] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.650543 11374 raft_consensus.cc:385] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.650876 11374 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.651863 11374 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.652899 11416 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:53.659483 11375 ts_tablet_manager.cc:1434] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:11:53.659629 11374 ts_tablet_manager.cc:1434] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:11:53.660672 11375 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.661191 11399 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.661851 11399 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.662688 11426 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.663236 11426 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.666373 11399 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.666481 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.667589 11374 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.668665 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:53.668785 11223 leader_election.cc:343] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.669385 11330 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:53.670884 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:53.671667 11426 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.671731 11403 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.672142 11403 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.672483 11403 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.675343 11416 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.672595 11376 tablet_bootstrap.cc:492] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.676210 11376 ts_tablet_manager.cc:1403] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.039s	user 0.027s	sys 0.003s
I20260709 12:11:53.666782 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.675891 11416 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.676774 11255 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:53.677183 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.677853 11181 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:53.678880 11374 tablet_bootstrap.cc:654] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.679090 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:53.679761 11416 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:53.679929 11379 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:53.680325 11379 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.680933 11379 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.681363 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:11:53.684386 11296 leader_election.cc:343] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.685076 11376 raft_consensus.cc:359] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.685863 11376 raft_consensus.cc:385] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.686193 11376 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.686261 11403 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.687072 11376 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.688241 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.689265 11376 ts_tablet_manager.cc:1434] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
W20260709 12:11:53.689958 11296 leader_election.cc:343] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.690258 11376 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.690403 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:53.691184 11383 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.696560 11375 tablet_bootstrap.cc:654] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.699757 11379 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.702387 11379 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.702814 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:53.703481 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:53.703634 11180 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.705103 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:53.704970 11403 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.706642 11255 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:53.706135 11223 leader_election.cc:343] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.710350 11376 tablet_bootstrap.cc:654] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.710114 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:53.712855 11255 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:53.715334 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.717116 11330 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:53.717835 11403 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.718605 11403 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:53.719327 11403 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.746491 11180 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:53.749917 11330 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:53.751780 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:53.753368 11418 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:53.753991 11383 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:53.754034 11418 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:53.755708 11418 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.759573 11374 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.761324 11374 ts_tablet_manager.cc:1403] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.101s	user 0.023s	sys 0.021s
I20260709 12:11:53.769228 11426 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:53.769910 11374 raft_consensus.cc:359] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.770606 11375 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.770689 11374 raft_consensus.cc:385] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.771180 11375 ts_tablet_manager.cc:1403] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.111s	user 0.020s	sys 0.023s
I20260709 12:11:53.771200 11374 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.772086 11374 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.774242 11374 ts_tablet_manager.cc:1434] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:11:53.775233 11374 tablet_bootstrap.cc:492] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.775161 11426 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:53.783466 11374 tablet_bootstrap.cc:654] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.801581 11376 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.802137 11376 ts_tablet_manager.cc:1403] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.112s	user 0.007s	sys 0.006s
I20260709 12:11:53.803633 11078 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:53.798985 11375 raft_consensus.cc:359] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.809054 11375 raft_consensus.cc:385] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.809432 11375 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.811661 11383 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.812284 11383 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.815322 11383 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:11:53.817183 11296 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:53.819381 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:53.820515 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.821358 11078 catalog_manager.cc:5697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:11:53.822217 11296 leader_election.cc:343] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:53.824090 11296 leader_election.cc:343] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.804720 11376 raft_consensus.cc:359] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.824827 11376 raft_consensus.cc:385] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.824653 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:53.826254 11375 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.826596 11383 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.825150 11376 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.828653 11375 ts_tablet_manager.cc:1434] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 12:11:53.828127 11376 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.829610 11375 tablet_bootstrap.cc:492] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.832047 11376 ts_tablet_manager.cc:1434] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.030s	user 0.002s	sys 0.004s
I20260709 12:11:53.833297 11376 tablet_bootstrap.cc:492] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.878957 11375 tablet_bootstrap.cc:654] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.885721 11376 tablet_bootstrap.cc:654] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.890566 11375 tablet_bootstrap.cc:492] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.890802 11374 tablet_bootstrap.cc:492] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.891017 11375 ts_tablet_manager.cc:1403] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.062s	user 0.026s	sys 0.028s
I20260709 12:11:53.891299 11374 ts_tablet_manager.cc:1403] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.116s	user 0.030s	sys 0.039s
I20260709 12:11:53.891245 11416 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:53.893658 11375 raft_consensus.cc:359] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.894093 11418 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.894352 11375 raft_consensus.cc:385] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.894683 11375 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.894220 11374 raft_consensus.cc:359] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.894552 11418 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.895113 11374 raft_consensus.cc:385] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.895519 11374 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.895565 11375 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.896343 11374 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.898591 11374 ts_tablet_manager.cc:1434] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:11:53.900906 11374 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.900728 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:11:53.902477 11222 leader_election.cc:343] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.903316 11375 ts_tablet_manager.cc:1434] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:11:53.904249 11375 tablet_bootstrap.cc:492] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.909168 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:53.910674 11223 leader_election.cc:343] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.911134 11223 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:53.911958 11399 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.913371 11374 tablet_bootstrap.cc:654] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.913837 11375 tablet_bootstrap.cc:654] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.916919 11376 tablet_bootstrap.cc:492] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.917372 11418 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.917532 11376 ts_tablet_manager.cc:1403] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.084s	user 0.033s	sys 0.004s
I20260709 12:11:53.922569 11376 raft_consensus.cc:359] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.923547 11376 raft_consensus.cc:385] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.923906 11376 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.924902 11376 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.930009 11376 ts_tablet_manager.cc:1434] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:11:53.930989 11376 tablet_bootstrap.cc:492] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:53.932222 11375 tablet_bootstrap.cc:492] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.932869 11375 ts_tablet_manager.cc:1403] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.004s
I20260709 12:11:53.935117 11375 raft_consensus.cc:359] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.935827 11375 raft_consensus.cc:385] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.936143 11375 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.936695 11376 tablet_bootstrap.cc:654] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.937036 11375 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.939462 11375 ts_tablet_manager.cc:1434] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:11:53.940508 11375 tablet_bootstrap.cc:492] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.943455 11374 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.943934 11374 ts_tablet_manager.cc:1403] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.004s
I20260709 12:11:53.946422 11375 tablet_bootstrap.cc:654] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.949182 11374 raft_consensus.cc:359] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.949899 11374 raft_consensus.cc:385] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.950169 11374 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.950925 11374 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.955566 11374 ts_tablet_manager.cc:1434] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:11:53.956318 11374 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:53.962807 11374 tablet_bootstrap.cc:654] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.963610 11482 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.964845 11375 tablet_bootstrap.cc:492] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:53.964437 11482 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.965335 11375 ts_tablet_manager.cc:1403] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.000s
I20260709 12:11:53.968428 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.968390 11375 raft_consensus.cc:359] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.969151 11375 raft_consensus.cc:385] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.969465 11375 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.970176 11375 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:53.974392 11148 leader_election.cc:343] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.974802 11375 ts_tablet_manager.cc:1434] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:11:53.973824 11482 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.975862 11375 tablet_bootstrap.cc:492] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:53.976619 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.978893 11416 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:11:53.980903 11147 leader_election.cc:343] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:53.981392 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:53.982271 11482 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:53.984579 11482 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.984783 11376 tablet_bootstrap.cc:492] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:53.981690 11374 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:53.985562 11376 ts_tablet_manager.cc:1403] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.055s	user 0.006s	sys 0.007s
I20260709 12:11:53.985167 11482 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.984843 11375 tablet_bootstrap.cc:654] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:53.988684 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:53.989353 11256 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:53.985749 11374 ts_tablet_manager.cc:1403] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.008s
I20260709 12:11:53.993570 11374 raft_consensus.cc:359] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.994448 11374 raft_consensus.cc:385] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:53.994772 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:53.995539 11482 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:53.994763 11374 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:53.995491 11331 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:53.996996 11374 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:53.997567 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:53.999470 11374 ts_tablet_manager.cc:1434] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:11:53.988730 11376 raft_consensus.cc:359] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.000026 11376 raft_consensus.cc:385] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.000542 11374 tablet_bootstrap.cc:492] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:54.000929 11482 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.001411 11482 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.001894 11482 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.000392 11376 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.006331 11376 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.007614 11374 tablet_bootstrap.cc:654] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.008790 11418 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:54.009559 11376 ts_tablet_manager.cc:1434] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:11:54.010869 11376 tablet_bootstrap.cc:492] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:54.021970 11374 tablet_bootstrap.cc:492] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:54.022523 11374 ts_tablet_manager.cc:1403] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 12:11:54.025427 11379 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:54.027004 11374 raft_consensus.cc:359] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.027825 11374 raft_consensus.cc:385] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.028141 11374 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.028304 11399 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.028846 11399 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.028957 11374 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.031709 11374 ts_tablet_manager.cc:1434] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:11:54.032734 11374 tablet_bootstrap.cc:492] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:54.034140 11399 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.035893 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.036679 11182 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:54.037891 11383 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:54.038002 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:54.039659 11374 tablet_bootstrap.cc:654] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.041817 11418 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.042196 11418 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.042522 11418 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.043756 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:54.045670 11223 leader_election.cc:343] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.058013 11383 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.063516 11374 tablet_bootstrap.cc:492] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:54.064280 11374 ts_tablet_manager.cc:1403] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.012s
I20260709 12:11:54.066852 11483 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.072039 11483 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.059015 11383 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.076854 11383 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:54.077067 11426 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.077560 11426 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.078536 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:54.079663 11426 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.080212 11418 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.081151 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.081496 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:54.079470 11257 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:54.082639 11482 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.082873 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:54.081912 11182 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:54.085606 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:54.090709 11483 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.097456 11399 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.097883 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:54.099231 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:54.102370 11376 tablet_bootstrap.cc:654] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.105836 11257 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:54.105837 11330 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:54.098320 11331 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:54.114933 11383 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.117482 11383 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.120398 11383 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:54.121384 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:54.122071 11182 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.110163 11420 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.124465 11420 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.124848 11420 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.124974 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:54.109943 11379 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.128428 11426 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.128919 11426 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.129269 11426 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.133015 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:54.107652 11399 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.127831 11379 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.155699 11418 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.193825 11374 raft_consensus.cc:359] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.199015 11374 raft_consensus.cc:385] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.199528 11374 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
W20260709 12:11:54.210109 11496 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:11:54.190083 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.191103 11483 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
W20260709 12:11:54.194245 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65851.4us
W20260709 12:11:54.198312 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65416.6us
I20260709 12:11:54.219396 11374 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.198817 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:54.234256 11374 ts_tablet_manager.cc:1434] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.170s	user 0.014s	sys 0.011s
I20260709 12:11:54.235271 11374 tablet_bootstrap.cc:492] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:54.236305 11375 tablet_bootstrap.cc:492] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:54.236732 11375 ts_tablet_manager.cc:1403] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.261s	user 0.059s	sys 0.065s
I20260709 12:11:54.221314 11179 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:54.239586 11482 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.252612 11499 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.253180 11499 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.315462 11403 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.317263 11403 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.317945 11403 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.318660 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:54.319274 11331 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.319178 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:54.317131 11399 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:54.320182 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68904.3us
I20260709 12:11:54.332525 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.333279 11376 tablet_bootstrap.cc:492] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:54.333791 11376 ts_tablet_manager.cc:1403] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.323s	user 0.019s	sys 0.011s
W20260709 12:11:54.333998 11222 leader_election.cc:343] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.332360 11500 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:11:54.337427 11375 raft_consensus.cc:359] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.338292 11375 raft_consensus.cc:385] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.338559 11375 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.339143 11375 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.341470 11375 ts_tablet_manager.cc:1434] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.104s	user 0.010s	sys 0.004s
I20260709 12:11:54.342535 11375 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:54.344923 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.345389 11418 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.345934 11418 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.348004 11418 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:54.328408 11147 leader_election.cc:343] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.349764 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:54.352025 11376 raft_consensus.cc:359] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.354557 11376 raft_consensus.cc:385] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.354801 11376 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.355326 11328 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:54.355803 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:54.355636 11376 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.356413 11254 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.353612 11420 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.359076 11331 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:54.359254 11420 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:54.367108 11376 ts_tablet_manager.cc:1434] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.002s
W20260709 12:11:54.365993 11223 leader_election.cc:343] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.374269 11506 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.374771 11506 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.376040 11403 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.376852 11506 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:54.376223 11426 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.368625 11379 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.369408 11148 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:54.377841 11181 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:54.378733 11483 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.379195 11483 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:54.380056 11483 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.380867 11182 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
W20260709 12:11:54.370452 11296 leader_election.cc:343] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.384590 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.385613 11179 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:54.386499 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:54.381100 11426 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.382227 11254 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:54.392086 11376 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:54.397472 11403 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.402033 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.402797 11181 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:54.403374 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:54.403826 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:54.404139 11257 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.404497 11179 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.405180 11502 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.405670 11502 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.406066 11502 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.395588 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:54.416706 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:54.419365 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:54.420181 11418 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.420655 11418 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.421042 11418 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.421584 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:54.422210 11254 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.422669 11257 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:54.426407 11375 tablet_bootstrap.cc:654] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.426752 11331 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.427404 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:11:54.448995 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:54.420974 11499 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:54.459663 11179 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:54.451246 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.419492 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:54.469820 11379 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.470329 11379 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:54.470752 11374 tablet_bootstrap.cc:654] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.471232 11483 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:54.461306 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:54.472503 11418 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.475582 11181 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:11:54.476761 11147 leader_election.cc:343] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.479614 11331 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:54.471232 11379 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.482810 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:11:54.484592 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74060.8us
I20260709 12:11:54.487192 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:54.497498 11255 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:54.499634 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:54.501112 11426 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.501853 11426 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:54.501711 11079 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:54.503141 11426 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:54.460661 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70410.4us
I20260709 12:11:54.515607 11507 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.516222 11507 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.516744 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:54.517714 11399 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.518287 11399 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.518455 11507 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.510360 11499 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.518781 11399 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.521279 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.522004 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:54.522019 11180 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:54.522989 11499 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:54.523329 11331 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:54.524034 11499 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.524857 11418 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.523062 11329 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.528205 11500 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:11:54.536697 11254 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:54.536904 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:54.538883 11326 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:54.533802 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:54.540416 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:54.540951 11418 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.541260 11325 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.547101 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:54.548215 11331 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:54.541391 11418 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.551949 11418 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.531880 11502 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.569766 11328 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:54.592211 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:54.533685 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:54.527508 11507 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:11:54.598064 11482 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:54.600883 11399 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.615103 11374 tablet_bootstrap.cc:492] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:54.615682 11374 ts_tablet_manager.cc:1403] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.381s	user 0.006s	sys 0.040s
I20260709 12:11:54.533968 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:54.594820 11502 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:54.620481 11399 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:54.623132 11222 leader_election.cc:343] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.623584 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:11:54.623998 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94870.9us
I20260709 12:11:54.624960 11500 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:54.625339 11500 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.625710 11500 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.624228 11182 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.626869 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:54.628057 11383 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:54.628923 11416 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:54.635811 11182 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:54.622162 11325 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:54.639685 11376 tablet_bootstrap.cc:654] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.644199 11374 raft_consensus.cc:359] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.667063 11374 raft_consensus.cc:385] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.667552 11374 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.664732 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:54.640926 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:54.671937 11418 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.672451 11181 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:54.674850 11399 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.675463 11399 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:54.676600 11399 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.654229 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:54.690701 11374 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.639202 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:54.698807 11379 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:54.700742 11374 ts_tablet_manager.cc:1434] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.085s	user 0.014s	sys 0.000s
I20260709 12:11:54.695032 11256 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.701788 11374 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:54.651261 11500 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.645762 11078 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:11:54.710239 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73394.1us
I20260709 12:11:54.714972 11418 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:54.723634 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:54.724272 11182 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.716409 11504 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:54.746151 11182 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:54.748497 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:54.750627 11379 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.751226 11379 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:54.752269 11379 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.774777 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:54.775682 11325 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:54.776516 11375 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:54.776912 11500 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:54.777544 11375 ts_tablet_manager.cc:1403] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.435s	user 0.029s	sys 0.015s
I20260709 12:11:54.778303 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:54.778877 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:54.777563 11376 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:54.780988 11376 ts_tablet_manager.cc:1403] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.389s	user 0.014s	sys 0.021s
I20260709 12:11:54.780045 11256 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:54.778906 11325 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:54.782490 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:54.784003 11504 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.784511 11504 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:54.784845 11374 tablet_bootstrap.cc:654] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.791553 11376 raft_consensus.cc:359] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.792309 11376 raft_consensus.cc:385] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.793081 11376 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.788425 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:54.801170 11376 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.808128 11325 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:54.808531 11375 raft_consensus.cc:359] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.809170 11375 raft_consensus.cc:385] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.809403 11375 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.810129 11375 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.805470 11426 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:11:54.812134 11376 ts_tablet_manager.cc:1434] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:11:54.813136 11376 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:54.816983 11182 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:54.815984 11504 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.830060 11375 ts_tablet_manager.cc:1434] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.052s	user 0.014s	sys 0.000s
I20260709 12:11:54.831076 11375 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:54.837116 11374 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:54.837553 11374 ts_tablet_manager.cc:1403] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.136s	user 0.011s	sys 0.004s
I20260709 12:11:54.838562 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:54.840970 11499 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:54.842159 11499 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:54.842357 11374 raft_consensus.cc:359] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.843070 11499 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.844347 11374 raft_consensus.cc:385] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.845417 11374 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.851394 11403 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:54.854332 11375 tablet_bootstrap.cc:654] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.861508 11376 tablet_bootstrap.cc:654] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.866344 11374 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.868351 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:54.870467 11501 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:54.869968 11079 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:54.872125 11374 ts_tablet_manager.cc:1434] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:11:54.873337 11374 tablet_bootstrap.cc:492] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:54.908262 11375 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:54.909054 11375 ts_tablet_manager.cc:1403] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.025s
I20260709 12:11:54.912035 11375 raft_consensus.cc:359] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.912767 11375 raft_consensus.cc:385] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.913064 11375 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.913796 11375 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.916534 11078 catalog_manager.cc:5697] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:54.918583 11078 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:54.919214 11375 ts_tablet_manager.cc:1434] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 12:11:54.920308 11375 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:54.920302 11078 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:54.924170 11374 tablet_bootstrap.cc:654] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:54.926026 11528 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:11:54.930099 11375 tablet_bootstrap.cc:654] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:54.933320 11376 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:54.933849 11376 ts_tablet_manager.cc:1403] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.121s	user 0.011s	sys 0.013s
I20260709 12:11:54.942883 11376 raft_consensus.cc:359] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.948364 11376 raft_consensus.cc:385] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.949046 11499 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.949611 11499 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.948690 11376 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.950870 11376 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.946821 11076 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:54.957581 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:54.958442 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:54.959081 11296 leader_election.cc:343] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.959790 11499 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:11:54.959827 11296 leader_election.cc:343] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:54.960355 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:54.961257 11501 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:54.955567 11076 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:54.963896 11376 ts_tablet_manager.cc:1434] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 12:11:54.964862 11376 tablet_bootstrap.cc:492] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:54.965778 11375 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:54.966259 11375 ts_tablet_manager.cc:1403] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.046s	user 0.022s	sys 0.014s
I20260709 12:11:54.969143 11375 raft_consensus.cc:359] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.969861 11375 raft_consensus.cc:385] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.970150 11375 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.970960 11375 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.973196 11375 ts_tablet_manager.cc:1434] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:11:54.974205 11375 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:54.977707 11374 tablet_bootstrap.cc:492] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:54.978173 11374 ts_tablet_manager.cc:1403] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.105s	user 0.012s	sys 0.004s
I20260709 12:11:54.978716 11501 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:54.980757 11375 tablet_bootstrap.cc:654] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:54.985939 11223 consensus_peers.cc:597] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:54.990655 11374 raft_consensus.cc:359] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.991369 11374 raft_consensus.cc:385] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:54.991628 11374 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:54.992305 11374 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:54.994386 11374 ts_tablet_manager.cc:1434] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:11:54.995359 11374 tablet_bootstrap.cc:492] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:54.997081 11376 tablet_bootstrap.cc:654] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.001317 11374 tablet_bootstrap.cc:654] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.006862 11375 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:55.007337 11375 ts_tablet_manager.cc:1403] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.009s
I20260709 12:11:55.009725 11375 raft_consensus.cc:359] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.010552 11375 raft_consensus.cc:385] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.010859 11375 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.011572 11375 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.012363 11376 tablet_bootstrap.cc:492] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:55.012907 11376 ts_tablet_manager.cc:1403] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.000s
I20260709 12:11:55.013902 11375 ts_tablet_manager.cc:1434] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:11:55.015144 11375 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:55.019134 11504 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:55.019426 11426 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:55.020210 11376 raft_consensus.cc:359] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.020887 11376 raft_consensus.cc:385] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.021149 11376 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.021379 11375 tablet_bootstrap.cc:654] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.029764 11180 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.039716 11376 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.040566 11501 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.041021 11501 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.048482 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:11:55.050329 11296 leader_election.cc:343] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:55.054237 11374 tablet_bootstrap.cc:492] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:55.054836 11374 ts_tablet_manager.cc:1403] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.060s	user 0.006s	sys 0.011s
I20260709 12:11:55.055073 11501 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:55.056402 11376 ts_tablet_manager.cc:1434] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.001s
I20260709 12:11:55.058028 11426 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:55.058166 11374 raft_consensus.cc:359] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.058992 11374 raft_consensus.cc:385] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.059319 11374 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.059264 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.060082 11254 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:55.063235 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:55.064141 11499 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:55.064538 11499 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.064972 11499 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.060173 11374 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.075569 11376 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:55.094188 11499 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.095640 11416 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:55.099089 11374 ts_tablet_manager.cc:1434] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:11:55.100046 11374 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:55.105005 11375 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:55.106983 11499 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:55.111038 11376 tablet_bootstrap.cc:654] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.113271 11374 tablet_bootstrap.cc:654] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.115756 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:55.116370 11254 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.105509 11375 ts_tablet_manager.cc:1403] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.010s
I20260709 12:11:55.132227 11375 raft_consensus.cc:359] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.133550 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:11:55.134862 11296 leader_election.cc:343] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:55.142516 11375 raft_consensus.cc:385] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.142936 11375 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.143893 11376 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:55.143664 11375 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.144384 11376 ts_tablet_manager.cc:1403] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.007s
I20260709 12:11:55.147514 11376 raft_consensus.cc:359] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.147848 11403 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.149322 11376 raft_consensus.cc:385] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.149684 11376 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.149492 11403 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.150447 11376 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.151681 11403 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.153311 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.156052 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.156857 11328 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:55.158145 11148 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:55.158840 11375 ts_tablet_manager.cc:1434] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 12:11:55.159356 11403 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.159880 11375 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:55.145279 11254 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:55.161712 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:55.162137 11376 ts_tablet_manager.cc:1434] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:11:55.162858 11499 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:55.163374 11499 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:55.164252 11499 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.166497 11375 tablet_bootstrap.cc:654] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.167273 11403 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:55.170580 11376 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:55.171959 11374 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:55.174268 11374 ts_tablet_manager.cc:1403] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.001s
I20260709 12:11:55.179942 11374 raft_consensus.cc:359] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.181742 11374 raft_consensus.cc:385] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.182119 11374 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.182998 11374 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.186043 11376 tablet_bootstrap.cc:654] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.187251 11374 ts_tablet_manager.cc:1434] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:11:55.188694 11374 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:55.194550 11374 tablet_bootstrap.cc:654] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.204918 11399 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.205514 11399 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.210068 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.211627 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:11:55.212102 11223 leader_election.cc:343] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:55.212610 11375 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:55.213088 11375 ts_tablet_manager.cc:1403] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.008s
I20260709 12:11:55.213346 11379 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:11:55.213621 11222 leader_election.cc:343] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:55.214099 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:55.214838 11379 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:55.215809 11375 raft_consensus.cc:359] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.216501 11375 raft_consensus.cc:385] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.216776 11375 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.217398 11375 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.219370 11375 ts_tablet_manager.cc:1434] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:11:55.218868 11077 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:55.220781 11375 tablet_bootstrap.cc:492] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:55.227905 11375 tablet_bootstrap.cc:654] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.228488 11399 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.243799 11379 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:55.254914 11403 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.256155 11374 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:55.255686 11403 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.256676 11374 ts_tablet_manager.cc:1403] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.068s	user 0.031s	sys 0.007s
I20260709 12:11:55.257849 11375 tablet_bootstrap.cc:492] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:55.258409 11375 ts_tablet_manager.cc:1403] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.003s
I20260709 12:11:55.259589 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.260344 11254 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:55.260396 11374 raft_consensus.cc:359] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.260846 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.261226 11374 raft_consensus.cc:385] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.261605 11374 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.261338 11375 raft_consensus.cc:359] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.262087 11375 raft_consensus.cc:385] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.262003 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.262502 11375 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.262521 11374 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.263010 11403 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.263278 11483 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:55.263330 11375 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.263860 11483 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:55.264187 11148 leader_election.cc:343] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:55.264322 11483 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.265316 11374 ts_tablet_manager.cc:1434] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:11:55.267339 11374 tablet_bootstrap.cc:492] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:55.271894 11483 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.272961 11376 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:55.273461 11376 ts_tablet_manager.cc:1403] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.103s	user 0.019s	sys 0.015s
I20260709 12:11:55.274392 11374 tablet_bootstrap.cc:654] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.275553 11375 ts_tablet_manager.cc:1434] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:11:55.276221 11376 raft_consensus.cc:359] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.277009 11376 raft_consensus.cc:385] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.277297 11376 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.278992 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:55.279850 11254 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.286409 11379 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:55.290002 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:11:55.291615 11148 leader_election.cc:343] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:55.292469 11483 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.293387 11375 tablet_bootstrap.cc:492] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:55.301589 11379 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.301913 11254 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:55.302229 11379 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.303999 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:55.305478 11379 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.305606 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:55.305855 11483 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:55.306553 11179 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:55.308002 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.309267 11503 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:55.309803 11503 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.310338 11503 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.313005 11483 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:55.314642 11483 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.317283 11376 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.319871 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.329406 11328 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:55.321424 11504 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:11:55.358757 11376 ts_tablet_manager.cc:1434] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.000s
I20260709 12:11:55.359917 11376 tablet_bootstrap.cc:492] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:55.360596 11374 tablet_bootstrap.cc:492] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:55.361433 11374 ts_tablet_manager.cc:1403] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.094s	user 0.031s	sys 0.030s
I20260709 12:11:55.361310 11503 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.364645 11374 raft_consensus.cc:359] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.365386 11374 raft_consensus.cc:385] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.365734 11374 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.366467 11374 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.371603 11374 ts_tablet_manager.cc:1434] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 12:11:55.372025 11416 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:55.372575 11374 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:55.368911 11376 tablet_bootstrap.cc:654] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.369668 11375 tablet_bootstrap.cc:654] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:55.374305 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 43620.9us
I20260709 12:11:55.414628 11503 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.427256 11403 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.428023 11403 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.440066 11379 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:55.439208 11403 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.462087 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.462873 11328 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:55.476279 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:55.478626 11483 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:55.479230 11483 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.479736 11483 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.487262 11501 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:11:55.497309 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 120673us
I20260709 12:11:55.499301 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:55.499989 11180 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.548903 11383 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.576534 11482 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:55.561543 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 167454us
I20260709 12:11:55.589958 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:55.590672 11325 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.590975 11375 tablet_bootstrap.cc:492] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:55.591509 11375 ts_tablet_manager.cc:1403] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.315s	user 0.016s	sys 0.026s
I20260709 12:11:55.591679 11416 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.592252 11416 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.594602 11416 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:55.594538 11375 raft_consensus.cc:359] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.595319 11375 raft_consensus.cc:385] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.595605 11375 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.582188 11482 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.596297 11375 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.596721 11563 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.628921 11376 tablet_bootstrap.cc:492] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:55.582185 11383 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.630966 11376 ts_tablet_manager.cc:1403] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.271s	user 0.032s	sys 0.061s
I20260709 12:11:55.636492 11426 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.638118 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.632874 11563 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.647637 11383 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:55.648614 11563 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:55.652438 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:55.655006 11482 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.659442 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.660346 11256 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:55.662281 11254 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:55.662690 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.664680 11403 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:55.665606 11403 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.666172 11403 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.667330 11374 tablet_bootstrap.cc:654] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:55.674113 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 117011us
I20260709 12:11:55.637184 11426 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.645490 11180 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:55.687954 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:55.688325 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:55.682320 11483 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.697093 11376 raft_consensus.cc:359] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.698081 11376 raft_consensus.cc:385] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:55.698460 11376 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:55.699327 11376 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:55.702663 11570 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.113s	user 0.004s	sys 0.000s
I20260709 12:11:55.700346 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.705550 11426 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.701565 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.707589 11253 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:55.703852 11376 ts_tablet_manager.cc:1434] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.001s
I20260709 12:11:55.710846 11376 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:55.710934 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:55.711709 11257 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:55.714200 11078 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:55.718230 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.692751 11483 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.724886 11574 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:55.724107 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.725483 11574 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.725926 11574 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.726054 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:55.724375 11375 ts_tablet_manager.cc:1434] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.133s	user 0.014s	sys 0.018s
I20260709 12:11:55.727716 11375 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:55.733170 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:55.739763 11251 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.729344 11416 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:55.743268 11416 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.743848 11416 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.753602 11177 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.730290 11379 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 12:11:55.754309 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:55.818269 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 86732.5us
I20260709 12:11:55.783826 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.821008 11326 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:55.820787 11325 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:55.843339 11567 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.843319 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.844110 11567 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.845885 11504 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:55.846550 11504 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:55.847409 11504 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.850066 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.850291 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:55.850708 11330 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:55.851207 11176 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:55.830745 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:55.859529 11403 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.881101 11567 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.884186 11574 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:55.886018 11251 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:55.901078 11574 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.890123 11576 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:55.889001 11328 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:55.890334 11148 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:55.905341 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:55.911355 11182 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:11:55.906778 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46039.1us
I20260709 12:11:55.910467 11327 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:55.905985 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.908164 11223 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:55.926285 11574 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:55.964843 11374 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:55.965408 11374 ts_tablet_manager.cc:1403] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.593s	user 0.034s	sys 0.078s
I20260709 12:11:55.967109 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:55.927784 11504 raft_consensus.cc:2764] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:11:55.973609 11376 tablet_bootstrap.cc:654] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.975251 11375 tablet_bootstrap.cc:654] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:55.929193 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:55.964859 11574 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:55.977677 11325 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:55.979508 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48492.8us
I20260709 12:11:55.988657 11403 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:55.989862 11383 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:11:56.009272 11374 raft_consensus.cc:359] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.010021 11374 raft_consensus.cc:385] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.010275 11374 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.010862 11374 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.011905 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.012521 11255 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.013935 11416 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.016052 11416 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.016304 11374 ts_tablet_manager.cc:1434] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:11:56.017246 11374 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.017992 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.018560 11251 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:56.012745 11223 leader_election.cc:343] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:56.021944 11222 leader_election.cc:343] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.022377 11329 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:56.022248 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:55.995718 11574 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.023141 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:56.037147 11181 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:56.038864 11482 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.039379 11482 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.044272 11251 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:56.047472 11482 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.049964 11582 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:11:56.058377 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:56.063010 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.066206 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.066123 11426 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.070428 11426 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:56.066377 11574 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:11:56.072402 11426 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.078614 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:56.023295 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:56.083510 11374 tablet_bootstrap.cc:654] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.068862 11255 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:56.075064 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.082202 11500 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:56.089881 11567 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:56.099701 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:56.100855 11482 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:11:56.084764 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72063.2us
I20260709 12:11:56.107635 11482 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:56.108534 11482 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.097651 11325 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:56.134567 11375 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.135040 11375 ts_tablet_manager.cc:1403] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.408s	user 0.011s	sys 0.019s
I20260709 12:11:56.136301 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:56.145241 11575 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:11:56.146134 11376 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.146636 11376 ts_tablet_manager.cc:1403] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.436s	user 0.024s	sys 0.004s
I20260709 12:11:56.138737 11328 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:56.154325 11375 raft_consensus.cc:359] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.155079 11375 raft_consensus.cc:385] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.155347 11375 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.155997 11375 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.165920 11375 ts_tablet_manager.cc:1434] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:11:56.167351 11375 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
W20260709 12:11:56.169027 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 150255us
I20260709 12:11:56.167567 11376 raft_consensus.cc:359] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.170404 11376 raft_consensus.cc:385] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.170742 11376 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.171747 11376 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.173926 11376 ts_tablet_manager.cc:1434] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 12:11:56.174731 11376 tablet_bootstrap.cc:492] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.214215 11076 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:56.223278 11076 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:56.232426 11374 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.242033 11376 tablet_bootstrap.cc:654] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.237015 11374 ts_tablet_manager.cc:1403] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.220s	user 0.013s	sys 0.009s
I20260709 12:11:56.250631 11376 tablet_bootstrap.cc:492] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.251135 11376 ts_tablet_manager.cc:1403] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.077s	user 0.013s	sys 0.001s
I20260709 12:11:56.251662 11374 raft_consensus.cc:359] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.253604 11374 raft_consensus.cc:385] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.254513 11374 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.258051 11374 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.260694 11076 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:56.256631 11375 tablet_bootstrap.cc:654] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.265659 11374 ts_tablet_manager.cc:1434] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 12:11:56.266063 11376 raft_consensus.cc:359] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.267035 11376 raft_consensus.cc:385] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.267370 11376 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.268395 11374 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.268244 11376 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.285776 11376 ts_tablet_manager.cc:1434] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.001s
I20260709 12:11:56.286808 11376 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.289937 11375 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.290458 11375 ts_tablet_manager.cc:1403] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.123s	user 0.012s	sys 0.006s
I20260709 12:11:56.294731 11327 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.296546 11375 raft_consensus.cc:359] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.297577 11375 raft_consensus.cc:385] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.297995 11375 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.298575 11482 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.299067 11482 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.299178 11375 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.301209 11482 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.302381 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.302834 11375 ts_tablet_manager.cc:1434] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.005s
W20260709 12:11:56.303958 11148 leader_election.cc:343] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.308812 11375 tablet_bootstrap.cc:492] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.314314 11376 tablet_bootstrap.cc:654] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.317886 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.318620 11252 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:56.319875 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:56.323150 11482 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.323525 11482 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.323839 11482 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.330119 11375 tablet_bootstrap.cc:654] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.330320 11077 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:56.332203 11482 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.333559 11374 tablet_bootstrap.cc:654] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.340698 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.341385 11252 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.343091 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:56.344110 11374 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.344585 11374 ts_tablet_manager.cc:1403] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.076s	user 0.015s	sys 0.003s
W20260709 12:11:56.344497 11148 leader_election.cc:343] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.345256 11482 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.347852 11376 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.347999 11252 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:56.348311 11376 ts_tablet_manager.cc:1403] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.000s
I20260709 12:11:56.349162 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:56.351886 11375 tablet_bootstrap.cc:492] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.352329 11375 ts_tablet_manager.cc:1403] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.000s
I20260709 12:11:56.353726 11482 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.354224 11482 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:56.354866 11375 raft_consensus.cc:359] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.355489 11375 raft_consensus.cc:385] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.355067 11482 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.355743 11375 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.356434 11375 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.352123 11376 raft_consensus.cc:359] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.358314 11375 ts_tablet_manager.cc:1434] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:11:56.358471 11376 raft_consensus.cc:385] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.358803 11376 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.359171 11375 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.359421 11376 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.362282 11574 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.362107 11374 raft_consensus.cc:359] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.362803 11374 raft_consensus.cc:385] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.362736 11574 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.363148 11374 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.363772 11374 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.364913 11574 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.365496 11376 ts_tablet_manager.cc:1434] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.003s
I20260709 12:11:56.365856 11374 ts_tablet_manager.cc:1434] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.001s
I20260709 12:11:56.366464 11376 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.366719 11374 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.367810 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:56.372242 11147 leader_election.cc:343] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.375192 11376 tablet_bootstrap.cc:654] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.377072 11374 tablet_bootstrap.cc:654] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.375234 11375 tablet_bootstrap.cc:654] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.380939 11503 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:56.389424 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.390513 11375 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.390997 11375 ts_tablet_manager.cc:1403] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.004s
W20260709 12:11:56.391125 11148 leader_election.cc:343] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.391582 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:56.393041 11574 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:56.393888 11375 raft_consensus.cc:359] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.394603 11375 raft_consensus.cc:385] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.394857 11375 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.395478 11375 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.397622 11375 ts_tablet_manager.cc:1434] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 12:11:56.401403 11375 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.401934 11416 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.402405 11416 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.404501 11416 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.406435 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.407157 11177 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:56.407522 11375 tablet_bootstrap.cc:654] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.413374 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:56.425271 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.426363 11252 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:56.421437 11577 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.430562 11577 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.430938 11577 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.450095 11503 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:56.465878 11577 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.468731 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:56.469341 11180 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.471624 11574 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:56.473035 11374 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.473546 11374 ts_tablet_manager.cc:1403] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.107s	user 0.033s	sys 0.027s
I20260709 12:11:56.476178 11374 raft_consensus.cc:359] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.476828 11374 raft_consensus.cc:385] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.477094 11374 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.478174 11577 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.480189 11078 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:56.481518 11374 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.484818 11374 ts_tablet_manager.cc:1434] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:11:56.479560 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.485522 11255 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.488894 11577 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.489509 11577 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.490026 11566 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.490876 11374 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.490502 11566 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.491959 11577 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.492617 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.493284 11182 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:56.494184 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.494781 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:11:56.495625 11222 leader_election.cc:343] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.496237 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.496975 11257 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:56.497042 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:56.498531 11223 leader_election.cc:343] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.499145 11223 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:56.492636 11566 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.499936 11567 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:56.504921 11255 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:56.507246 11375 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.507934 11375 ts_tablet_manager.cc:1403] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.107s	user 0.015s	sys 0.007s
I20260709 12:11:56.508960 11180 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:56.510129 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:56.510869 11416 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.511234 11416 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.511134 11375 raft_consensus.cc:359] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.511363 11482 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:56.511677 11416 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.511834 11375 raft_consensus.cc:385] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.512013 11374 tablet_bootstrap.cc:654] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.512200 11375 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.525385 11577 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.526540 11376 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.527129 11376 ts_tablet_manager.cc:1403] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.161s	user 0.025s	sys 0.026s
I20260709 12:11:56.521310 11416 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.530797 11375 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.532761 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:56.533344 11177 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.533416 11375 ts_tablet_manager.cc:1434] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:11:56.534511 11375 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.540647 11177 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:56.541373 11375 tablet_bootstrap.cc:654] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.541482 11482 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:56.541864 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:56.542606 11582 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.543097 11582 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:56.543955 11582 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.548715 11577 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:56.549623 11374 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.550132 11374 ts_tablet_manager.cc:1403] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.012s
I20260709 12:11:56.552649 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.553200 11374 raft_consensus.cc:359] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.549678 11577 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.553730 11416 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.553267 11255 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.560712 11375 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.561254 11375 ts_tablet_manager.cc:1403] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.000s
I20260709 12:11:56.560982 11575 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:56.564071 11375 raft_consensus.cc:359] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.564767 11375 raft_consensus.cc:385] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.565069 11375 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.565510 11482 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:56.565845 11375 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.567616 11575 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:56.568367 11375 ts_tablet_manager.cc:1434] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:11:56.574435 11376 raft_consensus.cc:359] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.575316 11574 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.575876 11574 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.578347 11574 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.578704 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.579991 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.580403 11256 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
W20260709 12:11:56.581427 11148 leader_election.cc:343] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.582077 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:56.575418 11376 raft_consensus.cc:385] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.583056 11376 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.583287 11574 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.583863 11574 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.583766 11376 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.584384 11574 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.586591 11255 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:56.588127 11374 raft_consensus.cc:385] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.588475 11374 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.593272 11574 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.593425 11374 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.596311 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.596977 11256 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.598392 11374 ts_tablet_manager.cc:1434] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 12:11:56.603919 11256 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:56.604962 11574 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.606093 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:56.606114 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:11:56.607735 11148 leader_election.cc:343] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.608675 11575 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.609211 11575 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:56.610432 11376 ts_tablet_manager.cc:1434] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.003s
I20260709 12:11:56.610065 11575 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.614660 11374 tablet_bootstrap.cc:492] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.619746 11374 tablet_bootstrap.cc:654] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.619925 11376 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.620808 11078 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:56.627734 11376 tablet_bootstrap.cc:654] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.629554 11375 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.634508 11374 tablet_bootstrap.cc:492] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.635505 11374 ts_tablet_manager.cc:1403] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.001s
I20260709 12:11:56.638263 11376 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.638723 11376 ts_tablet_manager.cc:1403] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.005s
I20260709 12:11:56.638391 11374 raft_consensus.cc:359] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.639178 11374 raft_consensus.cc:385] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.639494 11374 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.640165 11374 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.641060 11376 raft_consensus.cc:359] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.641676 11376 raft_consensus.cc:385] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.641943 11376 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.642215 11374 ts_tablet_manager.cc:1434] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:11:56.642365 11375 tablet_bootstrap.cc:654] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.642627 11376 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.643255 11374 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.644923 11376 ts_tablet_manager.cc:1434] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:11:56.645841 11376 tablet_bootstrap.cc:492] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.650199 11374 tablet_bootstrap.cc:654] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.651695 11376 tablet_bootstrap.cc:654] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.658931 11374 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.659394 11374 ts_tablet_manager.cc:1403] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.005s
I20260709 12:11:56.661852 11374 raft_consensus.cc:359] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.662549 11374 raft_consensus.cc:385] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.662804 11374 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.664294 11078 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:56.671741 11374 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.673796 11374 ts_tablet_manager.cc:1434] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:11:56.674788 11374 tablet_bootstrap.cc:492] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.676254 11375 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.676872 11375 ts_tablet_manager.cc:1403] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.003s
I20260709 12:11:56.680435 11374 tablet_bootstrap.cc:654] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.697115 11374 tablet_bootstrap.cc:492] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.697598 11374 ts_tablet_manager.cc:1403] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.015s
I20260709 12:11:56.699692 11375 raft_consensus.cc:359] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.700199 11374 raft_consensus.cc:359] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.701840 11375 raft_consensus.cc:385] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.702083 11374 raft_consensus.cc:385] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.702229 11375 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.702560 11374 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.701866 11077 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:56.703298 11374 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.705412 11375 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.706174 11374 ts_tablet_manager.cc:1434] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:11:56.707948 11375 ts_tablet_manager.cc:1434] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.003s
I20260709 12:11:56.708876 11375 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.708876 11374 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.713340 11376 tablet_bootstrap.cc:492] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.713850 11376 ts_tablet_manager.cc:1403] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.025s
I20260709 12:11:56.716807 11374 tablet_bootstrap.cc:654] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.716487 11376 raft_consensus.cc:359] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.717214 11376 raft_consensus.cc:385] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.717278 11375 tablet_bootstrap.cc:654] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.717569 11376 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.718446 11376 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.720772 11376 ts_tablet_manager.cc:1434] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:11:56.721679 11376 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.727269 11375 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.728073 11376 tablet_bootstrap.cc:654] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.728072 11375 ts_tablet_manager.cc:1403] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260709 12:11:56.730649 11567 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.731148 11567 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.732089 11374 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.732549 11374 ts_tablet_manager.cc:1403] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.015s
I20260709 12:11:56.733309 11375 raft_consensus.cc:359] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.734046 11375 raft_consensus.cc:385] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.734387 11375 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.735050 11375 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.735126 11374 raft_consensus.cc:359] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.735884 11374 raft_consensus.cc:385] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.736181 11374 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.736027 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.737044 11177 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:56.737707 11375 ts_tablet_manager.cc:1434] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:11:56.737150 11374 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.738739 11375 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.739720 11374 ts_tablet_manager.cc:1434] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:11:56.740620 11374 tablet_bootstrap.cc:492] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.745513 11375 tablet_bootstrap.cc:654] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.746368 11374 tablet_bootstrap.cc:654] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.748499 11376 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.749045 11376 ts_tablet_manager.cc:1403] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.007s
I20260709 12:11:56.752671 11575 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.753189 11575 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.755300 11575 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.755820 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:56.756704 11566 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.757437 11567 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.757041 11376 raft_consensus.cc:359] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.757998 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.757073 11566 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.759111 11566 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.758658 11256 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:56.758031 11376 raft_consensus.cc:385] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.761014 11376 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.761014 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.761919 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.761952 11376 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:56.763243 11148 leader_election.cc:343] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.762624 11327 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:56.764379 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:56.765026 11376 ts_tablet_manager.cc:1434] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:11:56.765767 11574 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:11:56.770351 11376 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.766700 11575 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.776775 11575 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.777287 11575 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.785146 11566 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.790423 11376 tablet_bootstrap.cc:654] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.795523 11575 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.800395 11575 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.800599 11504 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.801142 11504 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.803510 11504 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.804960 11376 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.805477 11376 ts_tablet_manager.cc:1403] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.002s
I20260709 12:11:56.808324 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.809070 11177 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:56.808655 11376 raft_consensus.cc:359] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.809300 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.809657 11376 raft_consensus.cc:385] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.809952 11251 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.810077 11376 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.810292 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:56.810868 11180 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.810972 11376 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.813987 11376 ts_tablet_manager.cc:1434] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:11:56.815205 11376 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.818018 11567 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.818545 11567 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.821144 11567 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.822331 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:56.822984 11331 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.823767 11376 tablet_bootstrap.cc:654] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:56.826020 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.826676 11330 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:56.830168 11330 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.836023 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:56.839344 11503 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.839787 11503 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.840160 11503 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.840240 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:56.817098 11251 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:56.852017 11503 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.854547 11482 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:56.856119 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:56.856515 11180 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
W20260709 12:11:56.857112 11223 leader_election.cc:343] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.862221 11375 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:56.862679 11375 ts_tablet_manager.cc:1403] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.124s	user 0.027s	sys 0.034s
I20260709 12:11:56.872752 11375 raft_consensus.cc:359] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.873466 11375 raft_consensus.cc:385] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.874471 11375 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.874666 11499 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.875183 11499 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.875262 11375 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.877276 11499 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.877923 11375 ts_tablet_manager.cc:1434] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.005s
I20260709 12:11:56.875614 11374 tablet_bootstrap.cc:492] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:56.879285 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.879521 11374 ts_tablet_manager.cc:1403] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.139s	user 0.016s	sys 0.008s
I20260709 12:11:56.880759 11375 tablet_bootstrap.cc:492] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:56.879905 11181 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:56.881871 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:56.883170 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:56.865311 11567 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.883776 11567 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.884120 11567 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.880015 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:56.803512 11566 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.867569 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.880493 11577 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.901139 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:56.901859 11577 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.900614 11376 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:56.903821 11376 ts_tablet_manager.cc:1403] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.089s	user 0.020s	sys 0.024s
I20260709 12:11:56.892325 11499 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:56.905834 11482 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.913872 11482 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:11:56.908636 11148 leader_election.cc:343] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.915824 11503 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.915704 11482 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.905918 11177 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:56.917281 11331 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:56.920790 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:56.907023 11499 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.921423 11177 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.921479 11499 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.926151 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.927382 11577 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.929241 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.929168 11499 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.931967 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:56.932562 11181 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:56.934239 11296 leader_election.cc:343] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.937083 11376 raft_consensus.cc:359] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.937462 11177 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:56.937846 11376 raft_consensus.cc:385] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.938401 11376 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.939972 11376 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.945781 11376 ts_tablet_manager.cc:1434] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 12:11:56.947322 11376 tablet_bootstrap.cc:492] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:56.947760 11374 raft_consensus.cc:359] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.948593 11374 raft_consensus.cc:385] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:56.948881 11374 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:56.949545 11374 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:56.934288 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:56.938108 11567 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.954169 11641 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.954685 11374 ts_tablet_manager.cc:1434] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.003s
I20260709 12:11:56.960130 11499 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:56.938669 11642 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.962247 11567 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.964270 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:56.966313 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:56.968248 11504 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:56.969156 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:56.971403 11374 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:56.963418 11642 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:56.973049 11078 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:56.974521 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:56.976485 11641 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.977276 11180 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.970754 11504 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:56.979352 11641 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:56.979574 11504 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:56.985001 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:56.986260 11296 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:56.987747 11577 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:11:56.987843 11416 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:11:56.993532 11223 leader_election.cc:343] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:56.995469 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:56.996129 11331 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:56.996188 11642 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.002075 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.002734 11327 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:57.002858 11180 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
W20260709 12:11:57.008302 11296 leader_election.cc:343] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.010974 11331 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:57.011132 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:56.981801 11181 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:57.014578 11375 tablet_bootstrap.cc:654] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.016023 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:57.017586 11567 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:57.018040 11567 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.017982 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:57.019052 11416 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.019567 11567 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.019520 11416 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:57.020584 11416 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.031001 11374 tablet_bootstrap.cc:654] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.034483 11577 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:57.026407 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:57.074034 11078 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:57.077889 11641 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.078495 11641 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:57.079416 11567 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.079711 11641 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.083959 11376 tablet_bootstrap.cc:654] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.086292 11416 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.081511 11567 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:57.084863 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61146.8us
W20260709 12:11:57.095391 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67513.9us
W20260709 12:11:57.114485 11222 leader_election.cc:343] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.122339 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:57.122968 11325 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.127533 11077 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:57.130803 11077 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:57.135535 11325 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:57.136622 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:57.137737 11504 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.138042 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:57.138365 11504 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:57.139580 11504 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:57.140983 11222 leader_election.cc:343] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.151762 11374 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:57.155791 11376 tablet_bootstrap.cc:492] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:57.160584 11499 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.175345 11376 ts_tablet_manager.cc:1403] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.228s	user 0.014s	sys 0.006s
I20260709 12:11:57.176313 11374 ts_tablet_manager.cc:1403] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.205s	user 0.027s	sys 0.024s
I20260709 12:11:57.185393 11654 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:57.188123 11642 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.188719 11642 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.191100 11642 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.194869 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.195559 11642 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.200455 11376 raft_consensus.cc:359] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:57.198654 11223 leader_election.cc:343] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.201574 11376 raft_consensus.cc:385] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.201933 11376 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.202315 11375 tablet_bootstrap.cc:492] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:57.202821 11375 ts_tablet_manager.cc:1403] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.322s	user 0.016s	sys 0.019s
I20260709 12:11:57.203519 11376 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:57.204614 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 114380us
I20260709 12:11:57.214901 11374 raft_consensus.cc:359] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.215588 11374 raft_consensus.cc:385] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.215859 11374 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.198132 11642 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.249382 11376 ts_tablet_manager.cc:1434] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.073s	user 0.016s	sys 0.009s
I20260709 12:11:57.250627 11376 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:57.249591 11374 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.249591 11375 raft_consensus.cc:359] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.255347 11375 raft_consensus.cc:385] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.255695 11375 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.254604 11577 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.262076 11375 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.258767 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.263437 11577 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.269651 11577 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:11:57.271809 11222 leader_election.cc:343] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.275679 11077 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:57.276784 11077 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:57.274127 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:57.278388 11641 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:57.278419 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:57.280169 11642 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.282022 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.282092 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.282899 11325 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:57.283860 11330 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.286760 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:57.293222 11375 ts_tablet_manager.cc:1434] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.090s	user 0.013s	sys 0.004s
I20260709 12:11:57.291299 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:57.294256 11642 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:57.294682 11642 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.295049 11642 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.299558 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.305090 11374 ts_tablet_manager.cc:1434] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.128s	user 0.007s	sys 0.008s
I20260709 12:11:57.306219 11374 tablet_bootstrap.cc:492] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
W20260709 12:11:57.307313 11222 leader_election.cc:343] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.313040 11577 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:11:57.318697 11148 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:57.313323 11642 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.328354 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:11:57.331032 11222 leader_election.cc:343] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.331259 11375 tablet_bootstrap.cc:492] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:57.331672 11374 tablet_bootstrap.cc:654] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.334153 11642 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.335623 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:57.336256 11325 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.336987 11376 tablet_bootstrap.cc:654] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.341827 11426 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:57.363088 11325 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:57.364378 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:57.365062 11376 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:57.365546 11376 ts_tablet_manager.cc:1403] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.115s	user 0.009s	sys 0.008s
I20260709 12:11:57.365482 11641 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.366189 11641 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:57.366961 11374 tablet_bootstrap.cc:492] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:57.367354 11374 ts_tablet_manager.cc:1403] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.003s
I20260709 12:11:57.368343 11641 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.371613 11375 tablet_bootstrap.cc:654] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.372723 11376 raft_consensus.cc:359] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.373442 11376 raft_consensus.cc:385] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.373776 11376 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.374490 11376 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.377187 11376 ts_tablet_manager.cc:1434] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:11:57.378167 11376 tablet_bootstrap.cc:492] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:57.378386 11374 raft_consensus.cc:359] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.379076 11374 raft_consensus.cc:385] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.379304 11374 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.380043 11374 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.387845 11376 tablet_bootstrap.cc:654] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.388458 11374 ts_tablet_manager.cc:1434] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 12:11:57.389293 11374 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:57.392740 11375 tablet_bootstrap.cc:492] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:57.393353 11375 ts_tablet_manager.cc:1403] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.062s	user 0.006s	sys 0.006s
I20260709 12:11:57.396356 11375 raft_consensus.cc:359] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.397260 11375 raft_consensus.cc:385] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.397671 11375 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.398500 11374 tablet_bootstrap.cc:654] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.398628 11375 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.403401 11375 ts_tablet_manager.cc:1434] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:11:57.405395 11375 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:57.406265 11079 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:57.409215 11376 tablet_bootstrap.cc:492] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:57.409749 11376 ts_tablet_manager.cc:1403] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.000s
I20260709 12:11:57.413462 11375 tablet_bootstrap.cc:654] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.413395 11376 raft_consensus.cc:359] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.414286 11376 raft_consensus.cc:385] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.414602 11376 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.415304 11374 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:57.415423 11376 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.416190 11374 ts_tablet_manager.cc:1403] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.010s
I20260709 12:11:57.418483 11642 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.419910 11376 ts_tablet_manager.cc:1434] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:11:57.420941 11376 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:57.425130 11374 raft_consensus.cc:359] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.426074 11374 raft_consensus.cc:385] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.426383 11374 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.427351 11374 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.430050 11376 tablet_bootstrap.cc:654] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.430850 11374 ts_tablet_manager.cc:1434] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 12:11:57.432363 11374 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:57.441318 11376 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:57.446645 11330 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.446863 11376 ts_tablet_manager.cc:1403] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.005s
I20260709 12:11:57.449398 11374 tablet_bootstrap.cc:654] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.451200 11376 raft_consensus.cc:359] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.451968 11376 raft_consensus.cc:385] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.452289 11376 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.453600 11376 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.456327 11376 ts_tablet_manager.cc:1434] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:11:57.458423 11376 tablet_bootstrap.cc:492] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:57.467993 11577 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:57.475886 11374 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:57.475886 11375 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:57.476665 11374 ts_tablet_manager.cc:1403] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.008s
I20260709 12:11:57.476729 11375 ts_tablet_manager.cc:1403] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.014s
I20260709 12:11:57.476749 11503 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:57.485538 11503 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:11:57.491343 11252 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.492385 11374 raft_consensus.cc:359] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.493186 11374 raft_consensus.cc:385] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.496881 11376 tablet_bootstrap.cc:654] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.499882 11375 raft_consensus.cc:359] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.503502 11375 raft_consensus.cc:385] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.503888 11375 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.493515 11374 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.506155 11374 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.512526 11504 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.514086 11504 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.504638 11375 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.516563 11376 tablet_bootstrap.cc:492] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:57.517323 11376 ts_tablet_manager.cc:1403] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.002s
I20260709 12:11:57.517910 11374 ts_tablet_manager.cc:1434] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 12:11:57.518935 11374 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:57.520316 11426 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.516628 11375 ts_tablet_manager.cc:1434] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.039s	user 0.001s	sys 0.005s
I20260709 12:11:57.521770 11375 tablet_bootstrap.cc:492] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:57.521838 11376 raft_consensus.cc:359] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.522886 11376 raft_consensus.cc:385] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.523269 11376 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.526273 11577 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.526793 11577 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.529048 11577 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:57.524365 11376 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.530968 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.532064 11376 ts_tablet_manager.cc:1434] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:11:57.533026 11376 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
W20260709 12:11:57.534437 11296 leader_election.cc:343] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.534962 11577 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.535589 11503 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.537492 11503 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.539251 11426 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.539762 11426 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.543557 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:57.544278 11252 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:57.554746 11426 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.557119 11503 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.575944 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:57.577776 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:57.578025 11416 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:57.592453 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.592689 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.584425 11416 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.582297 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.605865 11376 tablet_bootstrap.cc:654] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.603259 11416 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.612942 11499 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.613487 11499 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.616397 11325 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:57.630036 11223 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:57.630911 11503 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:11:57.631335 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:57.632287 11575 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:57.659950 11674 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:11:57.677772 11503 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.678391 11503 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.680522 11503 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.683444 11499 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:11:57.684429 11676 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:11:57.689519 11374 tablet_bootstrap.cc:654] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.694283 11375 tablet_bootstrap.cc:654] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:57.694694 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59728.9us
W20260709 12:11:57.690721 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59483.7us
I20260709 12:11:57.700338 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.708735 11574 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.709460 11574 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.711262 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:57.711949 11252 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:57.712260 11574 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:11:57.714067 11296 leader_election.cc:343] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.718766 11416 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:57.692045 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 62110.7us
I20260709 12:11:57.722613 11577 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.723141 11577 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.724118 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:57.715255 11574 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.726586 11574 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.728329 11499 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:57.728768 11499 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.729175 11499 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.729287 11574 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.736142 11575 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:57.737799 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:57.738667 11252 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:57.739893 11577 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:57.740669 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.742305 11416 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:57.743064 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:57.745560 11503 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.745704 11499 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.746877 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.754253 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:57.755046 11376 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:57.755590 11376 ts_tablet_manager.cc:1403] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.223s	user 0.035s	sys 0.011s
I20260709 12:11:57.760334 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:57.761018 11255 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:57.741349 11181 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:57.765877 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:57.766844 11656 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:57.768430 11656 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.768795 11656 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.769801 11680 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:57.774511 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:57.775136 11376 raft_consensus.cc:359] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.776163 11376 raft_consensus.cc:385] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.776571 11376 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.770187 11680 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.778046 11680 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.778115 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:57.779395 11376 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:57.781236 11296 leader_election.cc:343] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.783272 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.783720 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:57.784606 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:57.783905 11330 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:57.785913 11331 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:57.785795 11642 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:57.787935 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:57.788158 11642 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.788563 11642 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.786377 11252 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.781164 11567 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:11:57.790772 11376 ts_tablet_manager.cc:1434] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.003s
I20260709 12:11:57.791699 11376 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
W20260709 12:11:57.792121 11296 leader_election.cc:343] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.799441 11499 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:57.801415 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:57.802132 11374 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:57.802098 11256 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.802682 11374 ts_tablet_manager.cc:1403] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.284s	user 0.013s	sys 0.002s
I20260709 12:11:57.806840 11504 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:57.808355 11374 raft_consensus.cc:359] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.809062 11374 raft_consensus.cc:385] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.808769 11656 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.809911 11252 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
W20260709 12:11:57.811076 11223 leader_election.cc:343] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.811532 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:57.814064 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:57.814543 11375 tablet_bootstrap.cc:492] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:57.815089 11375 ts_tablet_manager.cc:1403] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.294s	user 0.009s	sys 0.015s
I20260709 12:11:57.815421 11426 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.818043 11656 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.818145 11375 raft_consensus.cc:359] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.818905 11375 raft_consensus.cc:385] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:57.819175 11375 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.819849 11375 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.809324 11374 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:57.821434 11374 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.822801 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:11:57.824775 11296 leader_election.cc:343] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.826442 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:57.827018 11331 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.827898 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:57.823976 11642 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.830088 11499 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.836377 11499 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:57.833513 11374 ts_tablet_manager.cc:1434] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:11:57.835788 11256 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:57.823817 11680 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.839448 11375 ts_tablet_manager.cc:1434] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:11:57.840358 11374 tablet_bootstrap.cc:492] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:57.841521 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:57.842289 11375 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:57.845577 11577 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.858489 11376 tablet_bootstrap.cc:654] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.855396 11426 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:57.851534 11251 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:57.874150 11503 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.898068 11680 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:57.837445 11499 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.893853 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:57.846122 11577 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:57.907752 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:57.887229 11331 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:57.846015 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:57.905829 11642 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.908751 11567 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:11:57.910728 11503 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.917420 11374 tablet_bootstrap.cc:654] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.917376 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:57.918438 11182 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:57.912544 11252 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:57.920028 11567 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:57.919912 11577 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:57.931315 11296 leader_election.cc:343] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:57.952421 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:57.942094 11503 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:57.967216 11641 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:57.954326 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:11:57.975538 11375 tablet_bootstrap.cc:654] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:57.980088 11656 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.980611 11656 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:57.981452 11656 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:57.984932 11252 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:57.986439 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:57.989171 11376 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:57.989616 11376 ts_tablet_manager.cc:1403] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.198s	user 0.012s	sys 0.037s
I20260709 12:11:57.997438 11687 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:57.998106 11687 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:58.008152 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.020232 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.021776 11376 raft_consensus.cc:359] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.022698 11376 raft_consensus.cc:385] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.023172 11376 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.027011 11375 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:58.027660 11375 ts_tablet_manager.cc:1403] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.002s
I20260709 12:11:57.999006 11687 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:58.034904 11222 leader_election.cc:343] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.036607 11374 tablet_bootstrap.cc:492] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:58.037205 11374 ts_tablet_manager.cc:1403] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.198s	user 0.012s	sys 0.014s
I20260709 12:11:58.040078 11376 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.043864 11376 ts_tablet_manager.cc:1434] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 12:11:58.045791 11374 raft_consensus.cc:359] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.046473 11374 raft_consensus.cc:385] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.046694 11374 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.047260 11374 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.048846 11376 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.051941 11375 raft_consensus.cc:359] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.053090 11375 raft_consensus.cc:385] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.057910 11656 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.058457 11656 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.053406 11375 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
W20260709 12:11:58.053354 11223 leader_election.cc:343] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.066478 11223 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:58.067689 11374 ts_tablet_manager.cc:1434] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 12:11:58.068980 11374 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:58.070281 11692 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:58.079372 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.080322 11252 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:58.082235 11076 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:58.087706 11076 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:58.088572 11375 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.093735 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.096108 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:58.098380 11654 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.099821 11375 ts_tablet_manager.cc:1434] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 12:11:58.100080 11376 tablet_bootstrap.cc:654] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.100871 11375 tablet_bootstrap.cc:492] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:58.092595 11656 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.106388 11654 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:58.112591 11374 tablet_bootstrap.cc:654] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:58.121140 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:58.133608 11376 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:58.134178 11376 ts_tablet_manager.cc:1403] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.011s
I20260709 12:11:58.134541 11375 tablet_bootstrap.cc:654] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.136576 11376 raft_consensus.cc:359] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.137081 11376 raft_consensus.cc:385] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.137250 11376 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.138628 11376 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.140720 11376 ts_tablet_manager.cc:1434] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:11:58.142220 11376 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.147069 11077 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:58.149803 11376 tablet_bootstrap.cc:654] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.163167 11376 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:58.163650 11376 ts_tablet_manager.cc:1403] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260709 12:11:58.166297 11376 raft_consensus.cc:359] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.166985 11376 raft_consensus.cc:385] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.167249 11376 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.167304 11499 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.167785 11499 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.168038 11376 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.170935 11376 ts_tablet_manager.cc:1434] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:11:58.172315 11376 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.172214 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.176107 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.179601 11296 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:58.180509 11577 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:58.182520 11376 tablet_bootstrap.cc:654] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.170807 11499 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.188591 11076 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:58.197594 11374 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:58.198252 11374 ts_tablet_manager.cc:1403] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.130s	user 0.011s	sys 0.003s
I20260709 12:11:58.201159 11374 raft_consensus.cc:359] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.201946 11374 raft_consensus.cc:385] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.202246 11374 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.203035 11375 tablet_bootstrap.cc:492] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:58.202978 11374 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.203576 11375 ts_tablet_manager.cc:1403] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.103s	user 0.012s	sys 0.052s
I20260709 12:11:58.206336 11375 raft_consensus.cc:359] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.206998 11375 raft_consensus.cc:385] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.207273 11375 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.207917 11375 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.210075 11374 ts_tablet_manager.cc:1434] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:11:58.211069 11374 tablet_bootstrap.cc:492] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:58.211651 11376 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:58.212097 11376 ts_tablet_manager.cc:1403] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.000s
I20260709 12:11:58.214731 11376 raft_consensus.cc:359] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.215368 11376 raft_consensus.cc:385] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.215636 11376 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.216212 11376 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.217113 11374 tablet_bootstrap.cc:654] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.210099 11375 ts_tablet_manager.cc:1434] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 12:11:58.218261 11376 ts_tablet_manager.cc:1434] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:11:58.218614 11375 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:58.219089 11376 tablet_bootstrap.cc:492] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.224977 11375 tablet_bootstrap.cc:654] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.225608 11376 tablet_bootstrap.cc:654] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.242565 11374 tablet_bootstrap.cc:492] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:58.243103 11374 ts_tablet_manager.cc:1403] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.005s
I20260709 12:11:58.245934 11374 raft_consensus.cc:359] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.246703 11374 raft_consensus.cc:385] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.247004 11374 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.250917 11374 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.252744 11374 ts_tablet_manager.cc:1434] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:11:58.253796 11374 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:58.266414 11374 tablet_bootstrap.cc:654] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.286242 11376 tablet_bootstrap.cc:492] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:58.286778 11376 ts_tablet_manager.cc:1403] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.014s
I20260709 12:11:58.289737 11376 raft_consensus.cc:359] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.290516 11376 raft_consensus.cc:385] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.290822 11376 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.291651 11376 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.293270 11376 ts_tablet_manager.cc:1434] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.006s
I20260709 12:11:58.293342 11375 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:58.293864 11375 ts_tablet_manager.cc:1403] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.075s	user 0.019s	sys 0.037s
I20260709 12:11:58.294402 11376 tablet_bootstrap.cc:492] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.296376 11375 raft_consensus.cc:359] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.297030 11375 raft_consensus.cc:385] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.297278 11375 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.298404 11375 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.301842 11375 ts_tablet_manager.cc:1434] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.003s
I20260709 12:11:58.302750 11375 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:58.306674 11376 tablet_bootstrap.cc:654] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.306879 11499 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.312740 11375 tablet_bootstrap.cc:654] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.314602 11499 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.316684 11499 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.319283 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.319489 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:11:58.321138 11296 leader_election.cc:343] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.324123 11374 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:58.324786 11374 ts_tablet_manager.cc:1403] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.071s	user 0.028s	sys 0.006s
W20260709 12:11:58.326149 11296 leader_election.cc:343] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.326694 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:58.329088 11374 raft_consensus.cc:359] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.330073 11374 raft_consensus.cc:385] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.330426 11374 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.330593 11687 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:58.331496 11374 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.335291 11374 ts_tablet_manager.cc:1434] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.002s
I20260709 12:11:58.337838 11374 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:58.339018 11331 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.340236 11376 tablet_bootstrap.cc:492] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:58.340682 11376 ts_tablet_manager.cc:1403] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.012s
I20260709 12:11:58.343467 11376 raft_consensus.cc:359] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.344492 11376 raft_consensus.cc:385] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.345618 11374 tablet_bootstrap.cc:654] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.344776 11376 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.347661 11376 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.350484 11376 ts_tablet_manager.cc:1434] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:11:58.352533 11376 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.358953 11376 tablet_bootstrap.cc:654] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.361470 11654 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.362162 11654 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.364584 11654 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.366489 11577 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.367137 11577 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.367129 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.371616 11689 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.377974 11374 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:58.378808 11374 ts_tablet_manager.cc:1403] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.003s
W20260709 12:11:58.380025 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46325us
I20260709 12:11:58.372255 11689 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.397668 11689 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.381618 11499 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.398327 11499 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.398864 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.399482 11330 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:58.400409 11499 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.400933 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:58.402328 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.382844 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.369956 11252 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:58.388218 11692 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.402503 11689 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:58.405293 11689 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.405778 11689 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.405335 11692 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.407699 11692 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.409178 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.409807 11328 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
W20260709 12:11:58.409963 11222 leader_election.cc:343] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.383816 11577 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.411051 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:58.411573 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.412026 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:58.412511 11692 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:58.412846 11692 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.413200 11692 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.413909 11690 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.414544 11690 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.415163 11656 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:58.426354 11222 leader_election.cc:343] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.427963 11575 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.428509 11575 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.428789 11690 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.431465 11575 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.432947 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.433977 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.434769 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.439848 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.442052 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.442334 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.442983 11252 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
W20260709 12:11:58.444105 11222 leader_election.cc:343] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.448232 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.456650 11328 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:58.456735 11689 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.458003 11177 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.458004 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:58.465529 11693 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:58.465652 11256 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:58.466041 11693 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.466423 11693 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.466619 11375 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:58.467124 11375 ts_tablet_manager.cc:1403] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.165s	user 0.061s	sys 0.075s
I20260709 12:11:58.450120 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.469929 11252 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:58.472004 11375 raft_consensus.cc:359] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.472710 11375 raft_consensus.cc:385] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.472977 11375 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.473030 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:58.474813 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:58.478686 11374 raft_consensus.cc:359] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.479660 11374 raft_consensus.cc:385] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.480017 11374 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
W20260709 12:11:58.480309 11296 leader_election.cc:343] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.481894 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:11:58.485860 11296 leader_election.cc:343] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.467010 11575 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:11:58.487907 11374 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.478046 11687 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:58.490983 11375 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.478046 11582 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:11:58.491004 11687 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.486968 11383 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.502367 11375 ts_tablet_manager.cc:1434] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.002s
I20260709 12:11:58.504535 11376 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:58.506711 11376 ts_tablet_manager.cc:1403] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.154s	user 0.031s	sys 0.033s
I20260709 12:11:58.519425 11656 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:58.495949 11687 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.521770 11375 tablet_bootstrap.cc:492] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:58.528555 11375 tablet_bootstrap.cc:654] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.528962 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:11:58.529830 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53532.7us
I20260709 12:11:58.496790 11689 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.496315 11577 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:11:58.532076 11567 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.539501 11567 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.542032 11374 ts_tablet_manager.cc:1434] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.153s	user 0.012s	sys 0.006s
I20260709 12:11:58.543474 11374 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:58.542520 11654 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:11:58.556856 11717 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:11:58.560756 11376 raft_consensus.cc:359] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.561491 11376 raft_consensus.cc:385] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.562291 11376 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.565442 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:11:58.570595 11222 leader_election.cc:343] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.571415 11376 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.577481 11376 ts_tablet_manager.cc:1434] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.070s	user 0.005s	sys 0.001s
I20260709 12:11:58.583603 11719 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.584139 11719 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.611006 11383 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:58.589064 11692 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.612994 11376 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.595008 11567 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.614902 11654 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.610361 11693 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.619148 11654 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.621196 11719 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.613528 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.618908 11724 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:11:58.623741 11222 leader_election.cc:343] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.608052 11687 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.621209 11654 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.650424 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.661788 11375 tablet_bootstrap.cc:492] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:58.663179 11375 ts_tablet_manager.cc:1403] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.152s	user 0.011s	sys 0.012s
I20260709 12:11:58.665941 11374 tablet_bootstrap.cc:654] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.663787 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.678203 11692 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.682004 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.683938 11250 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:58.692489 11687 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.721474 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.688380 11255 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:58.721316 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:58.747256 11325 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:58.697603 11426 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.752214 11376 tablet_bootstrap.cc:654] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.688668 11331 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:58.731340 11329 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.755663 11296 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:58.758137 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.757011 11680 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:11:58.756873 11577 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:11:58.761068 11724 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:58.753903 11426 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.766448 11426 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.768203 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:58.769170 11689 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:58.761456 11724 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.769791 11724 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.770385 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.771018 11182 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:58.774866 11375 raft_consensus.cc:359] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.775456 11375 raft_consensus.cc:385] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.775678 11375 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:58.769531 11689 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.778110 11680 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:11:58.785737 11330 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:58.774765 11254 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:11:58.780544 11689 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:58.790573 11727 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:11:58.782644 11693 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.792981 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:58.799516 11723 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:58.799929 11723 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.800242 11723 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.810372 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:58.811273 11255 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:58.812711 11582 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.813230 11582 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.814467 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:58.815258 11325 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:58.815403 11582 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.817242 11375 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.818866 11375 ts_tablet_manager.cc:1434] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.155s	user 0.008s	sys 0.000s
I20260709 12:11:58.818995 11148 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:58.819768 11375 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:58.824738 11725 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.828979 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:58.841053 11426 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:58.841471 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:58.846910 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:58.861825 11722 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:58.862393 11722 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:11:58.872113 11731 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:11:58.873315 11723 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.878005 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:58.881916 11723 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.882005 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.882905 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:58.888305 11728 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.884373 11724 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.890712 11330 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:58.889905 11728 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.891803 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:58.892371 11251 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.893417 11728 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.893751 11376 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:58.894248 11376 ts_tablet_manager.cc:1403] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.282s	user 0.016s	sys 0.022s
I20260709 12:11:58.904376 11374 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:58.905108 11374 ts_tablet_manager.cc:1403] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.362s	user 0.016s	sys 0.025s
I20260709 12:11:58.871294 11689 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.907828 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.909461 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:58.910236 11330 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:58.911998 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:58.914084 11690 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:58.914772 11690 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:58.921568 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:58.902522 11722 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.924889 11720 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:58.925747 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:58.927001 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:58.926028 11250 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:11:58.927692 11256 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:58.926435 11690 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.924889 11376 raft_consensus.cc:359] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.929101 11376 raft_consensus.cc:385] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:58.929551 11376 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
W20260709 12:11:58.934006 11222 leader_election.cc:343] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.930701 11376 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:58.951807 11296 leader_election.cc:343] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:58.952911 11376 ts_tablet_manager.cc:1434] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.002s
I20260709 12:11:58.954190 11376 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:58.955276 11689 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:58.945418 11251 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:11:58.956696 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:58.960501 11723 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:58.966145 11723 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:11:58.967051 11723 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:58.972390 11724 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:58.983919 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:11:58.984588 11330 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.985934 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:58.987643 11078 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:58.990171 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:58.984084 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:58.991441 11177 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:58.979404 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:58.994819 11330 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:58.997716 11375 tablet_bootstrap.cc:654] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.000660 11577 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:59.001065 11577 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.001399 11577 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.002774 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:59.003609 11376 tablet_bootstrap.cc:654] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:58.992473 11374 raft_consensus.cc:359] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.005430 11374 raft_consensus.cc:385] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.005888 11374 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.024729 11577 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.008044 11374 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.032032 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:59.032876 11255 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:11:59.026484 11383 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.033468 11577 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:59.033974 11383 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.036948 11374 ts_tablet_manager.cc:1434] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.132s	user 0.008s	sys 0.000s
I20260709 12:11:59.038267 11374 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:59.054430 11383 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:59.065868 11296 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:59.078768 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:59.079442 11255 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:59.080717 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:59.081869 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:59.086320 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:59.086925 11253 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.089674 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:59.071609 11499 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.064450 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:11:59.124457 11296 leader_election.cc:343] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:59.128407 11296 leader_election.cc:343] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:59.151096 11739 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:11:59.126119 11582 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:59.153916 11722 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:11:59.149432 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95375.6us
I20260709 12:11:59.159739 11582 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.168345 11722 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:11:59.179415 11374 tablet_bootstrap.cc:654] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.179733 11253 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:59.168613 11582 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:59.173954 11740 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:11:59.169385 11725 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.191074 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:59.197095 11077 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.179490 11722 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:59.202220 11741 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.111s	user 0.001s	sys 0.003s
I20260709 12:11:59.194185 11577 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:59.212795 11577 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:59.213675 11577 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.191689 11177 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:11:59.221706 11426 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.235045 11375 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:59.235575 11375 ts_tablet_manager.cc:1403] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.416s	user 0.022s	sys 0.019s
I20260709 12:11:59.258674 11582 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.287634 11376 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:59.287634 11374 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:59.288252 11376 ts_tablet_manager.cc:1403] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.334s	user 0.014s	sys 0.015s
I20260709 12:11:59.288321 11374 ts_tablet_manager.cc:1403] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.250s	user 0.016s	sys 0.021s
I20260709 12:11:59.291771 11376 raft_consensus.cc:359] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.292506 11376 raft_consensus.cc:385] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.292855 11376 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.293676 11376 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.296424 11376 ts_tablet_manager.cc:1434] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:11:59.297567 11376 tablet_bootstrap.cc:492] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:59.286031 11176 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:59.299778 11375 raft_consensus.cc:359] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.300534 11375 raft_consensus.cc:385] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.301182 11374 raft_consensus.cc:359] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.301846 11374 raft_consensus.cc:385] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.301751 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:59.300809 11375 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.302179 11374 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.304579 11374 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.303449 11375 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.306439 11753 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:59.306973 11753 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:59.307875 11374 ts_tablet_manager.cc:1434] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:11:59.308743 11374 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:59.307821 11753 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.310039 11375 ts_tablet_manager.cc:1434] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 12:11:59.311146 11375 tablet_bootstrap.cc:492] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
W20260709 12:11:59.313859 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:59.315553 11582 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:59.320590 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:59.320883 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:11:59.321231 11256 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:59.323064 11296 leader_election.cc:343] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:59.339036 11222 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:59.341576 11256 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:59.345692 11375 tablet_bootstrap.cc:654] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.350347 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:59.351105 11582 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:59.351581 11582 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:59.352442 11582 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.355337 11376 tablet_bootstrap.cc:654] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.368963 11374 tablet_bootstrap.cc:654] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.373579 11722 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:59.386339 11077 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:59.387600 11375 tablet_bootstrap.cc:492] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:59.388087 11375 ts_tablet_manager.cc:1403] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.001s
I20260709 12:11:59.389812 11722 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:59.391649 11374 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:59.392151 11374 ts_tablet_manager.cc:1403] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.001s
I20260709 12:11:59.394089 11376 tablet_bootstrap.cc:492] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:59.394523 11376 ts_tablet_manager.cc:1403] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.097s	user 0.008s	sys 0.018s
I20260709 12:11:59.396133 11374 raft_consensus.cc:359] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.396847 11374 raft_consensus.cc:385] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.397156 11374 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.397349 11375 raft_consensus.cc:359] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.401216 11375 raft_consensus.cc:385] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.401505 11375 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.402153 11375 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.397861 11374 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.403927 11375 ts_tablet_manager.cc:1434] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:11:59.400553 11376 raft_consensus.cc:359] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.404372 11376 raft_consensus.cc:385] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.404690 11376 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.404852 11375 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:59.405470 11376 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.407212 11376 ts_tablet_manager.cc:1434] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:11:59.408106 11376 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:59.410662 11375 tablet_bootstrap.cc:654] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.411206 11374 ts_tablet_manager.cc:1434] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 12:11:59.411736 11577 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.412178 11374 tablet_bootstrap.cc:492] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:59.412259 11577 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.416103 11577 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:59.417815 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:59.418286 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:59.418548 11253 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:11:59.419914 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:11:59.420858 11296 leader_election.cc:343] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.421664 11577 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:59.422084 11577 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.422439 11577 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.423146 11376 tablet_bootstrap.cc:654] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.429469 11577 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.435411 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:11:59.436800 11296 leader_election.cc:343] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.389160 11077 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:59.425768 11374 tablet_bootstrap.cc:654] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.438752 11375 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:59.439241 11375 ts_tablet_manager.cc:1403] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.001s
I20260709 12:11:59.441650 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:11:59.442248 11253 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.448103 11375 raft_consensus.cc:359] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.448833 11253 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:11:59.450475 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:11:59.451365 11582 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:59.451846 11582 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:11:59.452637 11582 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.460081 11076 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.448863 11375 raft_consensus.cc:385] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.465065 11375 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.464588 11076 catalog_manager.cc:5697] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.470363 11176 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.460151 11577 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:59.465875 11375 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.481462 11375 ts_tablet_manager.cc:1434] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.001s
I20260709 12:11:59.482484 11375 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:59.488353 11375 tablet_bootstrap.cc:654] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.489113 11574 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:59.511620 11718 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.512383 11718 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.516784 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:11:59.518850 11222 leader_election.cc:343] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.523538 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:59.523896 11718 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:59.526798 11376 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:59.527240 11376 ts_tablet_manager.cc:1403] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.119s	user 0.007s	sys 0.039s
I20260709 12:11:59.527392 11574 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.527873 11574 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.528764 11718 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:59.524782 11374 tablet_bootstrap.cc:492] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
W20260709 12:11:59.525286 11223 leader_election.cc:343] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.529902 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:59.530942 11692 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.531737 11374 ts_tablet_manager.cc:1403] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.120s	user 0.018s	sys 0.026s
I20260709 12:11:59.532371 11574 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:59.533737 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:59.533782 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:59.535270 11376 raft_consensus.cc:359] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.536091 11376 raft_consensus.cc:385] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.536438 11376 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.535259 11078 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:59.536206 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:59.538116 11374 raft_consensus.cc:359] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.537755 11078 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:59.538765 11374 raft_consensus.cc:385] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.539059 11574 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:11:59.539354 11376 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.541584 11376 ts_tablet_manager.cc:1434] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 12:11:59.539086 11374 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.543896 11374 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.546470 11374 ts_tablet_manager.cc:1434] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:11:59.547358 11374 tablet_bootstrap.cc:492] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:59.552810 11374 tablet_bootstrap.cc:654] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.555353 11376 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:11:59.561815 11728 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:59.562690 11376 tablet_bootstrap.cc:654] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.569401 11728 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.570155 11728 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.574584 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:59.599545 11728 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:59.613493 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:59.614665 11256 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:11:59.619546 11375 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:59.620128 11375 ts_tablet_manager.cc:1403] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.138s	user 0.057s	sys 0.061s
I20260709 12:11:59.623072 11375 raft_consensus.cc:359] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.623826 11375 raft_consensus.cc:385] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.624094 11375 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.624742 11375 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.626201 11680 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:59.626850 11375 ts_tablet_manager.cc:1434] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:11:59.627898 11375 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:59.634893 11375 tablet_bootstrap.cc:654] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.643421 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:59.648109 11375 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:59.648605 11375 ts_tablet_manager.cc:1403] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.004s
I20260709 12:11:59.649979 11722 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:59.660072 11181 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.667205 11656 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.670219 11680 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:11:59.672755 11375 raft_consensus.cc:359] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.673313 11375 raft_consensus.cc:385] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.673565 11375 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.674167 11375 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.676211 11375 ts_tablet_manager.cc:1434] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.000s
I20260709 12:11:59.676887 11375 tablet_bootstrap.cc:492] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:59.688664 11574 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.689368 11574 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.693087 11574 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:59.693253 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:59.697242 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:59.703126 11375 tablet_bootstrap.cc:654] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.723582 11577 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.724321 11577 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.726482 11577 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:59.750553 11722 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:59.760794 11720 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.761317 11720 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.763358 11720 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:59.765983 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:59.766916 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:11:59.768141 11222 leader_election.cc:343] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:59.768739 11223 leader_election.cc:343] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.769236 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:11:59.770612 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:59.771999 11722 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.772604 11177 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.774365 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:59.797835 11376 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:59.798560 11376 ts_tablet_manager.cc:1403] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.256s	user 0.023s	sys 0.046s
I20260709 12:11:59.802233 11376 raft_consensus.cc:359] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.803181 11376 raft_consensus.cc:385] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.803519 11376 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.804390 11376 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.811355 11374 tablet_bootstrap.cc:492] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:59.811903 11374 ts_tablet_manager.cc:1403] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.265s	user 0.018s	sys 0.052s
I20260709 12:11:59.814769 11374 raft_consensus.cc:359] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.815522 11374 raft_consensus.cc:385] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.815855 11374 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.816551 11374 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.818615 11374 ts_tablet_manager.cc:1434] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:11:59.818734 11376 ts_tablet_manager.cc:1434] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:11:59.819562 11374 tablet_bootstrap.cc:492] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:59.822352 11680 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.832484 11680 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:11:59.827801 11296 leader_election.cc:343] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.834218 11680 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:11:59.834470 11383 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:59.836024 11383 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:11:59.836599 11296 leader_election.cc:343] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.837013 11296 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:59.766067 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:59.837811 11680 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.838030 11574 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.842665 11176 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.842959 11376 tablet_bootstrap.cc:492] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
W20260709 12:11:59.847694 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 73030.2us
I20260709 12:11:59.848526 11375 tablet_bootstrap.cc:492] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:59.848982 11375 ts_tablet_manager.cc:1403] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.172s	user 0.054s	sys 0.055s
I20260709 12:11:59.851456 11375 raft_consensus.cc:359] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.852103 11375 raft_consensus.cc:385] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.852373 11375 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.853070 11375 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.853540 11374 tablet_bootstrap.cc:654] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.855258 11375 ts_tablet_manager.cc:1434] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:11:59.858868 11375 tablet_bootstrap.cc:492] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:11:59.860287 11376 tablet_bootstrap.cc:654] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.866196 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:59.876677 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:11:59.883085 11375 tablet_bootstrap.cc:654] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:59.884850 11383 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:11:59.891503 11296 leader_election.cc:343] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.889087 11383 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:11:59.898924 11374 tablet_bootstrap.cc:492] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:11:59.899417 11374 ts_tablet_manager.cc:1403] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.080s	user 0.016s	sys 0.002s
W20260709 12:11:59.901841 11296 leader_election.cc:343] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:59.912390 11374 raft_consensus.cc:359] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.916838 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:11:59.917214 11374 raft_consensus.cc:385] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:59.917608 11374 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:11:59.917760 11383 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:59.918473 11374 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.926149 11374 ts_tablet_manager.cc:1434] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.004s
I20260709 12:11:59.927079 11374 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:11:59.932128 11374 tablet_bootstrap.cc:654] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:59.948050 11780 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:11:59.950106 11375 tablet_bootstrap.cc:492] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:11:59.950567 11375 ts_tablet_manager.cc:1403] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.092s	user 0.010s	sys 0.011s
I20260709 12:11:59.954599 11180 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:59.955964 11376 tablet_bootstrap.cc:492] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:11:59.956585 11376 ts_tablet_manager.cc:1403] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.114s	user 0.015s	sys 0.006s
I20260709 12:11:59.977165 11720 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.977790 11720 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.982529 11383 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:59.987812 11720 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:11:59.989274 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:11:59.989974 11328 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:11:59.991115 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:11:59.992940 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:11:59.998943 11720 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.003690 11720 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:11:59.999568 11718 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:00.006996 11718 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.007513 11718 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.007789 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:00.008476 11179 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:00.011386 11720 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.011560 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:00.013139 11223 leader_election.cc:343] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.013681 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:00.014911 11692 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:12:00.014981 11222 leader_election.cc:343] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.015321 11692 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.015682 11692 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.022897 11376 raft_consensus.cc:359] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.054394 11376 raft_consensus.cc:385] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.054723 11376 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.022732 11692 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.052902 11718 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.071558 11376 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.072224 11375 raft_consensus.cc:359] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.073194 11375 raft_consensus.cc:385] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.073539 11375 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.074390 11375 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:00.083434 11784 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:12:00.096606 11718 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.099217 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:00.100525 11680 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:00.102373 11222 leader_election.cc:343] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.109588 11375 ts_tablet_manager.cc:1434] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.159s	user 0.007s	sys 0.002s
I20260709 12:12:00.112280 11375 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
W20260709 12:12:00.116793 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53077.6us
I20260709 12:12:00.134644 11376 ts_tablet_manager.cc:1434] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.178s	user 0.015s	sys 0.006s
I20260709 12:12:00.136001 11376 tablet_bootstrap.cc:492] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:12:00.145676 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:00.146432 11329 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.125422 11680 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.147959 11692 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.165885 11375 tablet_bootstrap.cc:654] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.170356 11795 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.170418 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:00.171211 11181 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.171042 11795 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.173426 11795 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:00.176436 11376 tablet_bootstrap.cc:654] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.179193 11374 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:12:00.179764 11374 ts_tablet_manager.cc:1403] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.253s	user 0.019s	sys 0.034s
I20260709 12:12:00.179899 11181 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:00.187100 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:00.199710 11374 raft_consensus.cc:359] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.200562 11374 raft_consensus.cc:385] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.200867 11374 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.153811 11680 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:00.206497 11793 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:12:00.235390 11680 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:00.211374 11374 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.206485 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:00.267254 11374 ts_tablet_manager.cc:1434] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.087s	user 0.009s	sys 0.001s
I20260709 12:12:00.274521 11374 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:12:00.260051 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:00.288820 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 133364us
W20260709 12:12:00.293035 11296 leader_election.cc:343] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.296011 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:00.300550 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:00.301200 11256 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:12:00.288483 11329 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:00.301280 11718 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:00.291109 11178 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:12:00.310659 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:00.311355 11256 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 0.
I20260709 12:12:00.318501 11376 tablet_bootstrap.cc:492] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:12:00.335953 11376 ts_tablet_manager.cc:1403] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.200s	user 0.012s	sys 0.043s
I20260709 12:12:00.319880 11375 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:12:00.339514 11375 ts_tablet_manager.cc:1403] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.227s	user 0.020s	sys 0.013s
I20260709 12:12:00.353266 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:00.354642 11383 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:00.355119 11383 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.355535 11383 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.356101 11376 raft_consensus.cc:359] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.356827 11376 raft_consensus.cc:385] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.357100 11376 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.357824 11692 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.357784 11376 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.358525 11692 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.359900 11376 ts_tablet_manager.cc:1434] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:12:00.360447 11692 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.321803 11718 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:00.364213 11376 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:12:00.364892 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:00.369791 11796 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:00.372820 11796 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.373196 11796 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.376605 11718 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.380297 11796 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.388793 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:00.413658 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:00.416846 11374 tablet_bootstrap.cc:654] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.416328 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:00.417868 11251 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.407219 11796 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:00.424607 11375 raft_consensus.cc:359] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.425336 11375 raft_consensus.cc:385] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.425663 11375 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
W20260709 12:12:00.426374 11296 leader_election.cc:343] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.426419 11375 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.428941 11375 ts_tablet_manager.cc:1434] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.089s	user 0.005s	sys 0.001s
I20260709 12:12:00.431174 11383 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.431761 11376 tablet_bootstrap.cc:654] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.433390 11383 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:00.433959 11181 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:00.435161 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:00.435926 11720 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:00.436259 11720 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.436582 11720 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.442770 11374 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:12:00.443292 11374 ts_tablet_manager.cc:1403] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.169s	user 0.013s	sys 0.000s
W20260709 12:12:00.445182 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 259729us
W20260709 12:12:00.446200 11223 leader_election.cc:343] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.448338 11376 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:12:00.448930 11376 ts_tablet_manager.cc:1403] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.006s
I20260709 12:12:00.449699 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:00.450464 11722 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:00.451120 11722 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:00.452828 11251 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:00.453927 11720 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.451980 11722 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.455704 11375 tablet_bootstrap.cc:492] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:12:00.457283 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:00.457901 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:00.458462 11180 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.458935 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:00.459497 11249 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.457923 11178 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.464111 11376 raft_consensus.cc:359] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.464800 11376 raft_consensus.cc:385] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.465075 11376 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.466845 11249 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:00.467974 11374 raft_consensus.cc:359] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.468657 11374 raft_consensus.cc:385] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.468941 11374 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.469647 11374 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.471557 11375 tablet_bootstrap.cc:654] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.465770 11376 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.472237 11720 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.474359 11376 ts_tablet_manager.cc:1434] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 12:12:00.475203 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:00.475337 11376 tablet_bootstrap.cc:492] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:12:00.476099 11795 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:00.479853 11795 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:00.480720 11795 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.477880 11374 ts_tablet_manager.cc:1434] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:12:00.485229 11374 tablet_bootstrap.cc:492] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:12:00.485296 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:12:00.533159 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:12:00.543268 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57847.7us
I20260709 12:12:00.553690 11795 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:00.488312 11376 tablet_bootstrap.cc:654] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.573688 11807 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.578982 11807 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.575409 11383 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.582171 11383 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.586186 11180 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:00.586202 11383 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:00.589108 11077 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:00.592170 11807 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.594384 11178 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:00.598342 11374 tablet_bootstrap.cc:654] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.599543 11375 tablet_bootstrap.cc:492] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:12:00.559470 11795 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:00.605762 11375 ts_tablet_manager.cc:1403] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.161s	user 0.015s	sys 0.002s
I20260709 12:12:00.610069 11795 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.599785 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:00.614480 11375 raft_consensus.cc:359] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.615571 11375 raft_consensus.cc:385] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.615942 11375 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.616782 11375 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.618722 11722 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:00.619369 11722 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:00.617240 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:00.620723 11722 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:00.631516 11222 leader_election.cc:343] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.635435 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:00.642171 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:00.643702 11175 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:00.648751 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:00.649673 11327 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:00.670014 11375 ts_tablet_manager.cc:1434] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.061s	user 0.002s	sys 0.008s
I20260709 12:12:00.673206 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:00.675737 11331 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.677058 11376 tablet_bootstrap.cc:492] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:12:00.683876 11376 ts_tablet_manager.cc:1403] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.209s	user 0.015s	sys 0.004s
I20260709 12:12:00.686102 11375 tablet_bootstrap.cc:492] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:12:00.686378 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:00.689360 11330 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:00.697877 11376 raft_consensus.cc:359] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.701021 11376 raft_consensus.cc:385] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.701408 11376 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.702486 11376 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.719031 11375 tablet_bootstrap.cc:654] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.720769 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:00.722749 11796 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.737421 11374 tablet_bootstrap.cc:492] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:12:00.739683 11374 ts_tablet_manager.cc:1403] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.255s	user 0.015s	sys 0.013s
I20260709 12:12:00.739971 11796 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:00.745867 11331 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:00.752462 11374 raft_consensus.cc:359] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.752790 11076 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:00.754473 11374 raft_consensus.cc:385] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.754849 11374 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.755580 11374 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.756520 11223 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:00.756639 11076 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:00.768880 11374 ts_tablet_manager.cc:1434] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.005s
I20260709 12:12:00.772085 11376 ts_tablet_manager.cc:1434] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.003s
I20260709 12:12:00.771751 11374 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:12:00.777056 11807 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:00.777782 11807 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.778271 11807 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.773483 11376 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:12:00.815040 11728 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.815826 11728 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.822697 11728 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.825372 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:00.831066 11720 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.846271 11720 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.852614 11720 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.887048 11807 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.889102 11376 tablet_bootstrap.cc:654] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:00.901085 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:00.902125 11331 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:00.904594 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:00.904881 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:00.905577 11327 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:00.905553 11728 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.926373 11375 tablet_bootstrap.cc:492] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:12:00.926837 11375 ts_tablet_manager.cc:1403] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.241s	user 0.018s	sys 0.014s
I20260709 12:12:00.935039 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:00.938308 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:00.943533 11222 leader_election.cc:343] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:00.936465 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:00.954401 11788 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:00.954921 11788 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.955322 11788 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.954614 11375 raft_consensus.cc:359] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.956804 11375 raft_consensus.cc:385] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:00.957132 11375 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:12:00.957974 11375 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:00.960271 11375 ts_tablet_manager.cc:1434] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:12:00.947503 11807 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:00.956365 11728 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:00.977597 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:00.979902 11328 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:00.981035 11176 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:01.009923 11374 tablet_bootstrap.cc:654] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:01.013929 11799 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:12:01.017138 11375 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:12:01.016340 11807 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:12:01.033914 11809 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:12:01.035426 11796 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:12:01.037894 11720 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:01.044443 11720 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:01.045696 11807 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:01.089859 11375 tablet_bootstrap.cc:654] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:01.077081 11328 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:01.084646 11788 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.107733 11799 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:12:01.131506 11223 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:01.132862 11720 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:01.133467 11720 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:01.134043 11376 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:12:01.135126 11376 ts_tablet_manager.cc:1403] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.362s	user 0.031s	sys 0.010s
W20260709 12:12:01.298925 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 122495us
W20260709 12:12:01.278417 11829 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:12:01.307106 11374 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:12:01.308071 11374 ts_tablet_manager.cc:1403] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.537s	user 0.028s	sys 0.046s
I20260709 12:12:01.310976 11375 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:12:01.311519 11375 ts_tablet_manager.cc:1403] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.320s	user 0.034s	sys 0.040s
I20260709 12:12:01.193153 11788 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:01.192421 11720 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.324059 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:01.324805 11329 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:01.325232 11376 raft_consensus.cc:359] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.326025 11376 raft_consensus.cc:385] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.326345 11376 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
I20260709 12:12:01.327066 11376 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.329553 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:01.330233 11181 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:01.331173 11814 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.001s	sys 0.011s
I20260709 12:12:01.351613 11376 ts_tablet_manager.cc:1434] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.216s	user 0.008s	sys 0.000s
I20260709 12:12:01.339990 11809 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:12:01.361850 11375 raft_consensus.cc:359] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.362658 11375 raft_consensus.cc:385] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.362959 11375 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:12:01.363714 11375 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.368419 11376 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Bootstrap starting.
I20260709 12:12:01.365470 11375 ts_tablet_manager.cc:1434] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.054s	user 0.002s	sys 0.002s
I20260709 12:12:01.373759 11181 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:01.376350 11374 raft_consensus.cc:359] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.377547 11374 raft_consensus.cc:385] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.378031 11374 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:12:01.378293 11375 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:12:01.378844 11374 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.447820 11795 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.448405 11795 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.450511 11795 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:01.451411 11376 tablet_bootstrap.cc:654] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:01.475451 11374 ts_tablet_manager.cc:1434] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.167s	user 0.010s	sys 0.011s
I20260709 12:12:01.486340 11374 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Bootstrap starting.
I20260709 12:12:01.500856 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:01.504904 11375 tablet_bootstrap.cc:654] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:01.538169 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46802.8us
W20260709 12:12:01.538511 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 62091.8us
I20260709 12:12:01.482095 11329 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
W20260709 12:12:01.551371 11296 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:01.552533 11296 leader_election.cc:343] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:01.554220 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:01.561286 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:01.575183 11722 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:01.596714 11722 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:01.597704 11722 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.607481 11376 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: No bootstrap required, opened a new log
I20260709 12:12:01.608021 11376 ts_tablet_manager.cc:1403] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent bootstrapping tablet: real 0.240s	user 0.015s	sys 0.011s
I20260709 12:12:01.634835 11079 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:12:01.644016 11375 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:12:01.644665 11375 ts_tablet_manager.cc:1403] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.267s	user 0.027s	sys 0.002s
W20260709 12:12:01.651994 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78829.3us
I20260709 12:12:01.653991 11375 raft_consensus.cc:359] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.654772 11375 raft_consensus.cc:385] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.655064 11375 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:12:01.661368 11375 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.664644 11375 ts_tablet_manager.cc:1434] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:12:01.665719 11375 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:12:01.667306 11376 raft_consensus.cc:359] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.668205 11376 raft_consensus.cc:385] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.668718 11376 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Initialized, Role: FOLLOWER
W20260709 12:12:01.677724 11296 leader_election.cc:343] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:01.678369 11296 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:01.682318 11374 tablet_bootstrap.cc:654] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:01.687741 11376 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.688426 11797 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:01.692747 11375 tablet_bootstrap.cc:654] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:01.717773 11374 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: No bootstrap required, opened a new log
I20260709 12:12:01.718317 11374 ts_tablet_manager.cc:1403] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent bootstrapping tablet: real 0.232s	user 0.011s	sys 0.006s
I20260709 12:12:01.720485 11376 ts_tablet_manager.cc:1434] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.001s
I20260709 12:12:01.739244 11176 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:01.735479 11374 raft_consensus.cc:359] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.740491 11374 raft_consensus.cc:385] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.740785 11374 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Initialized, Role: FOLLOWER
I20260709 12:12:01.741580 11374 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.744432 11374 ts_tablet_manager.cc:1434] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:12:01.766584 11077 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:01.769460 11077 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:01.770548 11127 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:01.793562 11375 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:12:01.794196 11375 ts_tablet_manager.cc:1403] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.129s	user 0.012s	sys 0.067s
I20260709 12:12:01.796840 11375 raft_consensus.cc:359] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.804519 11718 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.805305 11718 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.797492 11375 raft_consensus.cc:385] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.807392 11375 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:12:01.807658 11718 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:01.808358 11375 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.809492 11656 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.809984 11656 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.812114 11656 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:01.813568 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:01.814002 11375 ts_tablet_manager.cc:1434] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:12:01.814517 11326 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 1: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:01.815035 11375 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Bootstrap starting.
I20260709 12:12:01.817349 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:01.818063 11179 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:01.827622 11375 tablet_bootstrap.cc:654] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:01.827986 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:01.831499 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:01.832492 11852 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:01.833014 11852 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:01.833277 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:01.833461 11852 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:01.834378 11856 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:01.839105 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:01.839856 11326 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 0.
I20260709 12:12:01.880162 11856 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:01.880548 11375 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: No bootstrap required, opened a new log
I20260709 12:12:01.881007 11375 ts_tablet_manager.cc:1403] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent bootstrapping tablet: real 0.066s	user 0.021s	sys 0.032s
I20260709 12:12:01.895466 11852 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.896615 11375 raft_consensus.cc:359] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.897372 11375 raft_consensus.cc:385] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:01.897665 11375 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Initialized, Role: FOLLOWER
I20260709 12:12:01.897747 11852 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:01.898325 11375 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.900272 11375 ts_tablet_manager.cc:1434] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 12:12:01.900323 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:01.900898 11181 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:01.907940 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:01.908627 11328 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:01.919234 11181 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:01.929811 11328 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:01.931339 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:01.933846 11850 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:01.934360 11850 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:01.935191 11850 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:01.952600 11852 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:01.954885 11850 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:01.969442 11079 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:01.995839 11852 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:02.017140 11847 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:02.020287 11076 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 0 to 1, leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:02.035331 11574 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.035815 11574 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:02.037716 11574 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:02.039656 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:02.040297 11328 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:12:02.042048 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:02.042690 11253 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 0.
I20260709 12:12:02.043187 11148 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:02.044566 11574 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:02.044910 11574 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.045174 11574 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:02.052397 11574 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:02.054216 11574 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:02.055038 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:02.055191 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:02.055675 11253 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:02.055727 11328 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:02.062798 11328 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:02.063973 11253 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:02.065253 11148 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:02.066385 11574 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:02.066818 11574 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:02.067598 11574 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:02.079964 11076 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:02.363833 11847 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:02.375268 11852 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:02.544036 11859 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:02.558790 11859 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:03.102902 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:03.127941 11889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:03.127960 11888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:03.129282 11891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:03.134639 10011 server_base.cc:1061] running on GCE node
I20260709 12:12:03.136595 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:03.136929 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:03.137169 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599123137151 us; error 0 us; skew 500 ppm
I20260709 12:12:03.137866 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:03.141737 10011 webserver.cc:533] Webserver started at http://127.9.198.196:43065/ using document root <none> and password file <none>
I20260709 12:12:03.142284 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:03.142510 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:03.142848 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:03.144044 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-3-root/instance:
uuid: "786851d4227e4dd89b75ccdea0b1f03b"
format_stamp: "Formatted at 2026-07-09 12:12:03 on dist-test-slave-glnm"
I20260709 12:12:03.148790 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:12:03.152594 11897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:03.153484 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:12:03.153951 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-3-root
uuid: "786851d4227e4dd89b75ccdea0b1f03b"
format_stamp: "Formatted at 2026-07-09 12:12:03 on dist-test-slave-glnm"
I20260709 12:12:03.154330 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:03.191843 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:03.193778 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:03.195840 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:03.199440 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:03.199769 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:03.200085 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:03.200398 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:12:03.331667 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.196:39119
I20260709 12:12:03.331772 11959 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.196:39119 every 8 connection(s)
I20260709 12:12:03.337333 10011 auto_leader_rebalancer-test.cc:439] add a tserver: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:03.352547 11960 heartbeater.cc:344] Connected to a master server at 127.9.198.254:36219
I20260709 12:12:03.352984 11960 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:03.353816 11960 heartbeater.cc:507] Master 127.9.198.254:36219 requested a full tablet report, sending...
I20260709 12:12:03.355631 11077 ts_manager.cc:194] Registered new tserver with Master: 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:03.356952 11077 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56652
I20260709 12:12:03.903360 11324 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:03.916626 11254 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:03.916882 11179 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:03.918174 11892 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:03.919178 11383 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:03.937367 11296 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c40179116e14aa2aff8e95298cbcd4f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:03.944259 11383 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:03.946549 11179 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:03.949177 11254 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:03.967203 11077 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:03.986701 11254 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.001107 11324 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:04.002774 11718 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:04.006353 11179 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:04.008253 11718 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:04.017022 11222 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f736e38db6b45bfa87e35af564bffc1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:04.020152 11964 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.027261 11176 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.076056 11324 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.076942 11326 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.096132 11076 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:04.104736 11254 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:04.106542 11296 consensus_peers.cc:597] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88ddcb89ab6d47cd97e8c19734f0e365. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:04.107296 11797 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:04.112555 11179 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:04.115784 11383 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:04.121708 11972 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.167359 11326 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.176992 11254 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.178203 11181 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.201339 11079 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:12:04.209200 11296 consensus_peers.cc:597] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9dab54d96124869a4a52f9cc08e288a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:04.210536 11181 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:04.212622 11254 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:04.217417 11383 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:04.218502 11892 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:04.239303 11383 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.241750 11254 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.242059 11181 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.260255 11077 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:04.359234 11960 heartbeater.cc:499] Master 127.9.198.254:36219 was elected leader, sending a full tablet report...
I20260709 12:12:04.401981 11990 ts_tablet_manager.cc:933] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:04.403831 11990 tablet_copy_client.cc:323] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:04.417114 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 7f736e38db6b45bfa87e35af564bffc1 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:04.417730 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 7f736e38db6b45bfa87e35af564bffc1 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-7f736e38db6b45bfa87e35af564bffc1
I20260709 12:12:04.423205 11267 tablet_copy_source_session.cc:215] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:04.426519 11990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f736e38db6b45bfa87e35af564bffc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:04.439240 11990 tablet_copy_client.cc:806] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:04.439878 11990 tablet_copy_client.cc:670] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:04.444329 11990 tablet_copy_client.cc:538] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:04.453660 11990 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:04.478314 11990 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:04.479300 11990 tablet_bootstrap.cc:492] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:04.479959 11990 ts_tablet_manager.cc:1403] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.011s
I20260709 12:12:04.482920 11990 raft_consensus.cc:359] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.483764 11990 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:04.484404 11990 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.489874 11995 ts_tablet_manager.cc:933] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:04.499096 11990 ts_tablet_manager.cc:1434] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.005s
I20260709 12:12:04.508170 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-7f736e38db6b45bfa87e35af564bffc1 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:04.508719 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-7f736e38db6b45bfa87e35af564bffc1 on tablet 7f736e38db6b45bfa87e35af564bffc1 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:04.517395 11995 tablet_copy_client.cc:323] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:04.532447 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 4c40179116e14aa2aff8e95298cbcd4f from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:04.533084 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 4c40179116e14aa2aff8e95298cbcd4f from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-4c40179116e14aa2aff8e95298cbcd4f
I20260709 12:12:04.540762 11341 tablet_copy_source_session.cc:215] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:04.547840 11995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c40179116e14aa2aff8e95298cbcd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:04.562319 11995 tablet_copy_client.cc:806] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:04.562887 11995 tablet_copy_client.cc:670] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:04.566869 11995 tablet_copy_client.cc:538] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:04.575120 11995 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:04.648341 11995 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:04.649215 11995 tablet_bootstrap.cc:492] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:04.649945 11995 ts_tablet_manager.cc:1403] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.075s	user 0.033s	sys 0.033s
I20260709 12:12:04.652561 11995 raft_consensus.cc:359] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.653270 11995 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:04.653980 11995 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.656801 11995 ts_tablet_manager.cc:1434] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 12:12:04.658591 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-4c40179116e14aa2aff8e95298cbcd4f received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:04.659108 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-4c40179116e14aa2aff8e95298cbcd4f on tablet 4c40179116e14aa2aff8e95298cbcd4f with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:04.696866 11995 ts_tablet_manager.cc:933] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:04.698621 11995 tablet_copy_client.cc:323] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:04.700179 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 88ddcb89ab6d47cd97e8c19734f0e365 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:04.700641 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 88ddcb89ab6d47cd97e8c19734f0e365 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-88ddcb89ab6d47cd97e8c19734f0e365
I20260709 12:12:04.706162 11341 tablet_copy_source_session.cc:215] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:04.709303 11995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ddcb89ab6d47cd97e8c19734f0e365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:04.714792 11990 ts_tablet_manager.cc:933] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:04.716492 11990 tablet_copy_client.cc:323] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:04.717957 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet b9dab54d96124869a4a52f9cc08e288a from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:04.718406 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet b9dab54d96124869a4a52f9cc08e288a from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-b9dab54d96124869a4a52f9cc08e288a
I20260709 12:12:04.724691 11341 tablet_copy_source_session.cc:215] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:04.725502 11995 tablet_copy_client.cc:806] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:04.726902 11995 tablet_copy_client.cc:670] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:04.729548 11990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9dab54d96124869a4a52f9cc08e288a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:04.730717 11995 tablet_copy_client.cc:538] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:04.739977 11995 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:04.748797 11990 tablet_copy_client.cc:806] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:04.749436 11990 tablet_copy_client.cc:670] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:04.754494 11990 tablet_copy_client.cc:538] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:04.768476 11995 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:04.769745 11995 tablet_bootstrap.cc:492] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:04.770423 11995 ts_tablet_manager.cc:1403] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.007s
I20260709 12:12:04.772424 11990 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:04.773607 11995 raft_consensus.cc:359] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.774533 11995 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:04.775192 11995 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.787942 11995 ts_tablet_manager.cc:1434] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 12:12:04.790614 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-88ddcb89ab6d47cd97e8c19734f0e365 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:04.791052 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-88ddcb89ab6d47cd97e8c19734f0e365 on tablet 88ddcb89ab6d47cd97e8c19734f0e365 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:04.805405 11990 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:04.806506 11990 tablet_bootstrap.cc:492] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:04.807294 11990 ts_tablet_manager.cc:1403] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.004s
I20260709 12:12:04.810490 11990 raft_consensus.cc:359] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.811249 11990 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:04.811864 11990 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:04.814481 11990 ts_tablet_manager.cc:1434] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:04.816205 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-b9dab54d96124869a4a52f9cc08e288a received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:04.816514 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-b9dab54d96124869a4a52f9cc08e288a on tablet b9dab54d96124869a4a52f9cc08e288a with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:04.867249 11935 raft_consensus.cc:1217] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:04.924654 11935 raft_consensus.cc:1217] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:05.156522 11935 raft_consensus.cc:1217] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:05.308111 11935 raft_consensus.cc:1217] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:05.391789 11718 raft_consensus.cc:1064] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:05.394323 11718 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:05.401219 11935 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.401196 11326 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.402396 11176 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.404412 11850 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:05.405344 11964 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:05.407987 12005 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:05.438759 11176 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.436632 11964 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.443428 11326 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.485194 11076 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.487651 11383 raft_consensus.cc:1064] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:05.490057 11383 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:05.505920 11935 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.514638 11254 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.514746 11179 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.513300 11934 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.519709 11972 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:05.522617 11892 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:05.524777 11892 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:05.567740 11383 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.570453 11179 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.572157 11935 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.604537 11254 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.614626 11253 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:05.619453 11077 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.631685 11176 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:05.633940 11964 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:05.643323 11324 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:05.644976 11935 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:05.651079 12005 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.654541 11176 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:05.654994 11181 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.656818 11892 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:05.658267 12003 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:05.662596 11935 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:05.664171 11892 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.666594 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7f736e38db6b45bfa87e35af564bffc1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:05.676036 11179 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.682664 11077 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:05.685113 11972 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:12:05.677964 11935 raft_consensus.cc:2955] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.694188 11383 raft_consensus.cc:1064] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:05.698048 11934 raft_consensus.cc:2955] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.700508 11383 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:05.741583 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4c40179116e14aa2aff8e95298cbcd4f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:05.742036 11078 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:05.771807 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f736e38db6b45bfa87e35af564bffc1 with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:05.780767 12024 tablet_replica.cc:333] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:05.781984 12024 raft_consensus.cc:2243] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:05.781998 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c40179116e14aa2aff8e95298cbcd4f with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:05.782732 12024 raft_consensus.cc:2272] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:05.799571 12024 ts_tablet_manager.cc:1916] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:05.813746 11253 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.814113 11934 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.820581 11181 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.830734 11972 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:12:05.839444 12025 tablet_replica.cc:333] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:05.840523 12025 raft_consensus.cc:2243] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:05.841445 12025 raft_consensus.cc:2272] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:05.844118 12011 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:05.850538 12011 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:05.851125 12025 ts_tablet_manager.cc:1916] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:05.873553 12025 ts_tablet_manager.cc:1929] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:05.874085 12025 log.cc:1199] T 4c40179116e14aa2aff8e95298cbcd4f P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/4c40179116e14aa2aff8e95298cbcd4f
I20260709 12:12:05.875855 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet 4c40179116e14aa2aff8e95298cbcd4f (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:05.876863 12024 ts_tablet_manager.cc:1929] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:05.877316 12024 log.cc:1199] T 7f736e38db6b45bfa87e35af564bffc1 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/7f736e38db6b45bfa87e35af564bffc1
I20260709 12:12:05.888546 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 7f736e38db6b45bfa87e35af564bffc1 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:05.903687 12023 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.915405 11181 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.920748 11934 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.931739 11077 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:05.935932 12018 raft_consensus.cc:1064] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:05.940835 12018 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:05.934314 11253 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:05.954864 11249 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.956849 12018 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:05.968256 11181 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.970408 11934 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:05.971814 12023 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:05.974396 12023 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:05.979338 11326 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:06.001231 11934 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:06.001360 11251 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:06.003129 12011 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:06.003432 12018 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.007705 11176 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.010861 12023 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:06.016263 12023 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.021790 11935 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.029070 11079 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.018905 11934 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.049969 11251 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.051522 11079 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.056937 11324 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:06.064234 11253 raft_consensus.cc:2955] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.073675 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet b9dab54d96124869a4a52f9cc08e288a with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:06.088402 11253 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:06.088974 11934 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:06.090802 12018 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:06.092837 12034 tablet_replica.cc:333] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:06.093792 12018 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:06.094071 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b9dab54d96124869a4a52f9cc08e288a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:06.106043 12034 raft_consensus.cc:2243] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:06.106765 12034 raft_consensus.cc:2272] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:06.111559 11383 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.115159 12034 ts_tablet_manager.cc:1916] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:06.121021 11251 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.128937 11934 raft_consensus.cc:2955] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.139472 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 88ddcb89ab6d47cd97e8c19734f0e365 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:06.149601 11077 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:06.158793 12034 ts_tablet_manager.cc:1929] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:06.159333 12034 log.cc:1199] T b9dab54d96124869a4a52f9cc08e288a P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/b9dab54d96124869a4a52f9cc08e288a
I20260709 12:12:06.160871 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet 88ddcb89ab6d47cd97e8c19734f0e365 with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:06.161943 12034 tablet_replica.cc:333] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:06.162431 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet b9dab54d96124869a4a52f9cc08e288a (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:06.163340 12034 raft_consensus.cc:2243] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:06.164224 12034 raft_consensus.cc:2272] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:06.168280 12034 ts_tablet_manager.cc:1916] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:06.187143 12034 ts_tablet_manager.cc:1929] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:06.187624 12034 log.cc:1199] T 88ddcb89ab6d47cd97e8c19734f0e365 P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/88ddcb89ab6d47cd97e8c19734f0e365
I20260709 12:12:06.189393 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet 88ddcb89ab6d47cd97e8c19734f0e365 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:07.232288 11324 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.240398 11251 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:07.241035 11296 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 99332ff05c3e46579ef9ff825e42a628. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:07.242383 12011 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:07.242892 11176 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:07.244580 11383 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:07.252491 11383 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.254137 11176 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.255725 11251 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.272869 11077 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.306420 11181 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.314853 11324 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:07.319253 11983 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:07.321312 11251 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:07.323333 11983 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:07.335912 11983 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.336798 11326 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.339427 11251 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:12:07.341960 11147 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35c7899b1a314ae1b67824dfee6ff091. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:07.354830 11329 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.354939 11078 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:07.368837 11296 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73b50321123c433f91c5d0972791cf37. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:07.368528 11181 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:07.370968 11251 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:07.371125 11383 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:07.373518 11383 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:07.381881 12023 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.383319 11176 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.396054 11253 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.396100 11176 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.400987 11077 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:07.406174 11329 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:07.408756 11147 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53e6b498179b4c96b8b0be6240a36efb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:07.410429 11253 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:07.412453 12027 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:07.413825 11983 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:07.432204 12027 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.437467 11329 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.468035 11078 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:07.471081 11249 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:07.716343 12058 ts_tablet_manager.cc:933] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:07.766636 12058 tablet_copy_client.cc:323] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:07.771154 12061 ts_tablet_manager.cc:933] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:07.784220 12061 tablet_copy_client.cc:323] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:07.786458 11192 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet 35c7899b1a314ae1b67824dfee6ff091 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:07.787096 11192 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet 35c7899b1a314ae1b67824dfee6ff091 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-35c7899b1a314ae1b67824dfee6ff091
I20260709 12:12:07.787168 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 99332ff05c3e46579ef9ff825e42a628 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:07.787771 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 99332ff05c3e46579ef9ff825e42a628 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-99332ff05c3e46579ef9ff825e42a628
I20260709 12:12:07.795456 11341 tablet_copy_source_session.cc:215] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:07.797435 11192 tablet_copy_source_session.cc:215] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:07.801132 12061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99332ff05c3e46579ef9ff825e42a628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:07.802058 12058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35c7899b1a314ae1b67824dfee6ff091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:07.831594 12065 ts_tablet_manager.cc:933] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:07.833387 12061 tablet_copy_client.cc:806] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:07.834221 12061 tablet_copy_client.cc:670] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:07.844290 12061 tablet_copy_client.cc:538] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:07.859011 12061 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:07.910493 12061 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:07.911130 12058 tablet_copy_client.cc:806] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:07.912050 12061 tablet_bootstrap.cc:492] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:07.912533 12058 tablet_copy_client.cc:670] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:07.913066 12061 ts_tablet_manager.cc:1403] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.054s	user 0.031s	sys 0.014s
I20260709 12:12:07.914451 12065 tablet_copy_client.cc:323] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:07.916289 11192 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet 53e6b498179b4c96b8b0be6240a36efb from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:07.916761 11192 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet 53e6b498179b4c96b8b0be6240a36efb from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-53e6b498179b4c96b8b0be6240a36efb
I20260709 12:12:07.916988 12058 tablet_copy_client.cc:538] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:07.918982 12061 raft_consensus.cc:359] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.920083 12061 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:07.920958 12061 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.928920 12058 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:07.931154 12061 ts_tablet_manager.cc:1434] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.003s
I20260709 12:12:07.932472 11192 tablet_copy_source_session.cc:215] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:07.941242 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-99332ff05c3e46579ef9ff825e42a628 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:07.941788 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-99332ff05c3e46579ef9ff825e42a628 on tablet 99332ff05c3e46579ef9ff825e42a628 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:07.950112 12065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e6b498179b4c96b8b0be6240a36efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:07.969548 12058 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:07.972230 12058 tablet_bootstrap.cc:492] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:07.973088 12058 ts_tablet_manager.cc:1403] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.044s	user 0.023s	sys 0.001s
I20260709 12:12:07.978531 12065 tablet_copy_client.cc:806] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:07.979274 12065 tablet_copy_client.cc:670] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:07.981388 12058 raft_consensus.cc:359] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.982525 12058 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:07.983369 12058 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:07.985054 12065 tablet_copy_client.cc:538] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:07.986128 12061 ts_tablet_manager.cc:933] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:08.001176 12065 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:08.002444 12061 tablet_copy_client.cc:323] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:08.004446 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 73b50321123c433f91c5d0972791cf37 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:08.005044 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 73b50321123c433f91c5d0972791cf37 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-73b50321123c433f91c5d0972791cf37
I20260709 12:12:08.014155 11341 tablet_copy_source_session.cc:215] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:08.029171 12061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73b50321123c433f91c5d0972791cf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:08.030339 12065 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:08.031451 12065 tablet_bootstrap.cc:492] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:08.032244 12065 ts_tablet_manager.cc:1403] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.010s
I20260709 12:12:08.047297 12058 ts_tablet_manager.cc:1434] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 12:12:08.049923 12065 raft_consensus.cc:359] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:08.051031 12065 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:08.051812 12065 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:08.054654 12065 ts_tablet_manager.cc:1434] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.005s
I20260709 12:12:08.059442 11192 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-53e6b498179b4c96b8b0be6240a36efb received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:08.059880 11192 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-53e6b498179b4c96b8b0be6240a36efb on tablet 53e6b498179b4c96b8b0be6240a36efb with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:08.064540 11192 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-35c7899b1a314ae1b67824dfee6ff091 received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:08.064987 11192 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-35c7899b1a314ae1b67824dfee6ff091 on tablet 35c7899b1a314ae1b67824dfee6ff091 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:08.073864 12061 tablet_copy_client.cc:806] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:08.074831 12061 tablet_copy_client.cc:670] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:08.078131 12061 tablet_copy_client.cc:538] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:08.087807 12061 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:08.105120 12061 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:08.106040 12061 tablet_bootstrap.cc:492] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:08.106767 12061 ts_tablet_manager.cc:1403] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.010s
I20260709 12:12:08.109361 12061 raft_consensus.cc:359] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:08.110047 12061 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:08.110630 12061 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:08.116920 11934 raft_consensus.cc:1217] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:08.117321 12061 ts_tablet_manager.cc:1434] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 12:12:08.119318 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-73b50321123c433f91c5d0972791cf37 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:08.119799 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-73b50321123c433f91c5d0972791cf37 on tablet 73b50321123c433f91c5d0972791cf37 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:08.175856 11934 raft_consensus.cc:1217] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:08.434549 11934 raft_consensus.cc:1217] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:08.442224 11934 raft_consensus.cc:1217] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:08.642397 11983 raft_consensus.cc:1064] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:08.644099 11983 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:08.650519 11324 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.651885 11934 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.652487 11249 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.653055 11983 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:08.654251 12027 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:08.655260 12035 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:08.814134 11383 raft_consensus.cc:1064] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:08.815164 11983 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:12:08.819738 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53034.2us
I20260709 12:12:08.859150 11329 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:08.865711 11935 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:08.887794 12081 raft_consensus.cc:1064] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:08.823904 11249 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:08.828015 11383 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:08.890658 12081 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:08.906929 11933 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.907034 11251 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.910723 11176 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.910604 12085 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:08.911475 12052 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:08.912386 12082 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:08.930850 11078 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:08.932839 12027 raft_consensus.cc:1064] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:08.940716 11935 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.952627 12027 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:08.958590 12052 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:08.961114 11251 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.961519 11176 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:08.970635 12082 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:08.972193 11383 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:09.002799 12085 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.095163 12082 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.116298 11249 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.120949 11254 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.125655 11935 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.137580 11933 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:09.138727 11255 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:09.139338 12027 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:09.141754 12084 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:09.141114 11933 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.150529 11329 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:09.157828 11176 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.174921 11179 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.155676 12027 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:12:09.262996 12086 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.264703 11254 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.275147 11324 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.291702 11176 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:09.295274 11078 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.296766 11078 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.316229 11249 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.323625 12084 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:09.330164 11932 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.334643 12027 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.340603 11249 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.364605 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 35c7899b1a314ae1b67824dfee6ff091 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:09.365360 11983 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:09.370872 11933 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.375134 11932 raft_consensus.cc:2955] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.383365 11079 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:09.390496 11076 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.396780 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 35c7899b1a314ae1b67824dfee6ff091 with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:09.408787 12105 tablet_replica.cc:333] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:09.409979 12105 raft_consensus.cc:2243] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:09.410652 12105 raft_consensus.cc:2272] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:09.419493 12105 ts_tablet_manager.cc:1916] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:09.426256 11329 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:09.427205 11328 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:09.441217 11179 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:09.446410 12105 ts_tablet_manager.cc:1929] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:09.446949 12105 log.cc:1199] T 35c7899b1a314ae1b67824dfee6ff091 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/35c7899b1a314ae1b67824dfee6ff091
I20260709 12:12:09.448763 11249 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.450899 12086 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:09.454241 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 35c7899b1a314ae1b67824dfee6ff091 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:09.456828 11175 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.457562 11935 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.458339 11173 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.459632 11935 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.463085 11933 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:09.463680 12082 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:09.463202 12052 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:09.465716 12027 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:09.467100 12082 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:09.468282 12035 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.469547 12082 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:09.475791 11254 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.487222 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53e6b498179b4c96b8b0be6240a36efb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:09.490613 12078 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.491742 12085 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.495810 11079 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:09.498984 11179 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.503111 11933 raft_consensus.cc:2955] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.509743 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 99332ff05c3e46579ef9ff825e42a628 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:09.512087 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 73b50321123c433f91c5d0972791cf37 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:09.508701 11173 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.517597 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 53e6b498179b4c96b8b0be6240a36efb with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:09.518489 12105 tablet_replica.cc:333] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:09.543116 11932 raft_consensus.cc:2955] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.552070 11077 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:09.544171 11933 raft_consensus.cc:2955] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.574851 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet 99332ff05c3e46579ef9ff825e42a628 with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:09.581265 11077 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:09.590121 12105 raft_consensus.cc:2243] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:09.591758 12106 tablet_replica.cc:333] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:09.593255 12106 raft_consensus.cc:2243] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:09.597301 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet 73b50321123c433f91c5d0972791cf37 with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:09.602903 12105 raft_consensus.cc:2272] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:09.604250 12106 raft_consensus.cc:2272] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:09.606499 12105 ts_tablet_manager.cc:1916] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:09.608286 12106 ts_tablet_manager.cc:1916] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:09.627583 12106 ts_tablet_manager.cc:1929] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:09.627792 12105 ts_tablet_manager.cc:1929] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:09.628123 12106 log.cc:1199] T 99332ff05c3e46579ef9ff825e42a628 P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/99332ff05c3e46579ef9ff825e42a628
I20260709 12:12:09.628317 12105 log.cc:1199] T 53e6b498179b4c96b8b0be6240a36efb P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/53e6b498179b4c96b8b0be6240a36efb
I20260709 12:12:09.629837 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet 99332ff05c3e46579ef9ff825e42a628 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:09.630077 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 53e6b498179b4c96b8b0be6240a36efb (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:09.630285 12106 tablet_replica.cc:333] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:09.631283 12106 raft_consensus.cc:2243] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:09.631948 12106 raft_consensus.cc:2272] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:09.637085 12106 ts_tablet_manager.cc:1916] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:09.650696 12106 ts_tablet_manager.cc:1929] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:09.651053 12106 log.cc:1199] T 73b50321123c433f91c5d0972791cf37 P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/73b50321123c433f91c5d0972791cf37
I20260709 12:12:09.652292 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet 73b50321123c433f91c5d0972791cf37 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:10.794687 11255 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:10.801873 11324 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:10.803059 11179 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:10.804127 11222 consensus_peers.cc:597] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa8154ca19364187a13db1be3e0b4f53. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:10.804064 12110 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:10.805312 12060 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:10.831182 12107 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.836172 11328 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.841799 11173 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.858474 11324 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:10.858392 11077 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:10.877226 11176 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:10.877295 11255 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:10.879241 11296 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 458129fddd7e4b33b63d818f6af21f23. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:10.880744 12111 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:10.882292 12085 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:10.894862 12085 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.896912 11176 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.898386 11255 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.919943 11328 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:10.951012 11076 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.965783 11255 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:10.967325 11176 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:10.968981 12122 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:10.971415 12111 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:10.973210 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f62270698314885bc0e03e0de4507ae. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:10.989557 12123 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.993676 11173 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:10.994243 11249 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:11.022066 11076 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:11.030328 11255 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.038760 11328 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:11.040203 12107 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:11.041041 11222 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5fa73285733a4a67a3e705a20155f69d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:11.041330 11173 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:11.042703 12107 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:11.051899 12107 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:11.054468 11328 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:11.062266 11173 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:11.076704 11078 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:11.326741 12131 ts_tablet_manager.cc:933] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:11.328920 12131 tablet_copy_client.cc:323] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:11.331452 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 458129fddd7e4b33b63d818f6af21f23 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:11.332048 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 458129fddd7e4b33b63d818f6af21f23 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-458129fddd7e4b33b63d818f6af21f23
I20260709 12:12:11.342784 11341 tablet_copy_source_session.cc:215] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:11.346761 12131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 458129fddd7e4b33b63d818f6af21f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:11.351400 12133 ts_tablet_manager.cc:933] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:11.353286 12133 tablet_copy_client.cc:323] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:11.355584 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 6f62270698314885bc0e03e0de4507ae from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:11.357762 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 6f62270698314885bc0e03e0de4507ae from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-6f62270698314885bc0e03e0de4507ae
I20260709 12:12:11.364492 11341 tablet_copy_source_session.cc:215] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:11.366228 12131 tablet_copy_client.cc:806] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:11.367019 12131 tablet_copy_client.cc:670] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:11.368408 12133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f62270698314885bc0e03e0de4507ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:11.377874 12131 tablet_copy_client.cc:538] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:11.378065 12135 ts_tablet_manager.cc:933] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:11.388105 12133 tablet_copy_client.cc:806] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:11.388559 12131 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:11.388836 12133 tablet_copy_client.cc:670] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:11.391996 12135 tablet_copy_client.cc:323] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:11.394526 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet aa8154ca19364187a13db1be3e0b4f53 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:11.395138 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet aa8154ca19364187a13db1be3e0b4f53 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-aa8154ca19364187a13db1be3e0b4f53
I20260709 12:12:11.395351 12133 tablet_copy_client.cc:538] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:11.410229 12133 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:11.417579 11267 tablet_copy_source_session.cc:215] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:11.422366 12135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa8154ca19364187a13db1be3e0b4f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:11.442162 12131 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:11.443370 12131 tablet_bootstrap.cc:492] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:11.446277 12131 ts_tablet_manager.cc:1403] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.014s
I20260709 12:12:11.450557 12131 raft_consensus.cc:359] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.451682 12131 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:11.452566 12131 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.456818 12133 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:11.457870 12133 tablet_bootstrap.cc:492] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:11.458585 12133 ts_tablet_manager.cc:1403] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.049s	user 0.029s	sys 0.011s
I20260709 12:12:11.461603 12133 raft_consensus.cc:359] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.462525 12133 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:11.463465 12133 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.478765 12141 ts_tablet_manager.cc:933] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:11.497295 12133 ts_tablet_manager.cc:1434] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.038s	user 0.002s	sys 0.018s
I20260709 12:12:11.501560 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-6f62270698314885bc0e03e0de4507ae received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:11.502069 12141 tablet_copy_client.cc:323] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:11.502147 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-6f62270698314885bc0e03e0de4507ae on tablet 6f62270698314885bc0e03e0de4507ae with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:11.511741 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 5fa73285733a4a67a3e705a20155f69d from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:11.512331 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 5fa73285733a4a67a3e705a20155f69d from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-5fa73285733a4a67a3e705a20155f69d
I20260709 12:12:11.529343 11267 tablet_copy_source_session.cc:215] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:11.544595 12131 ts_tablet_manager.cc:1434] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.098s	user 0.026s	sys 0.020s
I20260709 12:12:11.546244 12141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa73285733a4a67a3e705a20155f69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:11.546460 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-458129fddd7e4b33b63d818f6af21f23 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:11.546911 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-458129fddd7e4b33b63d818f6af21f23 on tablet 458129fddd7e4b33b63d818f6af21f23 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:11.547878 12135 tablet_copy_client.cc:806] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:11.548622 12135 tablet_copy_client.cc:670] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:11.552883 12135 tablet_copy_client.cc:538] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:11.567723 12135 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:11.571633 12141 tablet_copy_client.cc:806] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:11.572383 12141 tablet_copy_client.cc:670] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:11.576047 12141 tablet_copy_client.cc:538] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:11.587211 12141 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:11.594784 12135 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:11.595793 12135 tablet_bootstrap.cc:492] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:11.596549 12135 ts_tablet_manager.cc:1403] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.008s
I20260709 12:12:11.600095 12135 raft_consensus.cc:359] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.601073 12135 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:11.601914 12135 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.604779 12135 ts_tablet_manager.cc:1434] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:12:11.608000 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-aa8154ca19364187a13db1be3e0b4f53 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:11.608428 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-aa8154ca19364187a13db1be3e0b4f53 on tablet aa8154ca19364187a13db1be3e0b4f53 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:11.612597 12141 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:11.613533 12141 tablet_bootstrap.cc:492] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:11.614234 12141 ts_tablet_manager.cc:1403] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.004s
I20260709 12:12:11.617591 12141 raft_consensus.cc:359] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.618481 12141 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:11.619300 12141 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:11.622184 12141 ts_tablet_manager.cc:1434] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:12:11.629376 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-5fa73285733a4a67a3e705a20155f69d received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:11.629953 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-5fa73285733a4a67a3e705a20155f69d on tablet 5fa73285733a4a67a3e705a20155f69d with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:11.808947 11933 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:11.922914 11933 raft_consensus.cc:1217] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:11.948872 11932 raft_consensus.cc:1217] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:11.970031 11933 raft_consensus.cc:1217] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:12.258971 12123 raft_consensus.cc:1064] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:12.261376 12123 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:12.269188 11933 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.269932 11254 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.272076 12085 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:12.273080 12122 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:12.278283 11176 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.281327 12122 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:12.294322 12122 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.305114 11254 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.315635 11932 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.318897 11176 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.331830 11076 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:12.363030 12123 raft_consensus.cc:1064] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:12.366847 12123 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:12.405598 11932 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.406133 11254 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.407193 12146 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:12.407706 11176 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.408845 12122 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:12.410373 12123 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:12.416213 11328 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:12.434942 12083 raft_consensus.cc:1064] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:12.439368 12083 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:12.457473 11933 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:12.461086 11254 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:12.466341 12146 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:12.469923 12085 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:12.474845 11324 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.475361 12123 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.478790 11932 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.480516 11176 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.483588 12110 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:12.494277 12085 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.494181 11935 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.504593 12107 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:12.506240 11078 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.512100 11254 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.512733 11249 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.522454 12083 raft_consensus.cc:1064] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:12.526434 12083 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:12.546936 11331 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:12.540721 11078 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.560984 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet 458129fddd7e4b33b63d818f6af21f23 with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:12.562877 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 458129fddd7e4b33b63d818f6af21f23 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:12.567875 11173 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.574664 12110 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:12.592125 11933 raft_consensus.cc:2955] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.601060 11932 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.611971 12107 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:12.646507 11176 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:12.652010 11173 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.665722 11932 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:12.688390 11329 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:12.691314 12169 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.698124 12168 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:12.703009 12170 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:12.718240 11173 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.721979 12163 tablet_replica.cc:333] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:12.721462 11329 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.723052 12085 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:12.775581 12163 raft_consensus.cc:2243] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.777279 12085 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:12.801515 12163 raft_consensus.cc:2272] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.806141 12163 ts_tablet_manager.cc:1916] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:12.823802 12163 ts_tablet_manager.cc:1929] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:12.824416 12163 log.cc:1199] T 458129fddd7e4b33b63d818f6af21f23 P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/458129fddd7e4b33b63d818f6af21f23
I20260709 12:12:12.827497 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet 458129fddd7e4b33b63d818f6af21f23 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:12.846792 11077 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.858603 11935 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.876192 12107 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.911464 12122 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.900175 11329 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.919689 11932 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.925702 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6f62270698314885bc0e03e0de4507ae with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:12.943928 11176 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.951232 11935 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.970504 11077 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:12.979990 11176 raft_consensus.cc:2955] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:12.997927 11076 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:13.006774 11249 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:13.013509 11329 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:13.013754 11933 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:13.015213 12110 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:13.017800 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f62270698314885bc0e03e0de4507ae with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:13.024359 12107 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:13.025750 12164 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:13.029229 12176 tablet_replica.cc:333] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:13.028990 11328 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:13.030287 12176 raft_consensus.cc:2243] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:13.032048 12176 raft_consensus.cc:2272] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:13.030850 11933 raft_consensus.cc:2955] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:13.039520 12176 ts_tablet_manager.cc:1916] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:13.046916 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet aa8154ca19364187a13db1be3e0b4f53 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:13.052112 11249 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:13.058670 12176 ts_tablet_manager.cc:1929] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:13.059309 12176 log.cc:1199] T 6f62270698314885bc0e03e0de4507ae P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/6f62270698314885bc0e03e0de4507ae
I20260709 12:12:13.060829 11935 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:13.061002 11176 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:13.063022 12110 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:13.064785 12110 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:13.064821 11078 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:13.064837 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet 6f62270698314885bc0e03e0de4507ae (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:13.075989 12164 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:13.080626 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet aa8154ca19364187a13db1be3e0b4f53 with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:13.082087 12163 tablet_replica.cc:333] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:13.083305 12163 raft_consensus.cc:2243] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:13.084118 12163 raft_consensus.cc:2272] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:13.083429 11176 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:13.087118 11935 raft_consensus.cc:2955] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:13.089668 12163 ts_tablet_manager.cc:1916] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:13.108875 11076 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:13.114338 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5fa73285733a4a67a3e705a20155f69d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:13.129253 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa73285733a4a67a3e705a20155f69d with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:13.133093 12163 ts_tablet_manager.cc:1929] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:13.133755 12163 log.cc:1199] T aa8154ca19364187a13db1be3e0b4f53 P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/aa8154ca19364187a13db1be3e0b4f53
I20260709 12:12:13.136830 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet aa8154ca19364187a13db1be3e0b4f53 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:13.137795 12177 tablet_replica.cc:333] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:13.139142 12177 raft_consensus.cc:2243] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:13.140007 12177 raft_consensus.cc:2272] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:13.143963 12177 ts_tablet_manager.cc:1916] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:13.161584 12177 ts_tablet_manager.cc:1929] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:13.162285 12177 log.cc:1199] T 5fa73285733a4a67a3e705a20155f69d P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/5fa73285733a4a67a3e705a20155f69d
I20260709 12:12:13.164207 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 5fa73285733a4a67a3e705a20155f69d (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:14.208920 11173 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:14.216990 11328 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:14.219142 11147 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ad54acf641349bc8119ce4c0eae446b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:14.219792 11249 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:14.220925 12130 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:14.223073 12175 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:14.230345 12130 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.232059 11328 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.233956 11255 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.249225 11255 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:14.253101 11078 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.259791 11328 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:14.260212 11222 consensus_peers.cc:597] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbf57c68621b4e90a77c76eb3974b80e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:14.262063 12107 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:14.266746 11176 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:14.268415 12164 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:14.298265 12107 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.300297 11328 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.315480 11176 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.329322 11329 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:14.348599 11296 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39571de4248f4cfd85f9b1062a776cc8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:14.349260 11254 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:14.346350 11077 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.351027 11176 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:14.351656 12085 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:14.352841 12122 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:14.362092 12085 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.364185 11254 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.370220 11173 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.378634 11176 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:14.390478 11078 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.392585 11328 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:14.395388 12130 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:14.397375 11254 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:14.399616 12130 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:14.403097 11147 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f62903fc48c140aca4aa949625fa88f6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:14.417181 12130 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.418766 11254 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.421339 11329 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.439930 11077 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:14.822650 12200 ts_tablet_manager.cc:933] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:14.825431 12200 tablet_copy_client.cc:323] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:14.827523 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet bbf57c68621b4e90a77c76eb3974b80e from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:14.828085 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet bbf57c68621b4e90a77c76eb3974b80e from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-bbf57c68621b4e90a77c76eb3974b80e
I20260709 12:12:14.836431 11267 tablet_copy_source_session.cc:215] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:14.840381 12202 ts_tablet_manager.cc:933] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:14.842980 12200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf57c68621b4e90a77c76eb3974b80e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:14.850344 12203 ts_tablet_manager.cc:933] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:14.852329 12202 tablet_copy_client.cc:323] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:14.855086 11192 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet 5ad54acf641349bc8119ce4c0eae446b from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:14.855643 11192 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet 5ad54acf641349bc8119ce4c0eae446b from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-5ad54acf641349bc8119ce4c0eae446b
I20260709 12:12:14.869251 12203 tablet_copy_client.cc:323] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:14.871071 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 39571de4248f4cfd85f9b1062a776cc8 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:14.871600 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 39571de4248f4cfd85f9b1062a776cc8 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-39571de4248f4cfd85f9b1062a776cc8
I20260709 12:12:14.876045 12200 tablet_copy_client.cc:806] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:14.876801 12200 tablet_copy_client.cc:670] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:14.877430 11192 tablet_copy_source_session.cc:215] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:14.881390 12200 tablet_copy_client.cc:538] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:14.885155 12202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad54acf641349bc8119ce4c0eae446b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:14.886848 11341 tablet_copy_source_session.cc:215] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:14.892832 12203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39571de4248f4cfd85f9b1062a776cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:14.898305 12200 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:14.915006 12202 tablet_copy_client.cc:806] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:14.916000 12202 tablet_copy_client.cc:670] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:14.920476 12202 tablet_copy_client.cc:538] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:14.920904 12203 tablet_copy_client.cc:806] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:14.922983 12203 tablet_copy_client.cc:670] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:14.926484 12200 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:14.927901 12200 tablet_bootstrap.cc:492] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:14.928653 12200 ts_tablet_manager.cc:1403] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.001s
I20260709 12:12:14.931842 12200 raft_consensus.cc:359] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:14.932761 12200 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:14.933282 12203 tablet_copy_client.cc:538] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:14.933524 12200 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:14.934181 12202 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:14.943539 12203 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:14.978086 12200 ts_tablet_manager.cc:1434] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.049s	user 0.004s	sys 0.009s
I20260709 12:12:14.980685 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-bbf57c68621b4e90a77c76eb3974b80e received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:14.981163 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-bbf57c68621b4e90a77c76eb3974b80e on tablet bbf57c68621b4e90a77c76eb3974b80e with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:14.994966 12200 ts_tablet_manager.cc:933] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:15.039455 12202 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:15.042692 12202 tablet_bootstrap.cc:492] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:15.043419 12202 ts_tablet_manager.cc:1403] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.109s	user 0.017s	sys 0.014s
I20260709 12:12:15.046404 12202 raft_consensus.cc:359] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:15.047178 12202 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:15.047731 12202 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:15.041255 12200 tablet_copy_client.cc:323] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:15.050688 11192 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet f62903fc48c140aca4aa949625fa88f6 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:15.051467 11192 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet f62903fc48c140aca4aa949625fa88f6 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-f62903fc48c140aca4aa949625fa88f6
I20260709 12:12:15.061390 11192 tablet_copy_source_session.cc:215] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:15.064545 12203 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:15.068651 12203 tablet_bootstrap.cc:492] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:15.069172 12200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62903fc48c140aca4aa949625fa88f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:15.069451 12203 ts_tablet_manager.cc:1403] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.126s	user 0.053s	sys 0.060s
I20260709 12:12:15.058983 12202 ts_tablet_manager.cc:1434] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:12:15.071606 11192 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-5ad54acf641349bc8119ce4c0eae446b received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:15.072151 11192 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-5ad54acf641349bc8119ce4c0eae446b on tablet 5ad54acf641349bc8119ce4c0eae446b with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:15.075379 12203 raft_consensus.cc:359] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:15.076503 12203 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:15.077318 12203 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:15.080345 12203 ts_tablet_manager.cc:1434] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:12:15.082329 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-39571de4248f4cfd85f9b1062a776cc8 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:15.082826 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-39571de4248f4cfd85f9b1062a776cc8 on tablet 39571de4248f4cfd85f9b1062a776cc8 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:15.091012 12200 tablet_copy_client.cc:806] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:15.091794 12200 tablet_copy_client.cc:670] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:15.099799 12200 tablet_copy_client.cc:538] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:15.106769 12200 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:15.143896 12200 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:15.144912 12200 tablet_bootstrap.cc:492] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:15.145859 12200 ts_tablet_manager.cc:1403] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.007s
I20260709 12:12:15.149071 12200 raft_consensus.cc:359] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:15.149991 12200 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:15.150739 12200 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:15.153249 12200 ts_tablet_manager.cc:1434] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:12:15.155357 11192 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-f62903fc48c140aca4aa949625fa88f6 received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:15.155809 11192 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-f62903fc48c140aca4aa949625fa88f6 on tablet f62903fc48c140aca4aa949625fa88f6 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:15.379694 11933 raft_consensus.cc:1217] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:15.392388 11933 raft_consensus.cc:1217] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:15.405016 11933 raft_consensus.cc:1217] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:15.432746 11933 raft_consensus.cc:1217] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:15.838032 12130 raft_consensus.cc:1064] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:15.842478 12130 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:15.858135 11329 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:15.858269 11933 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:15.860558 11254 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:15.860874 12183 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:15.869874 12192 raft_consensus.cc:1064] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:15.871584 12183 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:15.874188 12130 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:15.881186 12192 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:15.908043 11255 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:15.910429 11932 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:15.911101 11175 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:15.913105 12085 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:15.914726 12122 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:15.918720 12190 raft_consensus.cc:1064] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:15.923307 12085 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:15.938201 12216 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:15.943316 12190 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:15.950220 11254 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:15.962904 11078 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.968735 12215 raft_consensus.cc:1064] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:15.971969 12215 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:15.990460 11329 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:15.995575 12191 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:16.004808 11176 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:16.021198 12229 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:16.023298 11932 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:16.042377 11329 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.047618 12107 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:12:16.053619 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 73392.3us
I20260709 12:12:16.074191 11933 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.074713 12122 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.088861 11255 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:12:16.092244 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53218.8us
I20260709 12:12:16.099057 11935 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.160154 11173 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:16.173954 11076 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.177547 11935 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:16.180840 12212 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:16.187645 11255 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:16.190940 11328 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:16.192028 12212 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:12:16.196192 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 158268us
I20260709 12:12:16.200281 11176 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.226166 11932 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.227478 11249 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.229137 12218 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:16.230855 12212 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:16.232151 12218 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:16.245201 12215 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.255100 11932 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.257579 12183 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.259840 12230 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.265197 11176 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.269816 11329 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.278842 11079 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.283087 11249 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.284427 11935 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.290242 11255 raft_consensus.cc:2955] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.293007 11329 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.295634 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ad54acf641349bc8119ce4c0eae446b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:16.301410 11328 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:16.305193 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad54acf641349bc8119ce4c0eae446b with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:16.318697 11933 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.322868 12244 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:16.327854 11176 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.329222 11935 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.334242 12228 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:16.334259 11079 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.337275 12228 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.338846 11076 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.341840 12246 tablet_replica.cc:333] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:16.342846 12246 raft_consensus.cc:2243] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:16.343638 12246 raft_consensus.cc:2272] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:16.346376 11933 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.356906 11255 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:16.360158 12246 ts_tablet_manager.cc:1916] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:16.349370 11176 raft_consensus.cc:2955] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.364210 11328 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.369446 12107 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:16.372711 11179 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:16.373243 11078 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.380075 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 39571de4248f4cfd85f9b1062a776cc8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:16.387660 12246 ts_tablet_manager.cc:1929] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:16.388168 12246 log.cc:1199] T 5ad54acf641349bc8119ce4c0eae446b P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/5ad54acf641349bc8119ce4c0eae446b
I20260709 12:12:16.388402 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet 39571de4248f4cfd85f9b1062a776cc8 with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:16.365310 11932 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.391484 12229 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:16.391938 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 5ad54acf641349bc8119ce4c0eae446b (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:16.395368 12230 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.397109 11932 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.399343 12183 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:16.404436 11328 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:16.409205 12212 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:16.411962 12215 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.411803 11933 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.413172 11935 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.417994 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bbf57c68621b4e90a77c76eb3974b80e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:16.426244 11331 raft_consensus.cc:2955] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.429405 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f62903fc48c140aca4aa949625fa88f6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:16.431142 11328 raft_consensus.cc:2955] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.443117 12247 tablet_replica.cc:333] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:16.452049 11076 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:16.463655 12247 raft_consensus.cc:2243] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:16.463831 11079 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:16.464617 12247 raft_consensus.cc:2272] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:16.473250 12247 ts_tablet_manager.cc:1916] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:16.484270 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet f62903fc48c140aca4aa949625fa88f6 with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:16.492632 12247 ts_tablet_manager.cc:1929] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:16.493662 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet bbf57c68621b4e90a77c76eb3974b80e with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:16.493242 12247 log.cc:1199] T 39571de4248f4cfd85f9b1062a776cc8 P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/39571de4248f4cfd85f9b1062a776cc8
I20260709 12:12:16.498274 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet 39571de4248f4cfd85f9b1062a776cc8 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:16.501129 12247 tablet_replica.cc:333] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:16.502351 12248 tablet_replica.cc:333] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:16.502452 12247 raft_consensus.cc:2243] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:16.503234 12247 raft_consensus.cc:2272] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:16.503464 12248 raft_consensus.cc:2243] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:16.504238 12248 raft_consensus.cc:2272] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:16.507277 12247 ts_tablet_manager.cc:1916] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:16.508524 12248 ts_tablet_manager.cc:1916] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:16.523134 12247 ts_tablet_manager.cc:1929] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:16.523629 12247 log.cc:1199] T f62903fc48c140aca4aa949625fa88f6 P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/f62903fc48c140aca4aa949625fa88f6
I20260709 12:12:16.525272 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet f62903fc48c140aca4aa949625fa88f6 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:16.526574 12248 ts_tablet_manager.cc:1929] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:16.527045 12248 log.cc:1199] T bbf57c68621b4e90a77c76eb3974b80e P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/bbf57c68621b4e90a77c76eb3974b80e
I20260709 12:12:16.528342 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet bbf57c68621b4e90a77c76eb3974b80e (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:17.591810 11179 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:17.598074 11328 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:17.599236 11255 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:17.600168 12183 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:17.601613 12215 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:12:17.603992 11147 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5f804d45235415fad0f1442ea04aa47. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:17.608470 12250 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.610092 11328 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.611897 11255 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.628425 11255 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:17.629689 11077 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:17.635255 11329 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:17.636868 12230 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:17.641222 11179 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:17.643956 11222 consensus_peers.cc:597] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fb796253655415ba47e989105d7c807. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:17.646559 12107 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:17.667233 12230 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.668939 11179 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.677560 11329 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.690574 11179 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:17.701370 11147 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02e478aac72b4a62a8ed9124187f2dec. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:17.705464 11329 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:17.706604 12183 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:17.708025 11076 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.714386 11255 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:17.716002 12250 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:17.717909 12250 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.719415 11329 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.732568 11255 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.736142 11078 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.745029 11255 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:17.756420 11328 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:17.758515 12230 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:17.762650 11176 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:17.768673 12259 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:17.769662 12230 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.772409 11328 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:12:17.810209 11222 consensus_peers.cc:597] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9424adcaacdf40f3bbd2f639ef2a48c2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:17.822026 11173 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:17.823001 11076 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:18.286700 12274 ts_tablet_manager.cc:933] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:18.287333 12272 ts_tablet_manager.cc:933] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:18.297544 12274 tablet_copy_client.cc:323] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:18.298520 12272 tablet_copy_client.cc:323] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:18.299435 11192 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet 02e478aac72b4a62a8ed9124187f2dec from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:18.299964 11192 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet 02e478aac72b4a62a8ed9124187f2dec from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-02e478aac72b4a62a8ed9124187f2dec
I20260709 12:12:18.300314 11191 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet e5f804d45235415fad0f1442ea04aa47 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:18.300832 11191 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet e5f804d45235415fad0f1442ea04aa47 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-e5f804d45235415fad0f1442ea04aa47
I20260709 12:12:18.309087 11192 tablet_copy_source_session.cc:215] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:18.311439 11191 tablet_copy_source_session.cc:215] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:18.314289 12274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e478aac72b4a62a8ed9124187f2dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:18.350620 12274 tablet_copy_client.cc:806] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:18.359570 12272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5f804d45235415fad0f1442ea04aa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:18.362036 12274 tablet_copy_client.cc:670] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:18.375418 12274 tablet_copy_client.cc:538] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:18.384871 12274 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:18.387959 12278 ts_tablet_manager.cc:933] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:18.404572 12278 tablet_copy_client.cc:323] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:18.412849 12272 tablet_copy_client.cc:806] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:18.417196 12272 tablet_copy_client.cc:670] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:18.418737 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 9424adcaacdf40f3bbd2f639ef2a48c2 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:18.419317 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 9424adcaacdf40f3bbd2f639ef2a48c2 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-9424adcaacdf40f3bbd2f639ef2a48c2
I20260709 12:12:18.422297 12272 tablet_copy_client.cc:538] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:18.429347 11267 tablet_copy_source_session.cc:215] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:18.433531 12272 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:18.433823 12278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9424adcaacdf40f3bbd2f639ef2a48c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:18.446684 12280 ts_tablet_manager.cc:933] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:18.454701 12280 tablet_copy_client.cc:323] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:18.454917 12274 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:18.460147 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 1fb796253655415ba47e989105d7c807 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:18.460716 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 1fb796253655415ba47e989105d7c807 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-1fb796253655415ba47e989105d7c807
I20260709 12:12:18.461563 12274 tablet_bootstrap.cc:492] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:18.462430 12274 ts_tablet_manager.cc:1403] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.004s
I20260709 12:12:18.468484 12278 tablet_copy_client.cc:806] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:18.469158 12278 tablet_copy_client.cc:670] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:18.470237 12274 raft_consensus.cc:359] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.472038 12274 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:18.472585 12274 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.473865 11267 tablet_copy_source_session.cc:215] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:18.477713 12280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb796253655415ba47e989105d7c807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:18.482308 12274 ts_tablet_manager.cc:1434] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.009s
I20260709 12:12:18.484599 11191 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-02e478aac72b4a62a8ed9124187f2dec received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:18.485028 11191 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-02e478aac72b4a62a8ed9124187f2dec on tablet 02e478aac72b4a62a8ed9124187f2dec with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:18.488304 12278 tablet_copy_client.cc:538] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:18.497710 12272 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:18.498729 12272 tablet_bootstrap.cc:492] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:18.499081 12278 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:18.499451 12272 ts_tablet_manager.cc:1403] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.066s	user 0.021s	sys 0.007s
I20260709 12:12:18.502920 12272 raft_consensus.cc:359] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.503789 12272 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:18.504436 12272 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.512755 12272 ts_tablet_manager.cc:1434] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:12:18.516607 11191 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-e5f804d45235415fad0f1442ea04aa47 received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:18.517251 11191 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-e5f804d45235415fad0f1442ea04aa47 on tablet e5f804d45235415fad0f1442ea04aa47 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:18.513958 12280 tablet_copy_client.cc:806] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:18.527184 12280 tablet_copy_client.cc:670] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:18.530921 12280 tablet_copy_client.cc:538] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:18.534273 12278 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:18.535110 12278 tablet_bootstrap.cc:492] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:18.535686 12278 ts_tablet_manager.cc:1403] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.020s
I20260709 12:12:18.538383 12278 raft_consensus.cc:359] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.539194 12278 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:18.539901 12278 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.543699 12278 ts_tablet_manager.cc:1434] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:12:18.547639 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-9424adcaacdf40f3bbd2f639ef2a48c2 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:18.548079 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-9424adcaacdf40f3bbd2f639ef2a48c2 on tablet 9424adcaacdf40f3bbd2f639ef2a48c2 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:18.541116 12280 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:18.574609 12280 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:18.575326 12280 tablet_bootstrap.cc:492] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:18.576020 12280 ts_tablet_manager.cc:1403] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.007s
I20260709 12:12:18.578876 12280 raft_consensus.cc:359] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.579666 12280 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:18.580300 12280 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:18.582841 12280 ts_tablet_manager.cc:1434] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:12:18.584677 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-1fb796253655415ba47e989105d7c807 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:18.585062 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-1fb796253655415ba47e989105d7c807 on tablet 1fb796253655415ba47e989105d7c807 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:18.726926 11932 raft_consensus.cc:1217] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:18.752688 11932 raft_consensus.cc:1217] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:18.844771 11932 raft_consensus.cc:1217] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:18.888200 11932 raft_consensus.cc:1217] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:19.165926 12265 raft_consensus.cc:1064] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:19.168326 12265 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.174660 11329 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.175863 11932 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.177527 12271 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:19.186448 11251 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.191135 12270 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:19.195766 12265 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.198454 12255 raft_consensus.cc:1064] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:19.201324 12255 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.265075 12270 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.273859 11329 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.276939 11932 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:12:19.279129 11956 debug-util.cc:398] Leaking SignalData structure 0x7b080014d120 after lost signal to thread 10014
W20260709 12:12:19.280174 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002afe40 after lost signal to thread 11113
W20260709 12:12:19.281287 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002b3220 after lost signal to thread 11206
I20260709 12:12:19.282517 12275 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:12:19.284412 11956 debug-util.cc:398] Leaking SignalData structure 0x7b080032f660 after lost signal to thread 11281
W20260709 12:12:19.285611 11956 debug-util.cc:398] Leaking SignalData structure 0x7b080004ba40 after lost signal to thread 11355
W20260709 12:12:19.287155 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8160 after lost signal to thread 11959
I20260709 12:12:19.291574 11331 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.293057 11251 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.329551 11077 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:12:19.339309 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57387.7us
I20260709 12:12:19.347535 11932 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.350297 12230 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:19.370671 11173 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.379917 12297 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:12:19.406630 12298 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.411002 11935 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.428570 12296 raft_consensus.cc:1064] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:19.433698 11331 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.434231 12296 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.432844 11179 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.435257 12302 raft_consensus.cc:1064] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:19.466442 12302 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.489944 11179 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.484810 11077 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:19.511394 11331 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.514555 12270 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.517781 11935 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:19.535272 11251 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.520792 11329 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:19.524019 11255 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.519119 11932 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.538488 12300 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:19.549044 12300 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:19.555800 12265 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.557723 12265 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.558390 11935 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:19.563894 12275 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:19.568531 11324 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.569121 11179 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.571066 11329 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:19.570879 12298 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.582305 11932 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:19.582942 12230 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.584815 12230 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:19.587116 12265 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.603801 12230 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.607302 11932 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.619614 11328 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.596170 11324 raft_consensus.cc:2955] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.633694 12301 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.636456 12296 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:19.646915 11935 raft_consensus.cc:2955] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.653302 11934 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.683845 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 02e478aac72b4a62a8ed9124187f2dec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:19.700533 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1fb796253655415ba47e989105d7c807 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:19.715826 11076 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.718768 11079 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:19.729200 12296 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.731487 11077 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:19.740144 11331 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.742935 11330 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.764282 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e478aac72b4a62a8ed9124187f2dec with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:19.767344 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fb796253655415ba47e989105d7c807 with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:19.781944 12320 tablet_replica.cc:333] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:19.805778 12320 raft_consensus.cc:2243] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:19.808123 12320 raft_consensus.cc:2272] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:19.795322 11251 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.832656 11173 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.831760 11077 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.855242 12320 ts_tablet_manager.cc:1916] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:19.856318 11935 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.892840 11932 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:19.898298 12302 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:19.910226 11179 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.911772 12321 tablet_replica.cc:333] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:19.928392 12321 raft_consensus.cc:2243] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:19.929540 12321 raft_consensus.cc:2272] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:19.927637 12302 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.932343 11255 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:19.936245 11935 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.939230 11932 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:19.942667 12321 ts_tablet_manager.cc:1916] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:19.943470 11249 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:12:19.944270 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 118849us
I20260709 12:12:19.949507 12230 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:19.950889 12320 ts_tablet_manager.cc:1929] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:19.951333 12320 log.cc:1199] T 02e478aac72b4a62a8ed9124187f2dec P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/02e478aac72b4a62a8ed9124187f2dec
I20260709 12:12:19.953229 12300 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:19.953994 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet 02e478aac72b4a62a8ed9124187f2dec (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:19.962356 12322 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.964085 11934 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.965756 11251 raft_consensus.cc:2955] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.969889 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e5f804d45235415fad0f1442ea04aa47 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:19.976456 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9424adcaacdf40f3bbd2f639ef2a48c2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:19.984560 11078 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:19.992982 11331 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:19.998878 12321 ts_tablet_manager.cc:1929] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:20.009066 12321 log.cc:1199] T 1fb796253655415ba47e989105d7c807 P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/1fb796253655415ba47e989105d7c807
I20260709 12:12:20.010996 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet 1fb796253655415ba47e989105d7c807 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:20.014199 12324 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:20.020780 11331 raft_consensus.cc:2955] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:20.037979 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet e5f804d45235415fad0f1442ea04aa47 with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:20.041965 11079 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:20.066845 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet 9424adcaacdf40f3bbd2f639ef2a48c2 with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:20.067973 12321 tablet_replica.cc:333] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:20.069149 12321 raft_consensus.cc:2243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:20.070082 12321 raft_consensus.cc:2272] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:20.076759 12321 ts_tablet_manager.cc:1916] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:20.091511 12325 tablet_replica.cc:333] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:20.099603 12325 raft_consensus.cc:2243] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:20.100728 12325 raft_consensus.cc:2272] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:20.112680 12325 ts_tablet_manager.cc:1916] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:20.124661 12321 ts_tablet_manager.cc:1929] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:20.125285 12321 log.cc:1199] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/9424adcaacdf40f3bbd2f639ef2a48c2
I20260709 12:12:20.127189 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet 9424adcaacdf40f3bbd2f639ef2a48c2 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:20.136020 12325 ts_tablet_manager.cc:1929] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:20.136502 12325 log.cc:1199] T e5f804d45235415fad0f1442ea04aa47 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/e5f804d45235415fad0f1442ea04aa47
I20260709 12:12:20.138484 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet e5f804d45235415fad0f1442ea04aa47 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:21.193614 11173 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:21.214004 11147 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ddf8cca10e1244d8882abd13fa81113a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:21.220345 11251 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:21.222550 12332 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:21.223233 11331 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:21.227716 12333 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:21.253854 12300 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.255276 11251 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.262427 11330 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.279034 11078 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.306317 11330 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:21.314738 11251 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:21.315670 11179 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:21.316540 12228 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:12:21.320187 11296 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a629d78b61074c2cbd007926bd78fa29. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:21.322005 12228 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.348466 12328 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.351290 11249 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.353423 11179 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.369338 11251 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:21.381434 11077 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:21.438433 11222 consensus_peers.cc:597] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38d626794db04be2b9c36cc76fcf2183. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:21.440366 11179 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:21.440371 11330 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:21.442171 12230 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:21.443326 12324 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.464286 12324 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.470585 11179 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.483438 11330 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.489709 11077 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.515518 11176 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:21.539479 11330 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:21.540364 11147 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89068ffedf554d89af62a8b1fe0700c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:21.540975 12300 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.542796 11249 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:21.544034 12300 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.551601 12332 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.556488 11330 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.562403 11249 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:21.582799 11077 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.635136 12350 ts_tablet_manager.cc:933] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:21.637902 12350 tablet_copy_client.cc:323] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:21.640525 11191 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet ddf8cca10e1244d8882abd13fa81113a from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:21.641278 11191 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet ddf8cca10e1244d8882abd13fa81113a from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-ddf8cca10e1244d8882abd13fa81113a
I20260709 12:12:21.651401 11191 tablet_copy_source_session.cc:215] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:21.668037 12350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddf8cca10e1244d8882abd13fa81113a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:21.705044 12350 tablet_copy_client.cc:806] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:21.705745 12350 tablet_copy_client.cc:670] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:21.710469 12350 tablet_copy_client.cc:538] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:21.722420 12350 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:21.761168 12350 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:21.762257 12350 tablet_bootstrap.cc:492] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:21.763026 12350 ts_tablet_manager.cc:1403] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.012s
I20260709 12:12:21.766919 12350 raft_consensus.cc:359] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:21.767963 12350 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:21.768975 12350 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:21.793525 12350 ts_tablet_manager.cc:1434] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.004s
I20260709 12:12:21.795374 11191 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-ddf8cca10e1244d8882abd13fa81113a received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:21.795885 11191 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-ddf8cca10e1244d8882abd13fa81113a on tablet ddf8cca10e1244d8882abd13fa81113a with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:21.834100 12350 ts_tablet_manager.cc:933] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:21.835896 12350 tablet_copy_client.cc:323] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:21.838152 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet a629d78b61074c2cbd007926bd78fa29 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:21.838789 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet a629d78b61074c2cbd007926bd78fa29 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-a629d78b61074c2cbd007926bd78fa29
I20260709 12:12:21.852164 11341 tablet_copy_source_session.cc:215] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:21.855906 12350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a629d78b61074c2cbd007926bd78fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:21.883349 12350 tablet_copy_client.cc:806] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:21.884011 12350 tablet_copy_client.cc:670] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:21.888298 12350 tablet_copy_client.cc:538] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:21.907127 12350 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:21.951663 12350 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:21.952868 12350 tablet_bootstrap.cc:492] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:21.953944 12350 ts_tablet_manager.cc:1403] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.047s	user 0.029s	sys 0.002s
I20260709 12:12:21.957811 12350 raft_consensus.cc:359] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:21.958834 12350 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:21.959564 12350 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:21.962909 12350 ts_tablet_manager.cc:1434] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:12:21.969108 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-a629d78b61074c2cbd007926bd78fa29 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:21.969606 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-a629d78b61074c2cbd007926bd78fa29 on tablet a629d78b61074c2cbd007926bd78fa29 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:22.010895 12350 ts_tablet_manager.cc:933] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:22.031930 12350 tablet_copy_client.cc:323] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:22.035113 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 38d626794db04be2b9c36cc76fcf2183 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:22.035704 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 38d626794db04be2b9c36cc76fcf2183 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-38d626794db04be2b9c36cc76fcf2183
I20260709 12:12:22.044502 11267 tablet_copy_source_session.cc:215] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:22.048418 12350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d626794db04be2b9c36cc76fcf2183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:22.176111 12350 tablet_copy_client.cc:806] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:22.177034 12350 tablet_copy_client.cc:670] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:22.185009 11933 raft_consensus.cc:1217] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:22.193203 12356 ts_tablet_manager.cc:933] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:22.196058 12356 tablet_copy_client.cc:323] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:22.199662 11191 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet 89068ffedf554d89af62a8b1fe0700c0 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:22.200258 11191 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet 89068ffedf554d89af62a8b1fe0700c0 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-89068ffedf554d89af62a8b1fe0700c0
I20260709 12:12:22.234282 11191 tablet_copy_source_session.cc:215] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:22.254604 12350 tablet_copy_client.cc:538] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:22.263592 12350 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:22.273645 12356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89068ffedf554d89af62a8b1fe0700c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:22.297966 12356 tablet_copy_client.cc:806] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:22.298619 12356 tablet_copy_client.cc:670] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:22.303860 11935 raft_consensus.cc:1217] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:22.309477 12356 tablet_copy_client.cc:538] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:22.344256 12356 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:22.349792 12350 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:22.356178 12350 tablet_bootstrap.cc:492] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:22.357028 12350 ts_tablet_manager.cc:1403] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.094s	user 0.022s	sys 0.012s
I20260709 12:12:22.359899 12350 raft_consensus.cc:359] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:22.360531 12350 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:22.361078 12350 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:22.381736 12350 ts_tablet_manager.cc:1434] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 12:12:22.391981 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-38d626794db04be2b9c36cc76fcf2183 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:22.392484 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-38d626794db04be2b9c36cc76fcf2183 on tablet 38d626794db04be2b9c36cc76fcf2183 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:22.409725 12356 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:22.411221 12356 tablet_bootstrap.cc:492] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:22.412118 12356 ts_tablet_manager.cc:1403] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.071s	user 0.030s	sys 0.004s
I20260709 12:12:22.420761 12356 raft_consensus.cc:359] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:22.421835 12356 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:22.422616 12356 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:22.434410 12356 ts_tablet_manager.cc:1434] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:12:22.436389 11191 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-89068ffedf554d89af62a8b1fe0700c0 received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:22.436902 11191 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-89068ffedf554d89af62a8b1fe0700c0 on tablet 89068ffedf554d89af62a8b1fe0700c0 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:22.487680 11935 raft_consensus.cc:1217] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:22.576836 11935 raft_consensus.cc:1217] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:22.757145 12333 raft_consensus.cc:1064] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:22.760092 12333 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:22.779286 11329 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:22.782353 12332 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:22.830137 11249 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:22.831979 11935 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:22.995697 12300 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:23.017903 12333 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:12:23.092217 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 304690us
I20260709 12:12:23.125813 12368 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.137051 12373 raft_consensus.cc:1064] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:23.139778 12373 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:23.150359 12333 raft_consensus.cc:1064] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:23.158054 11251 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:12:23.173537 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60457.1us
I20260709 12:12:23.189446 11329 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.191985 12362 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:23.203428 12333 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:23.208036 11934 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.211737 11173 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.221410 12360 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:12:23.226475 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 48488.2us
W20260709 12:12:23.231724 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53689.7us
I20260709 12:12:23.236423 11079 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.249084 12378 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:12:23.256098 11933 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.269500 12357 raft_consensus.cc:1064] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:23.283790 11329 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.290206 12357 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:23.341583 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 111729us
I20260709 12:12:23.339144 11328 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.332918 12360 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.400980 11329 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.402522 12368 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:23.422652 11933 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.424908 12386 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:23.424892 11935 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.457062 11932 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.458544 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:23.466107 11255 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.474085 12333 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:12:23.495846 11173 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.509342 12388 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:23.517261 11255 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.544634 12376 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:23.548684 11173 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:23.560653 12388 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:12:23.628288 11076 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.737205 11933 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.763615 11251 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:23.771286 11328 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:23.773501 12373 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:23.775514 12357 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.734342 12368 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.781788 12296 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.783988 11329 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.785979 11251 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.790678 11934 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:23.796904 12378 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:23.804035 11174 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:23.818749 11331 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:12:23.822106 12404 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:12:23.803810 11934 raft_consensus.cc:2955] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.826982 11077 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.845070 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 38d626794db04be2b9c36cc76fcf2183 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:23.870337 11251 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.869477 11933 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:23.875125 11249 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:23.882478 11079 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.901592 11077 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.910396 12386 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:23.910189 11174 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:23.913486 12407 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:23.917984 11934 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.946767 11933 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:23.949154 11251 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:23.952996 12368 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.960911 11174 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:23.978490 12384 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:23.980185 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet 38d626794db04be2b9c36cc76fcf2183 with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:23.980546 12384 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:24.018947 11251 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.031787 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ddf8cca10e1244d8882abd13fa81113a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:24.047269 11933 raft_consensus.cc:2955] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.049844 12411 tablet_replica.cc:333] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:24.050953 12333 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.053321 11076 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:24.055706 11934 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.087980 12411 raft_consensus.cc:2243] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:24.088690 11331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fd765747f9648049b905c70696b206d"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:24.089484 11331 raft_consensus.cc:606] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:24.088953 12411 raft_consensus.cc:2272] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:24.096408 12411 ts_tablet_manager.cc:1916] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:24.113579 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet ddf8cca10e1244d8882abd13fa81113a with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:24.138095 12413 tablet_replica.cc:333] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:24.139439 11329 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.149828 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 89068ffedf554d89af62a8b1fe0700c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:24.154464 11934 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:24.156103 12399 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:24.159308 11079 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.167728 11249 raft_consensus.cc:2955] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.171500 11934 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.167407 12369 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.180858 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 89068ffedf554d89af62a8b1fe0700c0 with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:24.180894 11174 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:24.188277 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a629d78b61074c2cbd007926bd78fa29 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:24.195190 12399 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:24.202914 12413 raft_consensus.cc:2243] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:24.207657 12413 raft_consensus.cc:2272] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:24.210927 12413 ts_tablet_manager.cc:1916] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:24.230511 12411 ts_tablet_manager.cc:1929] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:24.231173 12411 log.cc:1199] T 38d626794db04be2b9c36cc76fcf2183 P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/38d626794db04be2b9c36cc76fcf2183
I20260709 12:12:24.252650 12413 ts_tablet_manager.cc:1929] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:24.260803 12376 raft_consensus.cc:993] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:24.261499 12413 log.cc:1199] T ddf8cca10e1244d8882abd13fa81113a P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/ddf8cca10e1244d8882abd13fa81113a
I20260709 12:12:24.261662 12376 raft_consensus.cc:1081] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:24.262444 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet 38d626794db04be2b9c36cc76fcf2183 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:24.263378 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet ddf8cca10e1244d8882abd13fa81113a (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:24.263579 12413 tablet_replica.cc:333] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:24.257035 11173 raft_consensus.cc:2955] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:24.264693 12413 raft_consensus.cc:2243] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:24.277464 11176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fd765747f9648049b905c70696b206d"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45582
I20260709 12:12:24.278259 11176 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.278270 12413 raft_consensus.cc:2272] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:24.278851 11176 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.278394 11079 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:24.281842 12413 ts_tablet_manager.cc:1916] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:24.287676 11176 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.291947 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:24.292695 11251 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.292914 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet a629d78b61074c2cbd007926bd78fa29 with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:24.292976 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:24.298434 11329 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:24.298864 11329 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:24.307366 11329 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.308946 11176 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:24.305162 11251 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:24.315316 12414 tablet_replica.cc:333] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:24.316658 11329 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.313556 12413 ts_tablet_manager.cc:1929] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:24.327508 12413 log.cc:1199] T 89068ffedf554d89af62a8b1fe0700c0 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/89068ffedf554d89af62a8b1fe0700c0
I20260709 12:12:24.329072 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 89068ffedf554d89af62a8b1fe0700c0 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:24.319908 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:24.331423 12384 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:24.331897 12384 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:24.332779 12384 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.339351 12414 raft_consensus.cc:2243] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:24.339354 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25983a67e23a4c708ae496061b992207"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:24.340361 12414 raft_consensus.cc:2272] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:24.340426 11249 raft_consensus.cc:606] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:24.346099 12414 ts_tablet_manager.cc:1916] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:24.347759 11329 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:24.348820 11328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:24.349426 11328 raft_consensus.cc:606] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:24.360292 11328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73b50321123c433f91c5d0972791cf37"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:24.360950 11328 raft_consensus.cc:606] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:24.363826 11328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:24.364380 11328 raft_consensus.cc:606] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:24.366992 11328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:24.367599 11328 raft_consensus.cc:606] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:24.370929 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:12:24.374195 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:24.380265 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:24.380394 12414 ts_tablet_manager.cc:1929] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:24.380960 12414 log.cc:1199] T a629d78b61074c2cbd007926bd78fa29 P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/a629d78b61074c2cbd007926bd78fa29
I20260709 12:12:24.382838 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet a629d78b61074c2cbd007926bd78fa29 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:24.389379 11079 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:24.397063 12357 raft_consensus.cc:993] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:24.397579 12399 raft_consensus.cc:1081] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:24.399796 11934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52012
I20260709 12:12:24.400314 11934 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.400651 11934 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.407322 11934 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.410449 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:24.410656 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:24.411109 11173 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.411151 11328 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:24.411607 11328 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:24.412317 11328 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.413309 11328 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.421984 11173 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:24.425866 11328 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:24.427196 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:24.428283 12394 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:24.434675 11934 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:24.450400 12394 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:24.451457 12394 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.497430 11079 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:24.511716 12296 raft_consensus.cc:993] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:24.512198 12296 raft_consensus.cc:1081] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:24.519544 11173 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25983a67e23a4c708ae496061b992207"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45566
I20260709 12:12:24.520151 11173 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.520733 11173 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.527985 11173 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.530611 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:24.530812 11173 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:24.531288 11251 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:24.531435 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:24.531652 11251 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:24.532030 11328 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.532531 11251 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.537060 11251 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.539927 11328 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:24.541167 11148 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:24.541960 12384 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:24.542474 12384 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:24.543699 11251 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:24.543390 12384 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.554214 11079 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:24.686981 12369 raft_consensus.cc:993] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:24.687479 12369 raft_consensus.cc:1081] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:24.694617 11173 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45582
I20260709 12:12:24.695199 11173 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.695562 11173 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.702620 11173 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.705343 11173 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:24.705852 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:24.706584 11251 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.707333 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:24.707963 11328 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:24.708279 11328 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:24.709170 11328 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.710682 11328 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.714262 11251 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:24.715340 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:24.716236 12384 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:24.716971 12384 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:24.718010 12384 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:24.721426 11328 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:24.737339 11079 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:24.823061 12399 raft_consensus.cc:993] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:24.823586 12357 raft_consensus.cc:1081] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:24.825343 11934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73b50321123c433f91c5d0972791cf37"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52012
I20260709 12:12:24.825026 12399 raft_consensus.cc:993] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:24.825999 11934 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.826514 11934 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.826481 12357 raft_consensus.cc:1081] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:24.828418 11932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52012
I20260709 12:12:24.828958 11932 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.829342 11932 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.835940 11932 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.836964 11934 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.839524 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:24.839663 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:24.840142 11331 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:24.840343 11249 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.840524 11331 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:24.840979 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:24.841545 11328 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:24.841920 11328 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:24.841399 11331 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.842655 11331 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.842636 11328 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.844337 11328 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.850793 11328 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:24.839524 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:24.851900 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:24.856900 12394 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:24.857064 11331 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:24.857422 12394 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:24.858382 12394 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.858528 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:24.859536 12416 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:24.854626 11249 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:24.855126 11932 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:24.872895 11934 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:24.883232 12416 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:24.855732 11173 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.945880 11249 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:12:24.949329 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 76780.8us
I20260709 12:12:24.950927 12333 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:24.950940 12416 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:24.954108 11249 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:24.956403 12333 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:24.958613 11331 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:24.966316 11173 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:24.964357 12386 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:12:24.992420 11176 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:25.011673 11079 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.025434 12416 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:25.061599 11077 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.078629 11329 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:25.080667 12416 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:25.091180 11328 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:25.092804 12386 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:25.117288 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59532.5us
I20260709 12:12:25.183383 11249 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:25.188328 12431 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:25.222291 11328 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:25.223724 12384 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:25.258492 11173 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:25.259949 12394 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:25.288422 11328 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:25.292411 12420 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:25.383562 11173 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:25.389807 11329 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:25.391033 11249 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:25.391079 12429 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:12:25.394124 11147 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80d10b7c9e4644e9a03edf10767df625. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:25.395915 12431 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:25.411199 12333 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.413700 11328 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.419416 11249 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.430806 11251 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:25.443141 11077 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:12:25.455032 11222 consensus_peers.cc:597] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e26c5b0a79b0482cb39b479148fba412. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:25.455452 11176 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:25.456681 11328 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:25.457561 12374 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:12:25.462810 12362 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:25.476104 11251 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:25.477689 12420 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:25.496263 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:25.504374 12374 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.507550 11173 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.515128 11328 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:25.516903 12422 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:25.530926 11331 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.565492 11079 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.567014 11251 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:25.611666 11328 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:25.613674 12374 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:25.626425 11174 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:25.631125 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a6ec18bb8c54ece99a794e109201789. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:25.633818 12375 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:25.663581 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:25.664330 11176 raft_consensus.cc:606] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:25.672412 12296 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.677830 11331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:25.678501 11331 raft_consensus.cc:606] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:25.681519 11251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:25.682215 11251 raft_consensus.cc:606] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:25.686038 11328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:25.686622 11328 raft_consensus.cc:606] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:25.688001 12399 raft_consensus.cc:993] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:25.687318 11331 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.688491 12357 raft_consensus.cc:1081] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:25.692040 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:25.696504 11932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52012
I20260709 12:12:25.697042 11932 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:25.697391 11932 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:25.705395 11173 raft_consensus.cc:606] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:25.712499 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:25.714242 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:25.715060 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:25.714356 11176 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.725703 11076 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.709374 11932 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:25.736687 11932 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:25.738013 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:25.739216 11328 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:25.739625 11328 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:25.743568 11328 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:25.745692 11174 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:25.746187 11328 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:25.750110 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:25.782936 11329 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:25.788669 11173 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:25.790226 12333 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:25.800912 11251 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:25.784929 11328 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:25.803511 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:25.804519 12422 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:25.805083 12422 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:25.800293 12386 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.806131 12422 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:25.813179 11329 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.829250 11076 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.831430 12431 raft_consensus.cc:993] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:25.832057 12384 raft_consensus.cc:1081] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
W20260709 12:12:25.832892 11147 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 25983a67e23a4c708ae496061b992207. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:25.865463 11934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52016
I20260709 12:12:25.866080 11934 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:25.866443 11934 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:25.879295 11934 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:25.882829 12431 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:25.883893 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:25.887660 11174 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:25.888092 11174 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:25.889149 11174 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:25.891363 11174 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:25.894207 11173 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:25.907541 11934 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:25.907701 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:25.908243 11328 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:25.910900 11255 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:25.912587 11174 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:25.913952 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:25.914844 12422 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:25.915504 12422 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:25.916231 11328 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:25.916402 12422 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:25.926386 11076 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 3, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:25.972275 11076 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.982512 12459 ts_tablet_manager.cc:933] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:25.998543 12459 tablet_copy_client.cc:323] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
I20260709 12:12:26.004091 11191 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet 80d10b7c9e4644e9a03edf10767df625 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:26.004729 11191 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet 80d10b7c9e4644e9a03edf10767df625 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-80d10b7c9e4644e9a03edf10767df625
I20260709 12:12:26.012096 12460 ts_tablet_manager.cc:933] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:26.039994 12296 raft_consensus.cc:993] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:26.040647 12296 raft_consensus.cc:1081] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:26.050242 12399 raft_consensus.cc:993] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:26.050875 12399 raft_consensus.cc:1081] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:26.055071 11932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52014
I20260709 12:12:26.055845 11932 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:26.056416 11932 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.064575 11934 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52012
I20260709 12:12:26.065310 11934 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:26.065737 11934 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.075809 12431 raft_consensus.cc:993] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:26.076535 12431 raft_consensus.cc:1081] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:26.086175 11191 tablet_copy_source_session.cc:215] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:26.092046 12459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80d10b7c9e4644e9a03edf10767df625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:26.120182 12459 tablet_copy_client.cc:806] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:26.121271 12459 tablet_copy_client.cc:670] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:26.162268 11935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52016
I20260709 12:12:26.162940 11935 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:26.163717 11935 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.170631 11935 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.173774 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:26.174439 11176 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:26.174786 11176 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:26.175596 11176 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.176518 11176 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.184082 11176 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:26.186137 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:26.187229 12422 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:26.187800 12422 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:26.188694 11935 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:26.188733 12422 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.198179 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:26.198843 11255 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.208000 11255 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
W20260709 12:12:26.224635 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46656.6us
I20260709 12:12:26.232072 12459 tablet_copy_client.cc:538] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:26.236232 12463 ts_tablet_manager.cc:933] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:26.255244 11076 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:26.262372 11932 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.264541 11932 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:26.267158 12464 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:12:26.245555 12459 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:26.269851 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:26.270582 11174 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.274472 11329 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.276479 12422 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:26.278928 11174 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:26.308992 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:26.310308 12422 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:26.310930 12422 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:26.311864 12422 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:26.331692 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61887.2us
I20260709 12:12:26.345089 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:26.345785 11251 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:26.346191 11251 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:26.346966 11251 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.351229 11251 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:26.366433 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 89639.5us
I20260709 12:12:26.269346 11934 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.379667 11251 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:26.390229 11934 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:26.397313 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:26.397940 11328 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:26.398217 11328 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:26.406421 12463 tablet_copy_client.cc:323] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:26.412377 11328 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.417181 11174 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.423871 12469 ts_tablet_manager.cc:933] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:26.431548 12416 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:12:26.435459 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50128.8us
I20260709 12:12:26.441228 11328 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.442926 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:26.443552 11251 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:26.456012 11331 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.457917 12394 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:26.468726 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 0a6ec18bb8c54ece99a794e109201789 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:26.495046 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 0a6ec18bb8c54ece99a794e109201789 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-0a6ec18bb8c54ece99a794e109201789
I20260709 12:12:26.502991 11328 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:26.495783 11251 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
W20260709 12:12:26.505556 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 109788us
I20260709 12:12:26.493942 11076 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:26.517251 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:26.515031 12460 tablet_copy_client.cc:323] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:26.515563 12469 tablet_copy_client.cc:323] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.193:45115
W20260709 12:12:26.528659 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63469.6us
I20260709 12:12:26.527585 12420 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:26.535068 11173 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.544512 11191 tablet_copy_service.cc:140] P 016411114c6d42039bd04025378dfc98: Received BeginTabletCopySession request for tablet 25983a67e23a4c708ae496061b992207 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:45486)
I20260709 12:12:26.544986 11266 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet e26c5b0a79b0482cb39b479148fba412 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:26.545687 11266 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet e26c5b0a79b0482cb39b479148fba412 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-e26c5b0a79b0482cb39b479148fba412
I20260709 12:12:26.545074 11191 tablet_copy_service.cc:161] P 016411114c6d42039bd04025378dfc98: Beginning new tablet copy session on tablet 25983a67e23a4c708ae496061b992207 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:45486: session id = 786851d4227e4dd89b75ccdea0b1f03b-25983a67e23a4c708ae496061b992207
I20260709 12:12:26.546911 12420 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:26.550541 12420 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:26.554806 12459 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:26.554772 12422 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:26.556586 12459 tablet_bootstrap.cc:492] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:26.557381 12459 ts_tablet_manager.cc:1403] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.312s	user 0.022s	sys 0.007s
I20260709 12:12:26.588030 12459 raft_consensus.cc:359] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:26.589797 12459 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:26.590490 12459 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:26.592851 12459 ts_tablet_manager.cc:1434] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.035s	user 0.001s	sys 0.003s
I20260709 12:12:26.596392 11192 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-80d10b7c9e4644e9a03edf10767df625 received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:26.604751 11192 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-80d10b7c9e4644e9a03edf10767df625 on tablet 80d10b7c9e4644e9a03edf10767df625 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:26.593569 11266 tablet_copy_source_session.cc:215] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:26.608428 11267 tablet_copy_source_session.cc:215] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:26.611878 12460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26c5b0a79b0482cb39b479148fba412. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:26.611765 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 96548.3us
I20260709 12:12:26.596995 11191 tablet_copy_source_session.cc:215] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:26.626319 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25983a67e23a4c708ae496061b992207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:26.627686 12463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a6ec18bb8c54ece99a794e109201789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:26.630769 11076 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:26.681738 12469 tablet_copy_client.cc:806] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:26.682466 12469 tablet_copy_client.cc:670] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:26.691722 12469 tablet_copy_client.cc:538] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:26.697885 12463 tablet_copy_client.cc:806] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:26.698521 12463 tablet_copy_client.cc:670] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:26.702081 12463 tablet_copy_client.cc:538] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:26.704278 12469 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:26.706975 12460 tablet_copy_client.cc:806] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:26.711508 12463 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:26.714165 12460 tablet_copy_client.cc:670] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:26.718032 11251 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.718211 12460 tablet_copy_client.cc:538] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:26.723295 12420 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:26.744707 12460 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:26.770076 11179 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.783269 11179 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.779143 12420 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:12:26.785305 12420 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:26.794431 12463 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:26.795495 12463 tablet_bootstrap.cc:492] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:26.803714 12463 ts_tablet_manager.cc:1403] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.011s
I20260709 12:12:26.816067 11249 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:26.817804 12420 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:26.828824 12463 raft_consensus.cc:359] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:26.829748 12463 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:26.830476 12463 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:26.832979 12463 ts_tablet_manager.cc:1434] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 12:12:26.839699 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-0a6ec18bb8c54ece99a794e109201789 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:26.840186 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-0a6ec18bb8c54ece99a794e109201789 on tablet 0a6ec18bb8c54ece99a794e109201789 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:26.928632 11935 raft_consensus.cc:1217] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:26.946722 11931 raft_consensus.cc:1217] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:12:26.953872 12469 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 12:12:26.954588 12469 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:26.956547 12469 tablet_bootstrap.cc:492] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:26.957350 12469 ts_tablet_manager.cc:1403] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.253s	user 0.027s	sys 0.012s
I20260709 12:12:26.966318 12460 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:26.967346 12460 tablet_bootstrap.cc:492] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:26.982477 12460 ts_tablet_manager.cc:1403] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.238s	user 0.023s	sys 0.008s
I20260709 12:12:26.960748 12469 raft_consensus.cc:359] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:26.986227 12469 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:26.987046 12469 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:26.989741 12469 ts_tablet_manager.cc:1434] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.003s
I20260709 12:12:26.997912 11191 tablet_copy_service.cc:342] P 016411114c6d42039bd04025378dfc98: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-25983a67e23a4c708ae496061b992207 received from {username='slave'} at 127.0.0.1:45486
I20260709 12:12:26.998420 11191 tablet_copy_service.cc:434] P 016411114c6d42039bd04025378dfc98: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-25983a67e23a4c708ae496061b992207 on tablet 25983a67e23a4c708ae496061b992207 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:27.007893 12460 raft_consensus.cc:359] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:27.008811 12460 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:27.009557 12460 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:27.019387 12460 ts_tablet_manager.cc:1434] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
I20260709 12:12:27.021144 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-e26c5b0a79b0482cb39b479148fba412 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:27.021574 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-e26c5b0a79b0482cb39b479148fba412 on tablet e26c5b0a79b0482cb39b479148fba412 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:27.027153 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:27.042704 11249 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:27.044443 12420 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:27.054647 11328 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:27.056121 12422 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:27.143033 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:27.143736 11249 raft_consensus.cc:606] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:27.174400 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:27.175036 11249 raft_consensus.cc:606] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:27.178233 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:27.178903 11249 raft_consensus.cc:606] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:27.182152 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:27.182791 11249 raft_consensus.cc:606] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:27.192432 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:27.195207 11249 raft_consensus.cc:606] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:27.215813 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:27.220111 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:27.221410 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:27.227322 12362 raft_consensus.cc:993] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:27.227829 12362 raft_consensus.cc:1081] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:27.229348 11179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45566
I20260709 12:12:27.229921 11179 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:27.230252 11179 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.239929 11179 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.248230 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:27.248803 11249 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:27.249112 11249 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:27.249743 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:27.249848 11249 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.250479 11328 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.253034 11249 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.260934 11328 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:27.268175 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:27.264850 11249 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:27.272154 12431 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:27.272650 12431 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:27.273577 12431 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.262641 11179 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:27.276137 11931 raft_consensus.cc:1217] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:27.300786 11077 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:27.369241 12470 raft_consensus.cc:993] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:27.369747 12374 raft_consensus.cc:1081] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:27.371142 11330 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39342
I20260709 12:12:27.371567 11330 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:27.371831 11330 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.377204 11330 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.378886 11330 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:27.381151 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:27.381783 11249 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:27.381883 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:27.382102 11249 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:27.382512 11935 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.382948 11249 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.383965 11249 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.384773 11931 raft_consensus.cc:1217] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:12:27.391635 12470 raft_consensus.cc:1064] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:27.390748 11249 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:27.392812 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:27.394718 12399 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:27.394308 12470 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.395210 12399 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:27.396168 12399 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.413743 11935 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:27.485316 11935 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:27.485316 11330 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:27.487897 11179 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:27.490386 12374 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:27.491281 12455 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:27.495456 11077 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:27.500823 12470 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:27.523556 12455 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.527825 11330 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.531121 11935 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.547191 11174 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.547797 11077 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.556891 12431 raft_consensus.cc:1064] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:27.560547 12431 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.596751 11330 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:27.597397 12455 raft_consensus.cc:993] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:27.597965 12455 raft_consensus.cc:1081] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:27.604373 11935 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:27.602751 12478 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:12:27.607075 12333 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:27.611642 11249 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:27.611722 11179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45566
I20260709 12:12:27.612450 11179 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:27.612906 11179 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.620414 12333 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:27.623288 11251 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.628580 11179 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.630559 11179 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:27.636422 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:27.636924 11931 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:27.636438 11179 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:27.638603 12362 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:27.640378 12362 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:27.637017 11251 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:27.642359 11251 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:12:27.645969 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46978.8us
I20260709 12:12:27.647450 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:27.648008 11330 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:27.650985 12504 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:12:27.653259 11251 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.664223 12374 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.665339 11174 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.672672 11251 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.679522 11330 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:27.686058 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:27.686932 12478 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:27.687891 12478 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:27.689687 12478 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.691612 12296 raft_consensus.cc:993] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:27.692045 12296 raft_consensus.cc:1081] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:27.693910 11179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45566
I20260709 12:12:27.694433 11179 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:27.694778 11179 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.695475 12470 raft_consensus.cc:993] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:27.696165 12455 raft_consensus.cc:1081] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:27.698350 11329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39342
I20260709 12:12:27.699028 11329 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:27.699433 11329 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.686352 11931 raft_consensus.cc:2955] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.708209 12333 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.719851 11179 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.721573 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a6ec18bb8c54ece99a794e109201789 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:27.725404 11249 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.727900 11179 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:27.730717 11251 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:27.734305 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:27.734926 11255 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:27.735208 11255 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:27.735946 11255 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.737082 11255 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.737732 11329 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.740805 11079 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.742709 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:27.743256 11330 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.744593 11329 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:27.750331 11328 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.755790 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a6ec18bb8c54ece99a794e109201789 with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:27.760437 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:27.761718 11251 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:27.762027 11251 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:27.766033 11251 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.776816 11330 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:27.777498 11331 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:27.777904 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:27.778523 11931 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.785250 11250 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:27.785728 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:27.786959 12431 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:27.787442 11251 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:27.791618 12506 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:27.787477 12431 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:27.794421 11255 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:27.803766 11330 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:27.805282 12506 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:27.794173 12431 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:27.810499 11077 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:27.823892 12505 raft_consensus.cc:1064] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:27.825055 11931 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:27.826874 12505 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.828737 11935 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:12:27.843493 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82450.3us
I20260709 12:12:27.829521 11251 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:27.845453 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:27.847854 12511 tablet_replica.cc:333] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:27.849045 12511 raft_consensus.cc:2243] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:27.849551 12357 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:27.852950 12357 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:27.853906 12357 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:27.857371 11328 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:12:27.864636 12333 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:27.866148 11934 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:12:27.867913 11255 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:12:27.875293 12506 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:27.877374 12506 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:27.884069 11079 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:27.888410 12511 raft_consensus.cc:2272] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:27.922149 12511 ts_tablet_manager.cc:1916] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:27.933028 12455 raft_consensus.cc:1064] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:27.936383 12455 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:27.981194 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54325.3us
I20260709 12:12:27.981885 11173 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.012967 11251 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:28.028102 12333 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.028337 12357 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:12:28.037252 12511 ts_tablet_manager.cc:1929] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:28.039036 11328 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.040395 12511 log.cc:1199] T 0a6ec18bb8c54ece99a794e109201789 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/0a6ec18bb8c54ece99a794e109201789
I20260709 12:12:28.042695 11932 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.043144 11078 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.046031 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 0a6ec18bb8c54ece99a794e109201789 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:28.048434 11931 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:28.051119 11330 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:28.052733 12478 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:28.055573 12505 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:28.057008 11330 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:28.058739 12374 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:28.070725 11173 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:28.074594 12374 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:28.076416 11935 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.086699 11931 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:12:28.089654 11932 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:28.065270 12478 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.089243 11250 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.088168 12374 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:28.108147 11077 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.110585 12470 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.112195 12357 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:28.124378 11330 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.138525 11076 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.149866 11179 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.173774 11251 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:28.176100 12333 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:28.183346 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59823.9us
I20260709 12:12:28.188689 11174 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.194792 11255 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:28.195437 11078 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.198673 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 80d10b7c9e4644e9a03edf10767df625 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:28.201192 11932 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:28.204639 12333 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:28.210963 12506 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:12:28.216133 12525 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.219728 11932 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.230260 11935 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.236485 11249 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:28.237474 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 25983a67e23a4c708ae496061b992207 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:28.238241 12506 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:28.245466 11256 raft_consensus.cc:2955] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:12:28.260661 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 208375us
I20260709 12:12:28.262362 11330 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:28.268862 11079 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New cstate: current_term: 1 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.279906 11330 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:28.285425 12506 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:28.285162 12505 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:28.289430 11330 raft_consensus.cc:2955] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.295982 11933 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:28.297940 11245 tablet_service.cc:1558] Processing DeleteTablet for tablet 80d10b7c9e4644e9a03edf10767df625 with delete_type TABLET_DATA_TOMBSTONED (TS fecfc03de52a4308bc9b045cad5aac1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:28.301563 12512 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:28.316543 11256 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:28.318686 12357 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:28.316406 11078 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 4 to 5, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.361231 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:28.386507 12538 tablet_replica.cc:333] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:12:28.404057 12538 raft_consensus.cc:2243] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:28.404989 12538 raft_consensus.cc:2272] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:28.409518 12538 ts_tablet_manager.cc:1916] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:28.414973 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 25983a67e23a4c708ae496061b992207 with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:39336
I20260709 12:12:28.415944 12511 tablet_replica.cc:333] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:12:28.418588 12511 raft_consensus.cc:2243] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:28.421417 12511 raft_consensus.cc:2272] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:28.427390 12511 ts_tablet_manager.cc:1916] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:28.469528 11257 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.479364 11933 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:28.481216 12470 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:28.492326 12362 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.494622 12538 ts_tablet_manager.cc:1929] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:28.495126 12538 log.cc:1199] T 80d10b7c9e4644e9a03edf10767df625 P fecfc03de52a4308bc9b045cad5aac1e: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/wals/80d10b7c9e4644e9a03edf10767df625
I20260709 12:12:28.495213 11935 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.496964 12511 ts_tablet_manager.cc:1929] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:12:28.497427 12511 log.cc:1199] T 25983a67e23a4c708ae496061b992207 P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/25983a67e23a4c708ae496061b992207
I20260709 12:12:28.499774 11065 catalog_manager.cc:5028] TS fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): tablet 80d10b7c9e4644e9a03edf10767df625 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:28.500591 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 25983a67e23a4c708ae496061b992207 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:28.505668 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e26c5b0a79b0482cb39b479148fba412 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:28.505599 11331 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:12:28.510480 12455 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:28.516548 11331 raft_consensus.cc:2955] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:28.514878 11079 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index 3 to 4, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:28.531955 11169 tablet_service.cc:1558] Processing DeleteTablet for tablet e26c5b0a79b0482cb39b479148fba412 with delete_type TABLET_DATA_TOMBSTONED (TS 016411114c6d42039bd04025378dfc98 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:28.550751 12541 tablet_replica.cc:333] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:12:28.553527 11256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fb796253655415ba47e989105d7c807"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:28.554399 11256 raft_consensus.cc:606] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:28.554540 12541 raft_consensus.cc:2243] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:28.556448 12541 raft_consensus.cc:2272] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:28.557308 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:28.557942 11174 raft_consensus.cc:606] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:28.560240 12541 ts_tablet_manager.cc:1916] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:28.566671 11331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:28.567299 11331 raft_consensus.cc:606] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:28.569799 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:28.570305 11174 raft_consensus.cc:606] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:28.572011 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:28.574129 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:28.575212 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:28.581990 12541 ts_tablet_manager.cc:1929] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:12:28.582517 12541 log.cc:1199] T e26c5b0a79b0482cb39b479148fba412 P 016411114c6d42039bd04025378dfc98: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/wals/e26c5b0a79b0482cb39b479148fba412
I20260709 12:12:28.586196 11065 catalog_manager.cc:5028] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): tablet e26c5b0a79b0482cb39b479148fba412 (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:12:28.648125 12455 raft_consensus.cc:993] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:28.648584 12470 raft_consensus.cc:1081] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:28.650081 11331 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fb796253655415ba47e989105d7c807"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39342
I20260709 12:12:28.650576 11331 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.650858 11331 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.657620 11331 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.659345 11331 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:28.662940 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:28.663473 11256 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:28.663749 11256 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:28.664453 11256 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.667760 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:28.668305 11935 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.670240 11256 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.687038 11256 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:28.688798 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:28.689760 12512 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:28.690296 12512 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:28.689069 11935 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:28.691166 12512 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.712741 11076 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.773739 12431 raft_consensus.cc:993] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:28.774214 12506 raft_consensus.cc:1081] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:28.775591 11935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52016
I20260709 12:12:28.776082 11935 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.776319 11935 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.780576 11935 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.782385 11935 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:28.783563 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:28.783694 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:28.784292 11174 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:28.784730 11174 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:28.784353 11257 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.785548 11174 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.787935 11174 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.795045 11257 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:28.798354 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:28.799507 12480 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:28.801465 12480 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:28.802516 12480 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.807976 11174 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:28.818192 11079 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:28.838593 12512 raft_consensus.cc:993] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:28.839219 12479 raft_consensus.cc:1081] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:28.840849 11174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45582
I20260709 12:12:28.841395 11174 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.841730 11174 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.846925 11174 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.848675 11174 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:28.849323 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:28.849675 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:28.849928 11331 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:28.850236 11935 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.850266 11331 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:28.851166 11331 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.852057 11331 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.857111 11935 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:28.857853 11331 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:28.858263 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:28.859099 12478 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:28.859567 12478 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:28.860428 12478 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:28.869845 11079 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.892652 12478 raft_consensus.cc:993] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:28.893123 12478 raft_consensus.cc:1081] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:28.894587 11256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:12:28.895026 11256 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.895331 11256 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.899996 11256 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:28.901885 11256 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:28.902071 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:28.902654 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:28.902778 11174 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:28.903247 11174 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:28.903263 11331 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.903963 11174 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:28.904986 11174 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.909260 11331 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:28.910486 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:28.911257 12470 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:28.911346 11174 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:28.911765 12470 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:28.912617 12470 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:28.922160 11079 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 1 to 2, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 2 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:29.238582 11173 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:29.239780 12532 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:29.245110 11257 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:29.249859 12480 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:29.287113 11257 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:29.299412 12479 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:29.306260 11935 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:29.315249 12512 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:29.382830 11174 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:29.383965 12455 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:29.393652 11328 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:29.396708 12296 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:29.412954 11935 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:29.415517 11331 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:29.415484 12505 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:29.416971 12505 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:29.605947 11331 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:29.611635 11249 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:12:29.611779 11296 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d225f8a873c4abe992018eb9353664f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:29.611836 11173 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:29.612931 12558 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:29.613772 12557 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:29.616385 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:29.621519 12557 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.624712 11173 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.627132 11249 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.638548 11331 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:29.646270 11079 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:29.657409 11296 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a634acd9ff543cca551c8a28e72d6c4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:29.659497 11249 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:29.661511 12512 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:29.661659 11173 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:29.668993 12512 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:29.678340 12558 raft_consensus.cc:2955] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.689076 11249 raft_consensus.cc:2955] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.709142 11174 raft_consensus.cc:2955] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.734251 11076 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.737793 11256 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:29.775843 11328 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:29.776259 11331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:29.776850 11331 raft_consensus.cc:606] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:29.777595 12296 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:29.778589 11222 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5df07764623640ce8ec29ad695812b7c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:29.779925 11173 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:29.780007 11256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:29.780582 11256 raft_consensus.cc:606] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:29.781531 12470 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:29.783421 11328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:29.784166 11328 raft_consensus.cc:606] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:29.787246 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:29.787892 11174 raft_consensus.cc:606] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:29.789330 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:29.790973 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:29.791647 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:29.797724 12296 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.799639 11328 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.810570 11173 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.822381 11078 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.857981 11256 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:29.865490 11331 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:29.866989 12455 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:12:29.868798 11222 consensus_peers.cc:597] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86c5246d860d4d559ea665671cc46b5c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:29.871011 11173 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:12:29.872244 12470 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:29.875869 12455 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.880702 11331 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.881767 11173 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.897861 11079 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 2, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 1 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:12:29.937179 12512 raft_consensus.cc:993] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:29.937667 12512 raft_consensus.cc:1081] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:29.941145 11173 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45582
I20260709 12:12:29.941716 11173 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:29.942054 11173 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:29.952142 11173 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:29.953851 11173 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:29.954496 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:29.954754 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:29.955173 11256 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:29.955312 11328 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:29.955718 11328 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:29.956470 11328 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:29.957404 11328 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:29.961822 11328 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:29.962632 11256 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:29.963801 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:29.964673 12505 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:29.965216 12505 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:29.966786 12505 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:29.981738 11079 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:30.037057 12455 raft_consensus.cc:993] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:30.037523 12470 raft_consensus.cc:1081] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:30.039011 11328 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39342
I20260709 12:12:30.039546 11328 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:30.039875 11328 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:30.045468 11328 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:30.050586 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:30.050886 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:30.051191 11932 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:30.060623 12505 raft_consensus.cc:993] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:30.061163 12505 raft_consensus.cc:1081] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:30.063076 11249 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:12:30.063630 11249 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:30.063297 11932 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:30.064160 11249 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:30.070065 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:30.070907 12558 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:30.071406 12558 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:30.072273 12558 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:30.078712 11249 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:30.080736 11249 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:30.063663 11256 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:30.081492 11256 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:30.081714 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:30.082335 11173 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:30.082293 11256 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:30.083459 11256 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:30.083377 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:30.085222 11331 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:30.085723 11328 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:30.082752 11173 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:30.090241 11173 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:30.091668 11173 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:30.095652 11256 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:30.102993 11331 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:30.105913 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:30.106639 12470 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:30.107775 12470 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:30.108698 12470 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:30.114642 11173 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:30.119501 12581 ts_tablet_manager.cc:933] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:30.208050 12581 tablet_copy_client.cc:323] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:31.024541 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:12:31.060935 11146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.019s	sys 0.014s
W20260709 12:12:31.079317 11146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.020s	sys 0.014s
W20260709 12:12:31.096210 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.030s	sys 0.038s
W20260709 12:12:31.096724 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.030s	sys 0.038s
I20260709 12:12:31.110901 12296 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.111411 12362 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.111511 12296 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.111961 12362 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.115887 12362 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.117880 12296 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.193576 12296 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:31.195508 12296 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.192719 12455 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:31.283306 12296 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.201925 12455 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.264204 12588 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:31.382681 12362 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.383560 12362 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:31.412467 12362 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:31.413030 12362 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.412606 12588 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:31.415441 12588 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:31.426502 12455 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.427364 12455 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.428634 12455 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:31.263178 12586 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.472944 12586 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.476004 12586 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.413476 12362 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.513388 12362 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.513917 12588 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:31.514828 12588 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:31.517836 12588 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:31.568477 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.006s	sys 0.036s
I20260709 12:12:31.569559 12594 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:31.570399 12594 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:31.549889 12296 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.577184 12296 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.579972 12296 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.632483 12362 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.633246 12362 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:31.635916 12362 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:31.636497 12362 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:31.682662 12455 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:31.569070 12588 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:31.703742 12588 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:31.706928 12588 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:31.573565 12594 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:31.794564 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.170s	user 0.005s	sys 0.005s
W20260709 12:12:31.795535 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.171s	user 0.006s	sys 0.005s
W20260709 12:12:31.799840 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.012s	sys 0.023s
W20260709 12:12:31.800402 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.012s	sys 0.024s
I20260709 12:12:31.861138 12595 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.862690 12595 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.872138 12595 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:31.569101 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.007s	sys 0.037s
I20260709 12:12:31.950006 12595 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:31.951290 12595 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:31.954592 12595 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:32.177613 12595 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:32.178898 12595 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.181986 12595 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.446075 12296 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.637055 12362 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:32.447196 12455 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:32.468039 12586 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:32.497298 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.002s	sys 0.019s
W20260709 12:12:32.498080 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.002s	sys 0.019s
I20260709 12:12:32.500614 12594 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:32.500403 12455 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.501490 12594 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.504281 12594 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.505264 12296 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:32.506603 12594 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:32.507622 12594 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:32.508018 12296 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:32.504297 12455 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.496713 12586 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.513800 12362 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:32.515039 12455 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:32.515903 12455 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.516359 12586 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.513309 12588 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:32.518942 12588 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.521235 12455 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.522404 12586 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:32.523763 11146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.414s	user 0.000s	sys 0.030s
W20260709 12:12:32.524226 11146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.426s	user 0.000s	sys 0.030s
I20260709 12:12:32.524058 12586 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:32.527266 12588 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.529055 12584 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:32.529259 12593 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:12:32.529908 11145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.396s	user 0.003s	sys 0.016s
W20260709 12:12:32.532619 11145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.432s	user 0.003s	sys 0.017s
I20260709 12:12:32.530160 12584 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:32.544459 12593 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.545944 12584 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:32.546998 12296 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:32.547088 12593 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.543038 12588 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:32.547844 12296 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:32.548745 12588 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.550760 12296 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:32.551354 12588 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.551749 12296 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:32.528653 12455 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:32.553231 12296 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.554713 12595 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:32.555450 12593 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:32.556931 12595 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:32.557410 12593 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.559509 12595 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:32.560667 12593 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.561702 12591 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:32.562676 12591 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.064579 12296 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:33.068872 11898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.005s	sys 0.006s
I20260709 12:12:32.552052 12588 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:33.070701 11898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.007s	sys 0.006s
I20260709 12:12:33.071249 12455 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:32.565614 12591 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:32.593396 12584 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.065402 12594 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:33.122193 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.013s	sys 0.044s
W20260709 12:12:33.122669 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.014s	sys 0.044s
I20260709 12:12:33.069344 12589 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.112607 12595 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.069106 12505 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.067330 12593 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.121274 12591 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.138069 12600 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.151285 12505 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:33.160722 11223 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.578s (SENT) [suppressed 2 similar messages]
I20260709 12:12:33.139258 12600 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:33.176816 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.035s	sys 0.077s
W20260709 12:12:33.177271 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.036s	sys 0.077s
I20260709 12:12:33.177623 12455 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.181391 12591 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.183764 12591 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.180203 12584 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.186949 12584 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.140247 12588 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:33.189723 12589 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.191855 12589 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.194530 12588 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.195115 12601 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.201654 12602 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.202517 12602 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:33.206658 12602 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.209254 12591 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.212600 12600 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.212970 12591 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:33.215582 12591 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.216053 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.217761 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.201086 12595 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.218191 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.219528 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.220098 11179 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.215590 12600 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.224099 12600 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.201264 12593 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.225129 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.226495 12600 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.226925 12593 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.227053 12600 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.227494 11180 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.227908 12600 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.229311 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.237684 12605 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.238549 12600 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.217329 11175 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.242518 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.243408 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.238476 12605 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.240449 12602 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.248708 12602 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.244688 12593 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.251647 12609 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.252530 12609 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.238313 12601 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.252434 12593 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:33.273818 11174 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.250252 12584 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.284719 12600 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.294401 12589 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.294945 12600 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.295327 12589 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.297228 12600 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.285990 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.287956 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.297902 12591 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.302821 12593 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.302968 12591 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.303571 12607 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.279994 12605 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.176857 12505 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.304363 12607 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.301251 12598 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.306883 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.294458 12604 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:33.247676 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87215us
I20260709 12:12:33.308508 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.310300 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.310833 11177 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.311878 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.312198 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.311910 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.314472 11179 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.310853 12598 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.297621 12589 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.311283 12604 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.307533 12591 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.320173 12598 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.319931 12584 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.320377 12600 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.323019 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.333415 11175 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:33.323098 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.323771 12604 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.335081 11179 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.329032 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.329421 12601 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.330960 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.336198 11174 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.342952 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.347754 12604 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.344770 12600 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.348444 12604 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.350816 12600 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.350960 12604 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.354413 11927 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:33.356029 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.357290 12588 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.357966 12588 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.360081 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1382) took 3184 ms. Trace:
I20260709 12:12:33.360535 12588 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.360628 12609 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.360558 11256 rpcz_store.cc:276] 0709 12:12:30.170537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.352818 (+3182281us) service_pool.cc:224] Handling call
0709 12:12:33.353366 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.353572 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:33.353608 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:33.353676 (+    68us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:33.353713 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.353767 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.354558 (+   791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.365589 12611 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.365782 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.366371 12611 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.369248 12611 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.399679 11181 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:33.400960 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.405984 12607 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.392606 12591 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.419462 12606 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.418380 11934 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:33.428457 12616 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.429004 12616 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.430891 12598 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.431082 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.431659 12598 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.432044 11176 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:33.430894 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.433276 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.433929 12591 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.434607 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.435448 11174 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.436358 12591 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.440117 12602 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.453728 12588 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.454623 12588 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.458469 12588 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.459301 12296 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.459250 12588 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.460232 12296 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.460238 12588 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.462844 12594 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.463671 12594 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.464147 12588 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.464927 12588 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.465679 12588 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.443889 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:33.420233 12606 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.475814 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.478091 11928 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:33.479947 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.480852 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.486713 12619 ts_tablet_manager.cc:933] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.490204 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.490437 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:33.492347 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 48557.8us
I20260709 12:12:33.491616 11934 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:33.496243 12612 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.498895 12588 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.499071 12615 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.506420 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:33.497181 12612 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.512504 12612 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.513168 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.515085 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.501451 12602 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.516522 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.515765 11932 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:33.501773 12610 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.526005 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.526786 11250 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:33.528426 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.519204 12602 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.529608 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.520356 12598 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.538806 12622 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.522210 12591 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.550751 12591 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.550885 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.551293 12591 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.551555 11934 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:33.532143 12612 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.550056 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.516016 12615 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.559387 12615 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.540953 12602 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.550885 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.563818 11255 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:33.564858 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.553469 12594 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.571265 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.539635 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.567651 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.573824 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.570528 12615 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.592730 12615 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.593389 12296 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.594837 12595 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.560650 12610 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:33.605424 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68669.8us
I20260709 12:12:33.604604 12619 tablet_copy_client.cc:323] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:33.606735 12594 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.606958 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.607297 12594 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.607671 11251 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:33.607908 12594 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.595324 12616 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.607771 12610 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.610152 12618 ts_tablet_manager.cc:933] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.604982 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.615736 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.618579 11251 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:33.605191 11928 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:33.574785 12612 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.641649 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.646883 12626 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.647735 12626 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.650622 12626 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.658290 12613 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.655092 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:33.576820 12622 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.669451 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.670452 11931 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:33.671655 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.668406 12598 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.679716 12598 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.680154 12598 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.671691 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.687427 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.686543 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.688756 12607 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:33.689145 12607 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.689494 12607 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:33.689821 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1412) took 3522 ms. Trace:
I20260709 12:12:33.693291 11173 rpcz_store.cc:276] 0709 12:12:30.167064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.265649 (+3098585us) service_pool.cc:224] Handling call
0709 12:12:33.266106 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.284642 (+ 18536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:33.284684 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:33.284753 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:33.284790 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.689425 (+404635us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b54002e3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:33.689468 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.689604 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36161280}
I20260709 12:12:33.695824 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.696467 12624 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.671892 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.697314 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.701198 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.697173 12624 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.702633 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.659106 12613 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.687675 12615 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.672663 12609 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.709506 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.718616 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.703871 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.720220 11257 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:33.744745 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.753170 12606 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.749962 11249 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:33.754308 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.754993 11257 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:33.757989 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.760198 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.714082 11252 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:33.762878 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.764287 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.765407 12455 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.766575 12455 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.766959 12455 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.771258 12633 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.774816 11255 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:33.775421 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.778337 12613 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.778815 12615 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.779250 12613 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.779677 12615 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.774085 12618 tablet_copy_client.cc:323] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:12:33.755887 12611 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:33.785213 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.780246 12613 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.805176 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.787189 12612 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.814039 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:33.791059 12611 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.757223 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 86c5246d860d4d559ea665671cc46b5c from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:33.807096 11253 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:33.821106 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.786552 12633 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.809093 12610 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:33.787673 12609 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.786568 12627 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.811087 12586 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.819326 12611 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:33.831400 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.819574 12612 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.828750 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.813249 12622 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.838349 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.838317 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:33.830485 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 86c5246d860d4d559ea665671cc46b5c from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-86c5246d860d4d559ea665671cc46b5c
I20260709 12:12:33.827137 12610 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:33.845126 12617 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.845697 12617 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.845199 12632 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.846294 12617 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.846573 12632 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.850003 12632 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.851665 12506 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:33.852265 12506 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.852689 12506 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:33.841454 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.837694 12593 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.865182 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.872655 12296 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.873148 12296 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.873519 12296 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.875617 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.854713 12610 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:33.853972 12615 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.833132 12607 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.857489 12622 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.879787 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:33.866185 12593 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.879081 12632 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.892133 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.895455 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:33.900568 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62362.8us
I20260709 12:12:33.901904 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.864590 12612 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.903169 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.885967 12621 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.865712 12627 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.891499 12593 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.927706 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:33.928174 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.906266 12505 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.941840 12505 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.942358 12505 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.939105 12607 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:33.931964 12632 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.940097 12613 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.914968 12595 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.950430 12595 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.950894 12595 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.929879 12625 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.941195 12627 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.912901 12586 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.913592 12588 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:33.947862 12633 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:33.948065 12635 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.948629 12641 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:33.963768 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.963872 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.897083 12622 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.906803 12630 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.955917 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.956595 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.957115 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 4a634acd9ff543cca551c8a28e72d6c4 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:33.957442 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:33.960258 12455 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.960904 12594 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.948710 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:33.966442 12640 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.967190 12632 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:33.967077 12591 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:33.973747 12629 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.974483 12636 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.960376 12586 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.963244 12588 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:33.962662 12625 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.981379 12633 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:33.967873 12641 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:33.956424 12639 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:12:33.992446 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43832.1us
I20260709 12:12:33.988011 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:33.998605 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.008455 12634 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.972021 12596 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:34.000712 12455 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.009554 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:34.002889 11254 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:34.003070 12628 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:33.959743 12621 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.991536 12609 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.010695 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.004103 12588 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:33.991272 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 4a634acd9ff543cca551c8a28e72d6c4 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-4a634acd9ff543cca551c8a28e72d6c4
I20260709 12:12:34.002255 12586 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:34.017395 12455 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:34.013886 12296 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.031227 12455 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:34.032688 12455 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:34.006824 12641 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.034806 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.035101 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.014632 12616 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:34.035336 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.015228 12633 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:34.040050 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.041226 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.042419 12632 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:34.003499 12635 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.028914 12591 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.011718 12634 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:33.983682 12630 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.035321 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:34.042997 12632 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.042642 12628 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.041270 12633 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.151952 12296 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.152503 12620 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:34.146237 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.162493 11328 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.163982 12296 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:34.164373 12296 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:34.164692 12296 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:34.164837 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 154220us
I20260709 12:12:34.165998 11326 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.018419 12640 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.151533 12622 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.038592 12609 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:34.160655 11931 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:34.184437 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002eaf40 after lost signal to thread 10014
I20260709 12:12:34.049891 12629 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.185851 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.028518 12636 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.197904 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.079663 12602 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.196426 12594 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.212286 12621 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.212595 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.213274 11329 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.037178 12639 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:34.220526 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002483e0 after lost signal to thread 11113
I20260709 12:12:34.220856 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.221514 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.226722 11329 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.200774 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.226814 11328 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.224140 12455 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.218550 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.205950 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.227681 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.230166 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.231410 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.232550 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.201198 12610 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.208002 12609 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.270627 12455 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.225988 11926 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:34.282126 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.282308 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.312144 11324 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.313333 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.315081 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.315879 11324 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:34.334136 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.334955 11324 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.359948 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.394109 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.270143 12625 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.233117 11322 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.448139 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.283267 12584 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.352761 11325 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
I20260709 12:12:34.312994 11078 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:34.486361 12506 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.014s
I20260709 12:12:34.486891 12506 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.489010 11931 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:34.489746 11173 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:34.494860 12640 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:34.496912 12598 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.816s	user 0.010s	sys 0.000s
W20260709 12:12:34.498366 12617 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.651s	user 0.014s	sys 0.011s
I20260709 12:12:34.502931 12296 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.498739 12617 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.514613 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.522358 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.523188 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.507716 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.525465 12296 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.509407 11252 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.527228 12629 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.527967 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.528189 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.528913 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.529366 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.512077 12635 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.530015 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.529235 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:34.531545 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002ca280 after lost signal to thread 11206
I20260709 12:12:34.531515 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.529543 11254 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:34.497356 12598 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.508409 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.513345 12610 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.510751 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.539944 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.523799 11341 tablet_copy_source_session.cc:215] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:34.541271 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:34.537242 11956 debug-util.cc:398] Leaking SignalData structure 0x7b080029ffc0 after lost signal to thread 11281
I20260709 12:12:34.545723 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.539028 12630 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:34.563760 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08003567e0 after lost signal to thread 11355
I20260709 12:12:34.563731 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.571439 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.572600 11329 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:34.559762 11931 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:34.560806 12506 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.580446 11267 tablet_copy_source_session.cc:215] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:34.580602 12598 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.581961 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.583341 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.584455 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.584292 12586 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.586016 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.559698 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.575296 12617 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:34.593220 12595 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.642s	user 0.015s	sys 0.006s
I20260709 12:12:34.572134 11175 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:34.535277 12643 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.354s
I20260709 12:12:34.576716 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.586373 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.584604 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:34.573516 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.019s	sys 0.037s
I20260709 12:12:34.610160 11256 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.585541 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:34.650741 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.022s	sys 0.040s
W20260709 12:12:34.655869 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 623611us
W20260709 12:12:34.593417 12611 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.004s
I20260709 12:12:34.656709 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:34.661731 11329 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
W20260709 12:12:34.650919 11956 debug-util.cc:398] Leaking SignalData structure 0x7b0800354080 after lost signal to thread 11959
I20260709 12:12:34.657810 12647 ts_tablet_manager.cc:933] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.658321 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.658390 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:34.645838 12593 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.000s
I20260709 12:12:34.659366 11931 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.630240 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.568960 12636 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:34.662458 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 492685us
W20260709 12:12:34.663718 11956 debug-util.cc:398] Leaking SignalData structure 0x7b0800354100 after lost signal to thread 11960
I20260709 12:12:34.664327 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:34.665181 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.666041 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.595860 12634 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.588574 12602 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.737856 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.737264 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.736971 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.610595 12609 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:34.643244 12505 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.006s
I20260709 12:12:34.736783 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:34.730489 12646 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:34.751609 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.609920 12595 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.625000 11250 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.652750 12624 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.774425 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.758620 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.644824 12586 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.653097 12612 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.763492 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.753238 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.756582 12506 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:34.769433 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.651345 12639 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.774258 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:34.772239 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.759059 12616 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:34.755174 12588 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.718s	user 0.009s	sys 0.009s
W20260709 12:12:34.754532 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 90227.8us
I20260709 12:12:34.777328 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:34.789255 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.790884 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.791007 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.792501 11250 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.767033 12505 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:34.793033 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.794405 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:34.783221 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.795552 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.796053 11174 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.796221 11329 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:34.797154 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.797341 12505 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.782783 12648 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:34.797996 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.798267 12648 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:34.798709 12648 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.779963 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.801921 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.782943 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.803968 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.798672 11326 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:34.752337 12593 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.805668 12617 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:34.808645 12617 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:34.809214 12617 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:34.773123 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:34.803364 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:34.812878 11256 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.803876 11329 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
W20260709 12:12:34.786691 12641 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.735s	user 0.004s	sys 0.008s
I20260709 12:12:34.780755 12634 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:12:34.822037 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.826613 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.827096 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.801512 11248 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:34.832931 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.833192 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.704627 12611 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.833910 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:34.835559 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:34.835805 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:34.825428 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3797979630001009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 12:12:34.837596 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:34.837781 11222 connection.cc:256] RPC call timeout handler was delayed by -1.358197963000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 12:12:34.838413 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3167979629999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
I20260709 12:12:34.838418 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:34.838879 11222 connection.cc:256] RPC call timeout handler was delayed by -1.2645979629999147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 12:12:34.839165 11222 connection.cc:256] RPC call timeout handler was delayed by -1.1988979630000358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
I20260709 12:12:34.792518 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.832907 11177 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.846465 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.847048 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:34.847726 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:34.848593 11181 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:34.851265 12632 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:34.854633 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.855294 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:34.833910 11250 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.837215 12616 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:12:34.858676 12610 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:34.806128 12506 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.858942 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:34.830076 12641 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.805066 12588 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.862217 12610 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:34.863235 12610 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.863793 11328 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:34.858819 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.864095 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.864598 12610 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.864776 12632 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:34.865231 12632 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:34.865684 12632 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.866132 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.843833 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:34.869827 11325 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:34.870070 11253 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:34.872385 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1085) took 1181 ms. Trace:
I20260709 12:12:34.872498 11255 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.872851 11253 rpcz_store.cc:276] 0709 12:12:33.691141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.906265 (+215124us) service_pool.cc:224] Handling call
0709 12:12:34.872155 (+965890us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b5400088ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:34.872326 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164740352}
I20260709 12:12:34.874181 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:34.875535 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.876192 11174 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:34.877200 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.862020 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:34.877262 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.861331 12506 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.879315 12593 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.879725 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.900046 11179 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:34.880410 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.848254 11326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:34.871862 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.881840 12595 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.857535 12617 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:34.884284 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:34.863823 12588 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:34.899117 11933 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:34.904711 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:34.907153 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52407.2us
I20260709 12:12:34.907779 12647 tablet_copy_client.cc:323] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:12:34.902055 11326 raft_consensus.cc:606] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:34.915257 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.912058 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.861547 12641 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.922379 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.917061 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.916919 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.917951 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.918000 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.923336 11179 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:34.924633 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:34.924590 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.925731 11181 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:34.905324 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:34.907617 12639 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:12:34.927183 12639 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:34.927465 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.927600 12639 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:34.916658 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.911337 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:34.932353 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.932765 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c5246d860d4d559ea665671cc46b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:34.932960 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:34.933741 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.933809 11179 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:34.935068 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.935319 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:34.926126 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87827.4us
I20260709 12:12:34.923923 12634 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:34.940016 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:34.949916 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.911736 12632 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:34.929315 12616 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:34.929517 12620 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:34.955441 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.946911 11179 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:34.957109 12296 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:34.957509 12296 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:34.957758 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.957929 12296 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.938001 12588 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:34.958652 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.958333 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:34.962471 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:34.963368 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:34.968873 12626 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:34.973145 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.981777 12296 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:34.984622 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.928522 12609 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:12:34.955264 12620 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:34.951129 12624 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:12:34.990581 12296 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:34.948414 12612 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:12:34.997848 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.998628 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:34.999305 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:34.969544 12626 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:34.972913 12635 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:34.960117 12615 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:12:35.012223 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.015990 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:35.019065 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 55729.8us
I20260709 12:12:35.020097 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.021620 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:35.000716 12643 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.005093 12620 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.028249 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.004657 12611 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.033291 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:34.997871 12658 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.025797 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.034577 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.039481 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.040652 11326 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:35.042183 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.043140 11326 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:35.044536 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.048450 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:35.054350 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 82335.7us
I20260709 12:12:34.985090 12648 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.055531 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.005245 12657 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:34.991008 12640 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:12:34.984203 12629 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:12:35.060647 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.062002 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.062641 12296 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.029206 12646 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.064826 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.070453 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.071214 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.072113 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.070395 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:35.073230 11180 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:35.073338 11254 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:35.066913 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:35.076421 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.076929 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:35.076325 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.077664 11933 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:35.077755 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:35.054386 12595 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.078619 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:35.076417 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77123.1us
I20260709 12:12:35.061259 12651 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.067921 12643 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.062075 12657 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:35.068320 12586 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.083940 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.016938 12639 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.087072 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.077677 12617 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.092502 12662 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.092722 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:35.093086 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.093218 12661 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.071082 12646 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.078570 12612 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.076220 12658 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.076992 12634 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.119429 12665 raft_consensus.cc:993] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:35.119700 12634 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.119903 12665 raft_consensus.cc:1081] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:12:35.120648 12634 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.121745 12634 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.098748 11326 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:35.099304 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.089048 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.081445 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 111) took 1123 ms. Trace:
I20260709 12:12:35.106009 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.133186 12296 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.107556 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1512) took 1132 ms. Trace:
I20260709 12:12:35.108842 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:35.082087 12616 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.109799 12662 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:35.111649 11178 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:35.112142 12660 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.108848 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1093) took 1123 ms. Trace:
I20260709 12:12:35.113698 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.115012 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.118132 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:35.115624 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.080366 12604 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.140324 11249 rpcz_store.cc:276] 0709 12:12:33.985778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.484181 (+498403us) service_pool.cc:224] Handling call
0709 12:12:35.108137 (+623956us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800196898. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:35.108336 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:35.108508 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:35.108534 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:35.108574 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:35.108607 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.108663 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.108761 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8089856}
I20260709 12:12:35.142112 12604 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.142701 12604 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.122851 12646 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.119210 12612 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.110285 12661 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.122061 12652 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.093724 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:35.126155 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.126933 12581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a634acd9ff543cca551c8a28e72d6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:35.127142 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 6d225f8a873c4abe992018eb9353664f from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:12:35.128507 12632 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:35.093953 12657 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.085394 12626 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.134083 11330 rpcz_store.cc:276] 0709 12:12:33.975198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.977745 (+  2547us) service_pool.cc:224] Handling call
0709 12:12:34.877899 (+900154us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7eee 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:35.106812 (+228913us) spinlock_profiling.cc:243] Waited 9.12 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7f8c 00007f3ec35c7ef6 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:35.106984 (+   172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:35.107177 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:35.107193 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:35.107280 (+    87us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:35.107298 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.107334 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.107420 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25222784}
I20260709 12:12:35.134864 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.144563 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.157286 12664 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.153818 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 6d225f8a873c4abe992018eb9353664f from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-6d225f8a873c4abe992018eb9353664f
I20260709 12:12:35.138382 12651 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.133421 11175 rpcz_store.cc:276] 0709 12:12:33.955837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.031611 (+ 75774us) service_pool.cc:224] Handling call
0709 12:12:35.069453 (+1037842us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54002a0154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:35.079493 (+ 10040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168963456}
I20260709 12:12:35.140671 12662 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.163360 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.085433 12643 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:35.149341 12612 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.177446 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.172617 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.171141 12660 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.175316 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.199952 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:12:35.199973 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65141.9us
I20260709 12:12:35.201797 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.201975 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.203392 12629 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.205740 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.163401 12648 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:35.214331 11341 tablet_copy_source_session.cc:215] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:35.212002 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.206382 12611 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.180686 12664 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:35.219861 12664 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.179775 12643 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.150547 12668 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.205657 12651 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.199568 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.225593 12628 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:35.230155 12661 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.231669 12660 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.232726 12506 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.208096 12595 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.207162 12665 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.233541 12629 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.154714 12652 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.240631 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.241112 12588 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:35.243624 12661 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.254608 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:35.257550 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.244623 12506 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.250960 12588 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.248821 12595 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:35.262168 12584 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.289776 12595 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.265219 12652 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:35.257081 12624 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.266289 12506 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.274935 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:35.276774 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.314632 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:35.321062 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.322844 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.323514 12652 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.324028 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.324560 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 96) took 1356 ms. Trace:
I20260709 12:12:35.298158 12624 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.328327 12616 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.328014 12624 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.328791 12616 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.325037 11331 rpcz_store.cc:276] 0709 12:12:33.968224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.968401 (+   177us) service_pool.cc:224] Handling call
0709 12:12:34.880729 (+912328us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:35.323461 (+442732us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7e9c 00007f3ec35c7a5b 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:35.323667 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:35.323870 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:35.323914 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:35.324081 (+   167us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:35.324128 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.324209 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.324353 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20807168}
I20260709 12:12:35.335033 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1541) took 1154 ms. Trace:
I20260709 12:12:35.335454 11328 rpcz_store.cc:276] 0709 12:12:34.180654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.865602 (+684948us) service_pool.cc:224] Handling call
0709 12:12:35.332015 (+466413us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:35.332208 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:35.332365 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:35.332389 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:35.332427 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:35.334775 (+  2348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.334842 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.334972 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12310784}
I20260709 12:12:35.339314 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 103) took 1118 ms. Trace:
I20260709 12:12:35.339664 11328 rpcz_store.cc:276] 0709 12:12:34.221331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:35.337519 (+1116188us) service_pool.cc:224] Handling call
0709 12:12:35.337866 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:35.338021 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:35.338044 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:35.338080 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:35.339080 (+  1000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.339153 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.339261 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:35.348286 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.350847 12639 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.309482 12612 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.329430 12624 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.367879 12651 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.369364 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.354897 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.377033 12620 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.374697 12662 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.386232 12595 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.358197 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 101) took 1165 ms. Trace:
I20260709 12:12:35.324082 12652 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.421439 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.448664 12638 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.450271 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.451301 12638 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.451776 12638 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:35.452306 12638 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.454487 11078 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 235) took 1292 ms. Trace:
I20260709 12:12:35.459085 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.460256 11076 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 1 to 2, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 2 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:35.460593 11323 rpcz_store.cc:276] 0709 12:12:34.192113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:35.324995 (+1132882us) service_pool.cc:224] Handling call
0709 12:12:35.325317 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:35.325484 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:35.325516 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:35.325579 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:35.325616 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.337930 (+ 12314us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.357283 (+ 19353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:35.419358 12593 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.464502 12593 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.467983 12626 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.451790 11079 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 164) took 1980 ms. Trace:
I20260709 12:12:35.471668 11079 rpcz_store.cc:276] 0709 12:12:33.471603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.471810 (+   207us) service_pool.cc:224] Handling call
0709 12:12:35.451555 (+1979745us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":8}
I20260709 12:12:35.454903 11078 rpcz_store.cc:276] 0709 12:12:34.161909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.193116 (+ 31207us) service_pool.cc:224] Handling call
0709 12:12:35.454424 (+1261308us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:34.626293 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:34.627025 (+   732us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:34.627079 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:12:34.627115 (+    36us) tablet.cc:662] Decoding operations
0709 12:12:34.627551 (+   436us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:34.627660 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:34.627702 (+    42us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:34.628047 (+   345us) tablet.cc:701] Row locks acquired
0709 12:12:34.628100 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:12:34.628279 (+   179us) write_op.cc:270] Start()
0709 12:12:34.628431 (+   152us) write_op.cc:276] Timestamp: P: 1783599154628213 usec, L: 0
0709 12:12:34.628474 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:12:34.629287 (+   813us) log.cc:844] Serialized 536 byte log entry
0709 12:12:34.908836 (+279549us) op_driver.cc:464] REPLICATION: finished
0709 12:12:35.079134 (+170298us) write_op.cc:301] APPLY: starting
0709 12:12:35.089378 (+ 10244us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:35.125617 (+ 36239us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:35.163588 (+ 37971us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:35.243837 (+ 80249us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:35.244144 (+   307us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:12:35.244229 (+    85us) write_op.cc:312] APPLY: finished
0709 12:12:35.262812 (+ 18583us) log.cc:844] Serialized 23 byte log entry
0709 12:12:35.285574 (+ 22762us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:35.285717 (+   143us) write_op.cc:454] Released schema lock
0709 12:12:35.285783 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:35.286211 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:35.286257 (+    46us) write_op.cc:454] Released schema lock
0709 12:12:35.327830 (+ 41573us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b180032b780. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
Metrics: {"mutex_wait_us":167418,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":114133,"apply.run_cpu_time_us":3859,"apply.run_wall_time_us":221738,"num_ops":1,"prepare.queue_time_us":71486,"prepare.run_cpu_time_us":69727,"prepare.run_wall_time_us":219772,"raft.queue_time_us":32418,"raft.run_cpu_time_us":37883,"raft.run_wall_time_us":116431,"replication_time_us":280148,"spinlock_wait_cycles":52738944,"thread_start_us":386528,"threads_started":4,"wal-append.queue_time_us":244422}]]}
I20260709 12:12:35.479256 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:35.489188 12662 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:35.423004 11328 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
W20260709 12:12:35.428218 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 295379us
I20260709 12:12:35.419011 11177 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
W20260709 12:12:35.495348 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 396106us
I20260709 12:12:35.446533 12643 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.490990 12636 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.503921 12643 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:35.506386 12595 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:35.499864 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.519104 12652 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.522989 12664 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.525385 12664 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.525705 11326 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:35.526113 12646 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.527557 12657 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.502661 12636 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.514231 12611 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.497754 12638 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:35.546316 12638 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:35.546916 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47127.6us
I20260709 12:12:35.552933 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.554064 12664 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.554591 12664 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.555069 12664 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.562418 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.564642 12658 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.566263 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.568588 12588 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.569306 12664 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.571904 12664 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.572687 12658 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.573156 12658 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.573510 12658 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.574385 12646 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.586387 11324 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:35.588840 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:35.589679 11179 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:35.594599 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:35.597431 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.004s	sys 0.029s
W20260709 12:12:35.597810 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.004s	sys 0.029s
I20260709 12:12:35.575630 12662 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.600950 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:35.603855 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:35.587359 12615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.000s	sys 0.049s
W20260709 12:12:35.605841 12615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.000s	sys 0.049s
W20260709 12:12:35.619625 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 123067us
W20260709 12:12:35.620484 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 231545us
I20260709 12:12:35.646066 11328 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
I20260709 12:12:35.648449 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:35.651369 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:35.652840 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66635.3us
I20260709 12:12:35.651729 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.653602 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.650995 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.599206 12620 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.577440 12604 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.666126 11177 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:35.666263 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:35.603261 12662 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.668077 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:35.671898 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:12:35.672078 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 213075us
I20260709 12:12:35.672641 11934 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:12:35.675159 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 469538us
I20260709 12:12:35.676995 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.667039 12610 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.680051 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.675864 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:35.682011 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.689011 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:35.689848 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:35.689844 12676 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:35.690336 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:35.690704 11931 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:35.692571 12641 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.693060 12609 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.693476 12609 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.694443 12609 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.695711 12609 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.695780 11325 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:35.701412 12647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d225f8a873c4abe992018eb9353664f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:35.698447 12651 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.701973 12651 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.682824 12612 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.704818 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.705431 12296 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:35.707175 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.709278 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:35.709579 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.711297 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.673779 12662 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.714069 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.683427 12664 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.683167 12639 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.717103 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.732967 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.716521 12643 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.674330 12672 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:12:35.728273 12657 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:35.729110 12588 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:35.713459 12624 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.716735 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.683605 12610 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.727255 12634 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.713512 11322 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:35.735170 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.713686 12626 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.729784 12664 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:35.732887 12638 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:35.741955 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 88412.8us
I20260709 12:12:35.669358 12636 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.778712 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.733820 12612 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.781297 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:35.734020 12643 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:35.709547 12506 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:35.738075 12615 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.801270 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.811090 12296 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.811585 12296 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.752118 12610 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:35.806609 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 71479.5us
I20260709 12:12:35.812340 12296 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.812701 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.751998 12634 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.749253 12624 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.777933 12664 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.818401 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.819705 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:35.791280 12661 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
I20260709 12:12:35.830266 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.831598 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:35.789872 12646 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.836005 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.748464 12674 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 12:12:35.833150 11181 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:35.836808 11329 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:35.837280 12584 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:35.814409 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:35.846419 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.844637 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.824923 12634 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.818470 12665 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:35.859942 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 149588us
I20260709 12:12:35.848021 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.851572 12641 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.852087 12669 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.840381 12584 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:35.852614 12612 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.857108 12626 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.827889 12639 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.829496 12624 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.861292 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.787735 12616 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.887710 12639 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:35.889148 12584 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.890406 12584 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.866344 12641 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.870110 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.841717 12615 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.848019 12661 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:35.863035 12673 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.877892 12612 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.866691 12296 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.868925 12665 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.872327 12657 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.877748 12677 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:12:35.907160 12681 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.901440 12604 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.902494 11930 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:35.907225 12641 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.913020 12669 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:35.935989 12506 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.895011 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:35.938513 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.912557 12586 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.920754 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:35.922060 11926 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:35.872717 12625 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:12:35.934182 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:35.904430 12679 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:35.943821 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:35.948026 12625 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:12:35.948272 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1535) took 1769 ms (client timeout 1824 ms). Trace:
I20260709 12:12:35.948452 12625 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:12:35.948748 11322 rpcz_store.cc:269] 0709 12:12:34.179078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:35.240307 (+1061229us) service_pool.cc:224] Handling call
0709 12:12:35.948078 (+707771us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b5400285fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:35.948208 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053006592}
I20260709 12:12:35.949494 12636 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.918242 12661 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:35.953474 12661 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.918829 12626 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.957811 12296 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.957949 12626 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.958518 12296 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:35.958669 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:35.958897 12626 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.959337 12296 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:35.959934 12626 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.960194 12296 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.960516 12296 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:35.917486 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:35.952087 12657 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.915707 12615 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:35.961896 12615 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.961978 12657 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.962368 12657 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:35.938954 12673 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.963236 12657 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:35.964437 12657 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.946247 12669 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:35.965807 12669 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:35.966220 12669 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.921104 12612 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.974237 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:35.967880 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50503.4us
I20260709 12:12:35.965400 12673 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:35.929956 12616 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:35.944762 12586 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:35.982621 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 76241us
I20260709 12:12:35.985847 12658 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:35.988044 12588 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:35.987288 12586 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:35.991707 12635 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:35.992146 12635 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:35.995158 12611 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:35.995602 12611 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:35.982942 12677 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:12:35.996500 12611 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.000319 12581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52012 (request call id 393) took 5884 ms. Trace:
I20260709 12:12:35.978641 12665 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.009306 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.003160 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.013723 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.004884 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.003772 12611 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:35.961273 12296 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:36.022711 12643 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.043125 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:36.020977 12586 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.023906 12584 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.032217 12664 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.036031 12634 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.037946 12677 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:36.038663 12658 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.039943 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:36.012401 12619 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.005s
I20260709 12:12:36.044183 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.072839 12619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52014 (request call id 271) took 2664 ms. Trace:
I20260709 12:12:36.073488 11929 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:36.000774 12581 rpcz_store.cc:276] 0709 12:12:30.115494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.115648 (+   154us) service_pool.cc:224] Handling call
0709 12:12:30.119849 (+  4201us) ts_tablet_manager.cc:934] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
0709 12:12:35.079627 (+4959778us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:35.686071 (+606444us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:35.686262 (+   191us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:35.779737 (+ 93475us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:35.839614 (+ 59877us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:35.999879 (+160265us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c81a2 00007f3ecc509889 00007f3ecb63ad88 00007f3ecb701973 00007f3ecb6fd70f 00007f3ecb6ff2d3 00007f3ecb730f4e 00007f3ecb715fdd 00007f3ecb715f89
0709 12:12:36.000023 (+   144us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:36.000220 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":310,"spinlock_wait_cycles":12437248,"tablet-copy.queue_time_us":3392,"thread_start_us":87089,"threads_started":2}
I20260709 12:12:36.074571 12581 tablet_copy_client.cc:806] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:36.047528 12641 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.075176 12581 tablet_copy_client.cc:670] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:36.075412 12634 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.048722 12610 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.076414 12641 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.049849 12684 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.069010 12677 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:36.061975 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:36.067509 12615 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.077886 12677 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.073695 12664 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:36.079198 12677 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:36.057583 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 321561us
I20260709 12:12:36.037256 12651 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.080672 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.079596 12684 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.081558 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:36.083875 12685 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:36.088553 12685 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.089434 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.090322 12610 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.090790 12685 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.095337 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.096195 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:36.094424 12296 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.096835 11328 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
W20260709 12:12:36.094733 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 83725.1us
I20260709 12:12:36.099663 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:36.100611 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.083825 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.050887 12625 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:36.073266 12619 rpcz_store.cc:276] 0709 12:12:33.408912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.409057 (+   145us) service_pool.cc:224] Handling call
0709 12:12:33.487213 (+ 78156us) ts_tablet_manager.cc:934] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
0709 12:12:34.927120 (+1439907us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:35.506761 (+579641us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:35.506903 (+   142us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:36.072090 (+565187us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:36.072397 (+   307us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:36.072694 (+   297us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:36.072755 (+    61us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":49402,"tablet-copy.queue_time_us":76742,"thread_start_us":141596,"threads_started":2}
I20260709 12:12:36.079129 12615 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.083160 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:36.058952 12665 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:12:36.152743 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.104023 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:36.155247 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51297.4us
I20260709 12:12:36.156517 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.111941 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.113511 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.119470 11255 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.158459 11176 raft_consensus.cc:2410] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:36.119992 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.159471 11326 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.121037 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.086802 12604 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.121548 12676 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:36.092965 12670 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:12:36.122520 12638 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:36.162142 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.162546 12638 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:36.163021 12638 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.092124 12684 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.123566 11252 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.111419 12643 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.150882 12624 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.125310 11248 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:12:36.171409 12624 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.126940 12625 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.127197 12584 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.128299 11179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45582
I20260709 12:12:36.130069 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.110855 12626 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:36.179800 11249 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.132488 12586 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.086452 12506 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:36.091732 12581 tablet_copy_client.cc:538] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:36.134897 11253 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.109381 12634 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:36.024503 12635 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.186508 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:36.109787 12674 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.188431 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.135993 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:36.139371 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.079923 12661 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.143031 12652 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:36.105304 11328 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:12:36.152092 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69027.4us
I20260709 12:12:36.153370 12611 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:36.161957 11926 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.174350 11179 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:36.157271 12593 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.175112 11326 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:36.226008 11179 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.101418 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.147604 12647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52012 (request call id 395) took 1934 ms. Trace:
I20260709 12:12:36.106812 12662 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.182716 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.186232 12678 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.186794 11253 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:12:36.135252 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.205677 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.192551 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.190165 12634 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:36.199174 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.209054 12652 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.192548 12506 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:36.184890 12626 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.242033 12611 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.258526 12626 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:36.258759 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.182659 12604 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.259609 12611 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.260661 12604 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.261042 12661 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.211095 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.197095 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.223555 11173 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.226943 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:36.229360 11256 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
W20260709 12:12:36.229513 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 128146us
I20260709 12:12:36.239213 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fd765747f9648049b905c70696b206d"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:36.239473 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.240278 11180 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.240397 12612 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:36.243832 11255 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.204170 12619 tablet_copy_client.cc:806] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:36.248047 11257 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.248767 11251 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.248242 12584 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.253099 12686 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:12:36.261808 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 287697us
I20260709 12:12:36.210469 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.184777 12676 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.283756 12611 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:36.294250 12611 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:36.294476 12673 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:36.294965 12673 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.295344 12611 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.266670 11249 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.273175 11182 raft_consensus.cc:606] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.247736 12634 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.271926 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:36.252099 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 43598.3us
I20260709 12:12:36.298516 12676 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.282068 12619 tablet_copy_client.cc:670] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:36.283154 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.258556 12663 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:36.263314 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:36.264847 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 164288us
I20260709 12:12:36.286949 11252 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:12:36.307273 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44057.8us
I20260709 12:12:36.286885 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:36.277726 12506 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.309800 12674 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.310355 12624 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:36.278685 12651 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.315171 12586 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.316679 12669 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.294278 11927 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:36.292007 12662 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.284560 12612 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.320731 12638 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.340334 12638 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:36.322513 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.322731 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.312584 11179 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.322114 12584 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:12:36.333328 12652 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.335098 12581 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:36.323146 12624 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:36.335669 12663 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:36.349682 12624 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.351614 11179 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.342509 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.347007 12671 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.349664 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.355329 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:36.355903 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25983a67e23a4c708ae496061b992207"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:36.356118 12296 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.350809 12663 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.228161 12647 rpcz_store.cc:276] 0709 12:12:34.204059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.205677 (+  1618us) service_pool.cc:224] Handling call
0709 12:12:34.750165 (+544488us) ts_tablet_manager.cc:934] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
0709 12:12:35.657379 (+907214us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:36.014405 (+357026us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:36.017500 (+  3095us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:36.049397 (+ 31897us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:36.056418 (+  7021us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:36.109993 (+ 53575us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:36.117870 (+  7877us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":308,"tablet-copy.queue_time_us":356782,"thread_start_us":375120,"threads_started":2}
I20260709 12:12:36.362540 12647 tablet_copy_client.cc:806] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:36.363324 12647 tablet_copy_client.cc:670] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:36.359328 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.364602 11331 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:36.340842 12609 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.360677 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:36.356690 11176 raft_consensus.cc:606] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:36.368175 12296 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:36.369141 12296 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:36.372454 12691 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:36.372669 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet 5df07764623640ce8ec29ad695812b7c from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:12:36.365826 12586 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:36.359370 12643 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:12:36.376149 12586 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:36.376765 12643 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:36.377159 12643 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:36.377197 12586 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.378331 12643 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.379599 12643 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:36.381597 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet 5df07764623640ce8ec29ad695812b7c from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-5df07764623640ce8ec29ad695812b7c
I20260709 12:12:36.386446 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.387648 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.387974 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.388921 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.389514 12612 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.388684 12692 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:12:36.380945 12624 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:36.393007 12296 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.381129 12691 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:12:36.396425 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:36.396588 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.369282 12609 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.401684 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.407006 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.407444 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.408185 11256 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.384064 12616 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:36.408484 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.412668 12593 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:36.410586 12677 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:12:36.413256 12593 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:36.413439 12677 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:36.413736 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.414047 12677 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:36.413475 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.414381 12593 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:36.417518 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.417519 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.410748 12686 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:36.422749 12638 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.425530 12686 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.426491 11248 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.428432 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.418853 12673 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.430070 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:36.431389 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.431510 12634 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.432261 11251 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.433322 11248 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.422145 12674 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:36.402985 12625 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.417678 11252 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.440531 11326 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:36.441094 12669 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:36.423336 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:36.442133 11325 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.442026 12638 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.443028 11252 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.443814 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.445824 11267 tablet_copy_source_session.cc:215] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:12:36.448307 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.452656 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.453455 11257 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.450093 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:36.189821 12639 consensus_queue.cc:1175] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "786851d4227e4dd89b75ccdea0b1f03b" responder_term: 2 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:12:36.464717 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.458565 11180 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.465483 11256 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:36.463909 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.462481 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.464565 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.446175 12691 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.475301 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.476943 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.415428 12616 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.478062 11252 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.479804 12686 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.480523 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.480839 11252 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.466625 12678 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.490702 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.491035 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.491796 11257 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.492956 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.481709 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.491847 11254 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.481542 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.455749 12595 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:36.497344 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.497936 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.498123 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.499503 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.500979 11257 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.415071 12677 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:36.502027 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.502861 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.501948 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:36.503666 11326 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:36.504076 11253 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.474959 12647 tablet_copy_client.cc:538] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:36.472187 12681 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:36.502874 11325 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:36.511404 11927 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.514977 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.515611 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.516286 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.517105 11323 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:36.518258 12681 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:36.508793 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.518172 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.511600 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:36.520983 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.507973 12678 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.500036 12595 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:36.522934 11327 raft_consensus.cc:2410] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:12:36.512305 12634 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:36.529753 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.530946 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.490810 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.536099 11330 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:36.541162 11324 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:36.541188 11256 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:36.572728 11330 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:12:36.546541 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:36.591017 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44553us
I20260709 12:12:36.591775 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.589535 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.594148 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.594318 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.595628 11330 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.596678 11330 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.519474 12681 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.602370 12692 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.612134 12661 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.612648 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.531044 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.532032 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.614950 12626 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:36.616061 11076 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 239) took 1773 ms. Trace:
I20260709 12:12:36.611878 12679 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:36.617991 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.641180 11255 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:36.618847 11250 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:36.619683 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.643275 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.644109 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.635486 12581 tablet_bootstrap.cc:700] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent opening tablet: real 0.107s	user 0.003s	sys 0.000s
I20260709 12:12:36.620422 11249 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.620657 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.620921 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.621975 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:36.625290 12638 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.533200 12609 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.633980 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.619860 12616 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:36.602924 12625 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.639153 11076 rpcz_store.cc:276] 0709 12:12:34.843039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.848970 (+  5931us) service_pool.cc:224] Handling call
0709 12:12:36.615992 (+1767022us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:35.714414 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:35.902538 (+188124us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:35.904418 (+  1880us) write_op.cc:435] Acquired schema lock
0709 12:12:35.907113 (+  2695us) tablet.cc:662] Decoding operations
0709 12:12:35.979164 (+ 72051us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:35.982509 (+  3345us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:35.983549 (+  1040us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:36.038317 (+ 54768us) tablet.cc:701] Row locks acquired
0709 12:12:36.038337 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:12:36.038494 (+   157us) write_op.cc:270] Start()
0709 12:12:36.038618 (+   124us) write_op.cc:276] Timestamp: P: 1783599156038446 usec, L: 0
0709 12:12:36.038656 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:12:36.039331 (+   675us) log.cc:844] Serialized 541 byte log entry
0709 12:12:36.155002 (+115671us) op_driver.cc:464] REPLICATION: finished
0709 12:12:36.374562 (+219560us) write_op.cc:301] APPLY: starting
0709 12:12:36.374673 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:36.374796 (+   123us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:36.382569 (+  7773us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:36.385314 (+  2745us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:36.385476 (+   162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:12:36.385522 (+    46us) write_op.cc:312] APPLY: finished
0709 12:12:36.436427 (+ 50905us) log.cc:844] Serialized 23 byte log entry
0709 12:12:36.467104 (+ 30677us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:36.471721 (+  4617us) write_op.cc:454] Released schema lock
0709 12:12:36.473704 (+  1983us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:36.479052 (+  5348us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:36.479085 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":600132,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":213651,"apply.run_cpu_time_us":2962,"apply.run_wall_time_us":104475,"num_ops":1,"prepare.queue_time_us":203214,"prepare.run_cpu_time_us":4502,"prepare.run_wall_time_us":353979,"replication_time_us":55869,"spinlock_wait_cycles":25472,"thread_start_us":252955,"threads_started":2}]]}
I20260709 12:12:36.649230 12664 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.649855 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.671485 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.640643 12679 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:36.589013 12634 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:12:36.674216 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.672891 12679 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.680660 11331 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:36.681145 11331 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:36.700366 11331 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.704792 11331 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.700919 11326 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:36.708293 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.716080 12669 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.680694 12679 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:36.678076 12663 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.682549 12662 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.728715 12677 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.657874 12595 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.734603 12595 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:36.718075 12664 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.722715 11331 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
W20260709 12:12:36.741707 12612 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:12:36.745366 12647 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:36.746989 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.751978 12635 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.755650 12663 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.757609 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.763024 12663 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.764281 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:36.766079 12663 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:36.766760 12663 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:36.767760 12663 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.783562 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.784198 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:36.784723 12670 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:36.785262 11934 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 2: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:36.786116 12670 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:36.787667 12636 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:36.788165 12636 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:36.789333 12636 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.790978 12636 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.791563 12625 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:36.792114 12625 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:36.794826 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:36.795451 11249 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:36.796341 12639 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.798502 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.806957 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.799762 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:36.812477 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:36.811113 11250 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 2: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:36.809755 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:36.821987 12612 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:36.822391 12612 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:36.822834 12612 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:36.823081 12664 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.828842 12687 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:36.830616 12669 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:36.831092 12669 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:36.804520 12639 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.793203 12625 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:36.820788 12662 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.847083 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.842113 12669 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:36.870959 12669 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.850296 12664 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:12:36.861439 12639 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:12:36.902427 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 312159us
I20260709 12:12:36.910406 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.914232 11934 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:36.915369 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:36.932992 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 480431us
I20260709 12:12:36.912024 12634 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.939870 11177 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:36.927901 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.906399 12643 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:36.922115 12670 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.942855 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.946144 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:36.943547 12672 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.944446 12643 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:36.952150 11176 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:36.928462 12625 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:36.959775 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:36.928462 12671 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:12:36.930467 12673 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:12:36.975250 12296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.029s	sys 0.010s
W20260709 12:12:36.975689 12296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.029s	sys 0.010s
I20260709 12:12:36.962364 12681 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:36.980130 11176 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:36.983441 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139438us
I20260709 12:12:36.963996 12686 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:12:36.994524 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:36.963654 12678 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
W20260709 12:12:36.998919 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57383.4us
I20260709 12:12:37.002734 12296 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:37.003232 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:37.010641 12662 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:37.011104 12686 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:37.011584 12686 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:37.012567 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:37.012486 12686 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.014040 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:37.014700 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69361.7us
W20260709 12:12:37.015334 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.005741 11177 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:37.007220 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.007926 12296 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:37.017553 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:37.021751 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.023033 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.026072 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.025323 12616 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:37.027304 12663 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:37.024356 12296 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.025444 12708 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:36.998209 12638 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:12:37.034317 12593 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:37.034761 12593 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:37.035645 12593 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.036149 12616 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:37.037257 12678 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:37.038985 12678 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:37.040005 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.039832 12678 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.040510 11328 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:37.024129 12673 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:37.041196 12678 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.041771 12681 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:37.042259 12681 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:37.043181 12681 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.023878 12662 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:36.996835 12643 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.053140 12662 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:37.058408 11327 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:37.058799 12663 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:12:37.061017 12703 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:12:37.061913 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df07764623640ce8ec29ad695812b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:37.004043 11250 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:37.053452 12686 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.065238 12663 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:37.066447 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:37.063242 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.071642 11249 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:37.072252 11249 raft_consensus.cc:614] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:37.052974 11322 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:37.075815 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:37.077826 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:37.015268 12612 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.079048 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:37.080257 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:37.080750 12612 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:37.080991 12611 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:37.081544 12611 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:37.071439 12663 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:37.082755 12611 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.084666 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:37.094018 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:37.094686 11329 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:37.095670 12708 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:37.096297 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:37.096992 11250 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.100646 11181 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:37.105835 12616 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:37.106125 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.107072 12709 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:37.107918 12673 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:37.070170 12506 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:37.070204 12586 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:37.111528 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:37.102384 12612 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.189028 12612 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.111430 12616 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:37.190579 11250 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:37.191411 12612 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:37.113561 12663 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.193599 12663 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.187588 11176 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.194205 11322 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:37.107899 12619 tablet_copy_client.cc:538] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:37.197835 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.196509 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:37.198959 11249 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:37.202499 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:37.157843 12673 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:37.156915 12643 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:37.156013 12611 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.206219 11326 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:37.206511 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:37.209722 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 98952.4us
I20260709 12:12:37.221339 11181 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:37.101823 12697 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 12:12:37.222061 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.200721 12672 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:37.207330 12612 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:37.224697 12612 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:37.225605 12612 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.224289 12672 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.231463 12672 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:37.233834 12717 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:37.225898 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.241690 12717 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:37.250118 11176 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:37.206789 12609 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:37.207011 12673 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.205812 12616 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.267824 11078 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:37.273904 12709 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:37.308667 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.341315 11182 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:37.313645 11328 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:37.346382 11249 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:37.348965 12638 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:37.349558 12638 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.351804 11328 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:37.351999 12638 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:37.355530 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:37.356405 11328 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:37.358201 11329 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.363157 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:37.361814 12674 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:12:37.367321 12674 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:37.370651 11329 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:37.382639 12652 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:37.335572 12609 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:37.391686 12716 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:12:37.393275 12709 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.338861 12625 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.407958 12593 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:37.332543 12664 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:12:37.400859 12609 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.402484 12697 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:12:37.429375 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 267924us
I20260709 12:12:37.411758 12651 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:37.411494 12635 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.377075 12616 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.460105 11223 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:37.472690 11329 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:37.444020 12677 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.424551 12625 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.445350 12651 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:37.441707 12697 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:37.525190 11934 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:37.512931 12612 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.513906 12698 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:37.517488 12707 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:37.467996 12724 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:12:37.447944 12669 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.111s
I20260709 12:12:37.460843 12635 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.517644 12723 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
W20260709 12:12:37.602677 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 239601us
W20260709 12:12:37.604475 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.525403 12697 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:37.522914 12677 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.617836 12736 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.618602 12736 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.623238 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.632936 12296 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.626611 12584 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.636343 12584 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.633702 12296 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.559671 12612 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:37.645651 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 281431us
I20260709 12:12:37.620069 12677 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:37.663348 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:37.663901 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.664311 11249 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:37.562125 12698 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.664600 11257 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:37.639768 11927 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:37.670109 12296 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:37.671890 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.672600 11176 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:37.673995 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:37.676118 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.664736 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:37.665414 12677 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.596683 12651 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.665571 12595 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:12:37.682132 12652 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:37.596496 12726 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:12:37.695312 11926 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
W20260709 12:12:37.734295 12678 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.002s
I20260709 12:12:37.735693 11934 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:37.699949 12652 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:37.738318 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:37.701949 12730 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:12:37.709196 12625 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:37.695120 12677 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:37.734390 12722 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:12:37.751824 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.735814 12647 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:37.758719 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:37.755766 12721 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:37.760468 12721 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:37.750239 12652 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.760609 12635 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:37.760968 12721 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:37.740588 12671 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.776135 12736 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:37.777544 12584 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:37.782720 12698 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:37.783144 12612 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:37.783501 12625 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:37.784365 12733 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:12:37.784065 12625 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.784965 12651 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:37.786535 12625 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:37.752916 12669 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:12:37.788118 12647 tablet_bootstrap.cc:492] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:37.788197 12616 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:37.789093 12647 ts_tablet_manager.cc:1403] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 1.044s	user 0.046s	sys 0.012s
I20260709 12:12:37.781185 12618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52014 (request call id 272) took 4371 ms. Trace:
W20260709 12:12:37.773835 12720 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:12:37.764407 12635 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:37.798900 12734 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:12:37.765538 12718 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:37.768831 12581 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:37.775053 12671 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.811573 12677 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:37.766481 12708 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:37.768810 12664 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:37.815680 12663 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.000s
I20260709 12:12:37.817687 12673 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.819144 12635 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:37.803966 12611 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.553s	user 0.005s	sys 0.004s
I20260709 12:12:37.818125 12618 rpcz_store.cc:276] 0709 12:12:33.409279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.411954 (+  2675us) service_pool.cc:224] Handling call
0709 12:12:33.611085 (+199131us) ts_tablet_manager.cc:934] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
0709 12:12:34.515124 (+904039us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4c00197500. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec49b8a20 00007f3ec49c9a29 00007f3ec49ca13f 00007f3ec498e2ab 00007f3ec49bbf3c 00007f3ec49bc461 00007f3ec49bc768 00007f3eca7422eb 00007f3ecb667d3b 00007f3ecb662008 00007f3ecb661405
0709 12:12:37.060315 (+2545191us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:37.257334 (+197019us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:37.299683 (+ 42349us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:37.780268 (+480585us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:37.780588 (+   320us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:37.780968 (+   380us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:37.781115 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":81403,"spinlock_wait_cycles":33394560,"tablet-copy.queue_time_us":129279,"thread_start_us":142108,"threads_started":2}
I20260709 12:12:37.820024 12609 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.836042 12581 tablet_bootstrap.cc:492] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:37.841306 12721 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:37.836971 12619 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:37.842868 12635 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:37.838580 12731 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.836486 12662 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:37.858119 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.899546 11257 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:37.862141 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.901536 11176 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:37.867156 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.869308 11179 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:37.870622 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.872709 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.875264 12739 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:37.875803 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.878885 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.857918 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:37.883773 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 199066us
I20260709 12:12:37.886538 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:37.895349 11328 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:37.896907 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:37.921731 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.922129 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.926805 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.927166 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:37.928637 11174 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
W20260709 12:12:37.922776 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 159055us
I20260709 12:12:37.881186 12618 tablet_copy_client.cc:806] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:37.839442 12664 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.921404 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:37.925166 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:37.880914 12663 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.918290 12581 ts_tablet_manager.cc:1403] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 1.584s	user 0.060s	sys 0.054s
I20260709 12:12:37.920009 11173 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:37.930577 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.929467 11326 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:37.919783 11324 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:37.932457 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:37.902915 12686 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.951301 11328 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:37.917831 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:37.918012 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:37.954563 11251 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:37.959766 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:37.918447 12742 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:37.899641 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 254751us
I20260709 12:12:37.971596 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:37.982370 12737 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:37.983233 12737 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:37.983793 12737 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.936735 12595 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.954288 12618 tablet_copy_client.cc:670] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:37.917429 12662 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.902105 12697 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.936468 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:37.964075 12707 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:37.934128 12701 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:12:38.020646 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:37.959914 12663 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:37.990718 12736 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:37.970239 12612 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:38.030154 11324 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:37.899602 12731 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:37.963529 12686 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.011965 12673 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.057977 12736 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.058916 12736 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:38.000924 12635 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:12:38.027980 12676 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:38.070616 11250 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:38.010800 12595 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.048147 11323 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:38.037915 12677 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:37.950842 12739 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:38.083716 12676 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.015786 12742 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.036518 12612 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.090835 12662 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.091414 12671 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.091908 12737 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:38.093096 12697 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.073002 12707 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:38.095963 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.034250 12593 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
W20260709 12:12:38.143285 11147 consensus_peers.cc:597] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:38.142218 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:38.144852 11255 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:38.144721 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 211928us
I20260709 12:12:38.119171 12676 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:38.088119 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 56617.4us
I20260709 12:12:38.064308 12647 raft_consensus.cc:359] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:38.086546 12669 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:12:38.105208 11251 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:38.126608 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:38.133828 12721 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.128448 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.165908 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:38.173844 11257 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:38.174664 11253 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:38.174901 11254 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:38.113511 12609 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.145728 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:38.097349 12724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.027s
I20260709 12:12:38.181586 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.198459 11324 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:38.199831 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.183889 11254 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:38.186132 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:38.188660 12506 raft_consensus.cc:993] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: : Instructing follower 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:38.192201 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:38.226214 11256 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:38.223857 11929 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:38.149577 12664 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.202334 11257 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:38.162245 12647 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:38.124058 12581 raft_consensus.cc:359] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:38.229919 11178 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.286614 12744 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 12:12:38.287669 11926 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:38.289153 12612 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.289795 12729 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.289975 12612 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.290140 12663 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:38.225185 12724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.000s	sys 0.028s
I20260709 12:12:38.292270 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.290530 12612 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.316174 11324 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:38.290427 12729 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:38.294385 12652 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.533s	user 0.005s	sys 0.005s
I20260709 12:12:38.295210 12737 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.295715 12609 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:12:38.296115 12651 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 12:12:38.291399 12712 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:38.291303 12581 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:38.296849 12686 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.246675 12674 raft_consensus.cc:1081] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Signalling peer 786851d4227e4dd89b75ccdea0b1f03b to start an election
I20260709 12:12:38.233816 12721 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.298166 12748 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.299788 11251 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:38.300256 11178 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.303661 11927 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.318535 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.229943 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:38.297559 12750 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.112s	user 0.001s	sys 0.001s
I20260709 12:12:38.331588 12743 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.323243 12584 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:12:38.330840 12609 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:38.350275 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:38.313917 11255 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.351163 12741 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.344475 11249 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:38.353300 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:38.329878 12730 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.348057 12712 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.312004 12736 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.359850 12679 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:38.322475 12697 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:12:38.225064 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:38.364856 12738 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:38.365396 12735 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:12:38.371825 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:38.373598 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:38.373888 12745 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.374410 12584 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:38.331347 12647 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:38.332569 12701 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:12:38.376415 12581 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:38.362681 11254 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:38.390105 12712 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:38.383895 11180 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:38.361270 12748 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.358088 12737 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.349462 12721 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.392232 12612 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.394124 12736 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.385107 12726 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.360338 12742 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.330240 12652 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.378465 12609 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.402880 12738 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.403353 12697 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:38.403460 12735 consensus_queue.cc:1243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:12:38.385313 12663 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.356644 12685 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:12:38.414177 12647 ts_tablet_manager.cc:1434] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.625s	user 0.011s	sys 0.006s
I20260709 12:12:38.415056 12581 ts_tablet_manager.cc:1434] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.461s	user 0.007s	sys 0.002s
I20260709 12:12:38.416806 12748 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.403764 12584 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:38.403739 11248 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
W20260709 12:12:38.420112 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 465104us
I20260709 12:12:38.454527 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:38.455653 11934 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:38.391108 12724 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.409737 11175 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:38.446221 12742 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.446341 12729 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.362895 11323 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:38.382738 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:38.438120 12697 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:38.401187 12743 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.465029 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:38.438169 12738 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.433701 12737 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:38.469012 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 116311us
I20260709 12:12:38.472779 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.473800 11933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25983a67e23a4c708ae496061b992207"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:52016
W20260709 12:12:38.447042 12647 ts_tablet_manager.cc:1467] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Tablet startup took 657ms
I20260709 12:12:38.474340 11933 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:38.481112 11933 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.481715 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.405431 12730 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.482667 12685 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:38.483676 12685 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:38.484272 12730 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.461905 12686 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.487742 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.406960 12741 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.484804 12730 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:38.474264 12708 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:38.448652 12663 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.484907 12685 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:38.427186 12745 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.434763 12726 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.505587 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.493305 12698 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:38.496332 12758 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:38.467873 12742 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.514856 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:38.496062 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:38.493623 12756 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.499321 12730 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.466954 12724 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.494086 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:38.541775 12666 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.567459 11929 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:38.463110 12584 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.569018 12584 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:38.518648 12697 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.520890 12726 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.509012 11174 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:38.496166 12736 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:38.524005 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.539862 11933 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.503564 12685 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:38.546598 11182 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:38.503836 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:38.521799 12662 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:38.591054 12662 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.591615 12662 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.493822 12686 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.597043 12686 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.540374 12671 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.522106 12698 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:38.605326 12743 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.607167 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.611423 11331 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:38.610085 11933 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.572551 12730 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.561619 12721 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:38.620012 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 491689us
I20260709 12:12:38.601609 12687 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.623035 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:38.623550 11181 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.610769 12698 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:38.632475 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.607656 11928 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:38.559288 12758 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.567461 12666 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.643615 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:38.619599 12729 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:38.607283 12736 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.642940 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:38.652721 11930 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:38.653232 12741 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.636842 12687 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.634176 12697 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.633550 12748 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:38.610982 12671 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.673200 12745 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.667732 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:38.675400 11927 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:38.677058 12758 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.677435 12609 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.649214 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:38.683595 12666 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.685487 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.670156 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:38.611066 12756 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:38.699642 12676 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.003s
I20260709 12:12:38.700205 12676 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.664634 12729 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.712685 12742 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.688225 12737 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.718066 12756 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.693087 12745 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:38.692782 12765 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.670713 12748 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.692886 11325 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:38.739523 12745 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.715798 11929 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.689953 12685 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.755139 12745 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.759573 12724 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.773765 12687 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.741468 12765 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.776706 12676 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.772974 11927 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:38.787768 12685 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:38.801085 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 168684us
I20260709 12:12:38.798537 12671 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.806035 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.810945 12765 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.813833 12756 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:38.796191 12736 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.816768 12756 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.817255 12756 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.814030 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:38.480948 12647 ts_tablet_manager.cc:1469] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Trace:
0709 12:12:34.204059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.205677 (+  1618us) service_pool.cc:224] Handling call
0709 12:12:34.750165 (+544488us) ts_tablet_manager.cc:934] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
0709 12:12:35.657379 (+907214us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:36.014405 (+357026us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:36.017500 (+  3095us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:36.049397 (+ 31897us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:36.056418 (+  7021us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:36.109993 (+ 53575us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:36.117870 (+  7877us) inbound_call.cc:177] Queueing success response
0709 12:12:36.744882 (+627012us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:12:37.789491 (+1044609us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:12:37.795080 (+  5589us) tablet_replica.cc:261] Starting instrumentation
0709 12:12:37.796079 (+   999us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":308,"raft.queue_time_us":424,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":4036,"spinlock_wait_cycles":89600,"tablet-copy.queue_time_us":356782,"tablet-download-pool.queue_time_us":717,"tablet-download-pool.run_cpu_time_us":2520,"tablet-download-pool.run_wall_time_us":107158,"thread_start_us":375120,"threads_started":2}
I20260709 12:12:38.820135 12652 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.848152 11181 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.853269 12739 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.848389 12730 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.856863 11322 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:38.856787 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.858136 12652 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:38.858319 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.858759 12652 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.859692 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.860371 11323 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:38.860975 12652 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.861346 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.862040 11323 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:38.862701 12780 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:12:38.863824 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49891.4us
I20260709 12:12:38.864023 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.849433 12729 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.850281 12662 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.864018 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.866058 11329 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:38.866364 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.866973 11330 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:38.867290 12662 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.864606 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:38.867801 11178 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.864728 11326 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:38.868541 12780 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:38.869896 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.870851 11331 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:38.871019 12738 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.872037 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:38.872869 12652 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:38.873412 12652 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.886324 12780 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:38.887905 12780 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:38.889008 12780 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.894346 12756 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:38.895270 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:38.896138 11331 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:38.898783 11329 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:38.899900 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.900235 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.900641 11331 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:38.867925 12748 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.908427 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.909950 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:38.910275 12652 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:38.910712 11929 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.901203 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.850142 12698 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.907295 12756 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:38.907168 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:38.913138 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.912853 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:38.914287 11331 raft_consensus.cc:2368] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3.
I20260709 12:12:38.914969 11926 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:38.915048 12669 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:38.915818 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:38.888521 12712 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:38.925904 12619 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:38.915938 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.902592 12612 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:38.930522 12612 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.931016 12677 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:38.931244 12612 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:38.931682 12762 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:38.932199 12762 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.932618 12762 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:38.914287 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.937227 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.912220 12752 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:38.941473 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:38.942032 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.912103 12738 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.931613 12677 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:38.944315 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:38.927105 12619 tablet_bootstrap.cc:492] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:38.939210 12744 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:38.945758 12736 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:38.924875 12669 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.931272 12712 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.927544 12652 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:12:38.941530 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:38.941747 12752 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.029255 11252 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:38.984733 12753 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:39.172363 12619 ts_tablet_manager.cc:1403] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 1.382s	user 0.061s	sys 0.069s
I20260709 12:12:38.946765 12723 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.226s
I20260709 12:12:38.939769 12756 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:12:39.169670 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 110710us
I20260709 12:12:39.032035 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:39.170121 12669 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:39.170533 12662 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:39.170715 12736 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:39.171010 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 226762us
I20260709 12:12:38.944581 12748 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:39.172377 12753 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:39.036554 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 581669us
W20260709 12:12:39.184499 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 152516us
I20260709 12:12:39.182000 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:38.946429 12707 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:12:39.191452 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.194002 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:39.193953 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:39.201941 11254 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.169006 11927 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:39.208078 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.201917 11249 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:39.208714 11177 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:39.171525 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:39.027117 12635 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.171495 11252 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:39.169075 12752 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:39.204872 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.206542 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:39.216552 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:39.217026 11223 leader_election.cc:400] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3.
I20260709 12:12:39.206939 12763 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.207330 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.099576 12709 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:39.023564 12730 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:39.210019 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.204916 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:39.212621 11253 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:39.212658 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:39.255502 11254 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:38.927977 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:39.034953 12731 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:39.107481 12765 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:39.036260 12745 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.032959 11325 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:39.206682 12685 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:39.195386 12756 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:39.192157 12737 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:39.306396 12737 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:39.306711 12756 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.307461 12737 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:39.309823 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.294004 12765 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.310500 11180 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:39.192021 12662 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.311707 12662 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.314437 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:39.200515 12748 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:39.314249 12765 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:39.183235 12744 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.319344 12744 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.314966 12595 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:39.320353 12662 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.222150 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.197759 12701 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.322214 12701 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.322630 12662 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:38.947929 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:39.323834 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:39.324085 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.324417 11254 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.324810 11253 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:39.217214 11177 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.198606 12736 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:39.244745 11078 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 165) took 1998 ms. Trace:
I20260709 12:12:39.246031 11223 leader_election.cc:403] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:39.250945 12758 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:39.254045 11929 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:39.256102 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:39.213421 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:39.256954 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52110.8us
I20260709 12:12:39.214908 12741 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.250869 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:39.251318 12635 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.265539 12711 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:39.266534 11252 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.269152 11249 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:39.278200 11173 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:39.257045 12709 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.237743 12506 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:39.244534 12707 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.269249 12762 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.317868 12748 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.228008 12663 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:39.315115 12612 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.325974 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:39.236860 12786 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:39.326658 11179 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:39.327723 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.329221 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:39.351694 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 155705us
I20260709 12:12:39.260712 12742 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:39.323308 12662 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.337370 11078 rpcz_store.cc:276] 0709 12:12:37.246068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.246247 (+   179us) service_pool.cc:224] Handling call
0709 12:12:39.244665 (+1998418us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:37.627745 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:37.716228 (+ 88483us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:37.737579 (+ 21351us) write_op.cc:435] Acquired schema lock
0709 12:12:37.745794 (+  8215us) tablet.cc:662] Decoding operations
0709 12:12:37.802152 (+ 56358us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:37.802238 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:37.802274 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:37.802497 (+   223us) tablet.cc:701] Row locks acquired
0709 12:12:37.802518 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:12:37.802680 (+   162us) write_op.cc:270] Start()
0709 12:12:37.802844 (+   164us) write_op.cc:276] Timestamp: P: 1783599157802620 usec, L: 0
0709 12:12:37.802869 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:12:37.803735 (+   866us) log.cc:844] Serialized 536 byte log entry
0709 12:12:38.521693 (+717958us) op_driver.cc:464] REPLICATION: finished
0709 12:12:38.744655 (+222962us) write_op.cc:301] APPLY: starting
0709 12:12:38.744754 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:38.744866 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:38.772618 (+ 27752us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:38.795787 (+ 23169us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:38.801450 (+  5663us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:12:38.809233 (+  7783us) write_op.cc:312] APPLY: finished
0709 12:12:38.809475 (+   242us) log.cc:844] Serialized 23 byte log entry
0709 12:12:38.809704 (+   229us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:38.810450 (+   746us) write_op.cc:454] Released schema lock
0709 12:12:38.810495 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:38.810867 (+   372us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:38.810901 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13806,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":222682,"apply.run_cpu_time_us":2655,"apply.run_wall_time_us":66213,"num_ops":1,"prepare.queue_time_us":246407,"prepare.run_cpu_time_us":61629,"prepare.run_wall_time_us":437619,"raft.queue_time_us":117861,"raft.run_cpu_time_us":8208,"raft.run_wall_time_us":169794,"replication_time_us":718580,"spinlock_wait_cycles":3840,"thread_start_us":601234,"threads_started":4,"wal-append.queue_time_us":222667}]]}
I20260709 12:12:39.256893 12676 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:39.407935 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.408802 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:39.255295 12763 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.336194 12698 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:39.374912 12687 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.388687 12724 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.351649 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.336941 12685 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.387660 12758 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.412566 12666 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:39.505103 12783 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:12:39.765178 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:39.793951 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.793704 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.795141 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:39.795331 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.424053 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:39.796614 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:39.799023 11255 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:39.548910 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:39.802230 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.803098 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.803550 11177 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:39.804127 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:39.804926 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.805385 11177 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:39.806092 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.792939 11935 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:39.408404 12671 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.814947 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:39.816840 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.818640 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.798491 11182 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:39.807825 11180 raft_consensus.cc:2368] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3.
I20260709 12:12:39.421279 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:39.851079 12709 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:39.858141 12635 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:39.858788 12635 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:39.859162 12635 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.859501 12635 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.860201 12758 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:39.862072 11174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1616) took 1488 ms (client timeout 1592 ms). Trace:
W20260709 12:12:39.862577 11174 rpcz_store.cc:269] 0709 12:12:38.373244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.373364 (+   120us) service_pool.cc:224] Handling call
0709 12:12:39.861863 (+1488499us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54000d7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:39.862000 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248484736}
I20260709 12:12:39.863849 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:39.789897 12677 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.601s	user 0.006s	sys 0.003s
I20260709 12:12:39.843284 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:39.848742 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56281.4us
I20260709 12:12:39.864879 12677 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.849280 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.849785 11252 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:39.825773 12687 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.814486 11926 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.498428 12753 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:39.519840 12752 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.846797 11173 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.501186 12609 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:39.498656 11927 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:12:39.422058 11930 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:39.843727 11256 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:39.861205 12758 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:39.791268 12742 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.798133 12711 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.810045 12707 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:39.870061 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 448781us
W20260709 12:12:39.872859 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.021s	sys 0.006s
I20260709 12:12:39.427729 12701 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:39.876650 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:39.991849 11249 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:39.878552 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.808991 12506 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:39.998057 12793 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:39.998288 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:39.999007 11182 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:39.998654 12793 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.999794 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:40.000702 11182 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:40.000898 12793 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:39.807099 12712 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3.
I20260709 12:12:39.600865 12584 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:39.811637 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:40.002219 12685 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.002439 12763 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:39.808270 12765 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:39.843603 12787 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:39.899101 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:39.843604 12651 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:39.842948 12698 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:12:39.868191 12737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.030s	sys 0.010s
I20260709 12:12:39.789829 12676 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.789969 12666 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.919504 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:39.922959 11173 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:39.826488 12662 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.826275 12724 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.843062 12741 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.929708 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.020438 11256 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:40.021593 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:40.022192 11256 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.907003 11178 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:39.842461 12619 raft_consensus.cc:359] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:39.926290 12742 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:39.914316 12735 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.847308 12671 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:39.909811 12663 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:39.827672 12786 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.923792 12609 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:40.002238 12584 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:39.905017 12723 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.879740 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:39.982712 11329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.020s	sys 0.028s
W20260709 12:12:39.982796 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.021s	sys 0.006s
I20260709 12:12:39.983569 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:40.099555 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 116099us
I20260709 12:12:40.100646 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:39.927338 12780 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:39.996126 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:40.102284 11329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.020s	sys 0.028s
I20260709 12:12:39.996232 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:40.103036 11249 raft_consensus.cc:2368] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3.
I20260709 12:12:39.891310 11255 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:40.104242 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.002487 11177 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:39.928300 12794 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:12:39.961285 12669 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.764s	user 0.004s	sys 0.005s
I20260709 12:12:39.960217 12745 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:40.001415 12793 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.016525 11935 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:40.016464 11324 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:40.118909 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.122970 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:39.959604 12747 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:39.991901 12652 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:12:40.011541 12756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.012s	sys 0.006s
W20260709 12:12:40.014643 12737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.031s	sys 0.010s
I20260709 12:12:40.027099 11257 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:40.027271 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:40.013080 12787 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.012760 12651 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.035383 11254 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:39.982401 12758 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.075459 11181 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:40.076452 11253 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:39.942431 12711 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:40.014904 12795 raft_consensus.cc:2764] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:12:40.105671 11177 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:40.144831 11177 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.146281 11177 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:40.026264 12619 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:40.036088 12677 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.016842 12698 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.014971 12765 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.133174 11256 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:40.127609 11328 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:40.165477 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:40.165824 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:40.166141 11328 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:40.119565 11255 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:40.097409 12612 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:40.140533 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 884431us
I20260709 12:12:40.102290 12709 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:12:40.118552 12744 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.799s	user 0.012s	sys 0.000s
I20260709 12:12:40.073539 12735 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.108176 12729 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:12:40.126907 12756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.012s	sys 0.007s
I20260709 12:12:40.117812 12796 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.126163 11325 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:40.096133 12663 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.206552 12663 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.122036 12794 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:40.208091 12794 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.211229 11182 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:40.211971 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:40.120025 11175 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:40.139799 12787 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:40.133030 12745 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.175002 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:40.177181 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:40.132016 12652 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.180387 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.143090 12651 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:40.121286 12669 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.141956 12762 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.142164 12687 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.149904 12737 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.118227 12780 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.175316 12701 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:40.215430 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:40.101745 12609 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:40.132129 12747 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.150223 12681 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:12:40.212616 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1152) took 1017 ms. Trace:
I20260709 12:12:40.186033 12709 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.219019 12763 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:40.221851 12666 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.181744 12753 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:40.179540 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.128149 12793 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:40.109987 12723 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.218750 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:40.186369 12744 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:40.521843 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.005s	sys 0.009s
W20260709 12:12:40.579973 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.007s	sys 0.010s
I20260709 12:12:40.522078 12724 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.482100 12671 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.380633 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1481) took 1177 ms. Trace:
W20260709 12:12:40.532581 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.014s	sys 0.034s
I20260709 12:12:40.219722 12756 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:40.584403 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.015s	sys 0.035s
I20260709 12:12:40.584573 12756 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:40.585033 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 678334us
I20260709 12:12:40.196089 12752 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.586930 12752 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.587119 12756 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:40.587188 12698 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.587908 12752 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.589569 12698 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.589267 12821 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.590504 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:40.583527 12819 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:12:40.288722 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 107533us
I20260709 12:12:40.608987 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:40.586174 12744 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:40.279237 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 178640us
I20260709 12:12:40.611424 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:40.434209 12677 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:40.610659 11182 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:40.591987 12701 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.612586 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:40.613510 11180 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:12:40.602840 11146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.000s	sys 0.014s
W20260709 12:12:40.626323 11146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.000s	sys 0.014s
W20260709 12:12:40.626806 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:40.449023 12801 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.279s	user 0.000s	sys 0.000s
W20260709 12:12:40.578152 12799 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 12:12:40.583303 11181 rpcz_store.cc:276] 0709 12:12:39.203342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.204622 (+  1280us) service_pool.cc:224] Handling call
0709 12:12:40.380406 (+1175784us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54000d8754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:40.380552 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974569344}
I20260709 12:12:40.618041 11179 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:40.590281 12698 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.430264 11177 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:40.630510 11900 leader_election.cc:400] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3.
I20260709 12:12:40.652189 11900 leader_election.cc:403] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:12:40.483639 12736 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.096s	user 0.002s	sys 0.009s
I20260709 12:12:40.609193 11255 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:12:40.484611 11926 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.002s
W20260709 12:12:40.481554 11252 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.001s
W20260709 12:12:40.637372 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51283.5us
W20260709 12:12:40.366551 12728 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.012s	sys 0.022s
I20260709 12:12:40.494195 12741 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.222754 12676 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.632720 11175 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:40.642320 11250 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:40.643865 12609 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:40.256958 12745 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.653221 11328 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:40.366454 12731 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.494772 12662 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.478722 12584 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.219192 12729 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:40.638501 11324 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:12:40.626997 12677 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:40.715049 12748 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.815s	user 0.007s	sys 0.005s
I20260709 12:12:40.715420 12677 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:40.715560 12748 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.716074 12652 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.716794 11327 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:40.659169 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:40.718364 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:40.718865 12793 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.719138 11255 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:40.655893 11256 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:40.613198 11927 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:40.184036 12635 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.683727 12817 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.683864 12808 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.674489 12736 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.633237 12821 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.655846 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:40.636090 12701 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:40.690727 12758 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:40.210409 12685 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.438s
I20260709 12:12:40.210986 12796 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:40.738628 12811 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:12:40.674383 12618 tablet_copy_client.cc:538] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:40.706440 12609 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:40.848601 12609 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.849124 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 317) took 1555 ms. Trace:
W20260709 12:12:40.675872 12728 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.015s	sys 0.027s
I20260709 12:12:40.699501 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:40.301421 12804 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.703022 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:40.656359 11926 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:40.859802 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:40.609076 12737 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.366499 12803 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.657106 12786 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.628875 12763 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.697592 12741 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.640687 12753 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:12:40.727382 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 345) took 1476 ms. Trace:
I20260709 12:12:40.688587 12819 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.696203 12676 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.847589 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:40.703709 11252 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:40.844521 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:40.850818 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 195036us
I20260709 12:12:40.902659 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:40.903290 11324 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:40.904964 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:40.905429 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:40.906250 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:40.847136 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:40.695376 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 483454us
I20260709 12:12:40.825281 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:40.728498 11076 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:40.907958 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:40.908995 12805 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:40.908860 11076 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 2 to 3, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:40.909477 12796 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.910320 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:40.911319 12807 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:40.910748 11076 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:40.911931 12807 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:12:40.827023 11257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1460) took 1330 ms (client timeout 1688 ms). Trace:
I20260709 12:12:40.743464 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 160) took 1424 ms. Trace:
I20260709 12:12:40.913723 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:40.912992 11329 rpcz_store.cc:276] 0709 12:12:39.318756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.330016 (+ 11260us) service_pool.cc:224] Handling call
0709 12:12:39.330308 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:39.330699 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:39.330727 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:39.349128 (+ 18401us) log.cc:844] Serialized 26 byte log entry
0709 12:12:40.102836 (+753708us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:40.102877 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.102943 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.743147 (+640204us) raft_consensus.cc:1685] finished
0709 12:12:40.743180 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.743382 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":49516,"thread_start_us":687112,"threads_started":1,"wal-append.queue_time_us":742068}
I20260709 12:12:40.914527 12612 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:40.915117 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:40.916002 11329 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:40.915915 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:40.856801 11328 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
W20260709 12:12:40.863535 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1453) took 1524 ms (client timeout 1970 ms). Trace:
W20260709 12:12:40.917202 11253 rpcz_store.cc:269] 0709 12:12:39.328787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.328941 (+   154us) service_pool.cc:224] Handling call
0709 12:12:40.826272 (+1497331us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54001cedd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:40.852934 (+ 26662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633963264}
I20260709 12:12:40.917737 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:40.918651 11328 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:40.919062 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:40.726838 12809 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:12:40.685598 12824 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.922250 12824 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.923287 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1487) took 1712 ms. Trace:
I20260709 12:12:40.923902 11182 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
I20260709 12:12:40.712400 12735 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.913003 12807 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.867863 12709 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:40.868968 12609 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.704954 12765 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:40.742885 12707 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.796s	user 0.012s	sys 0.000s
I20260709 12:12:40.928422 12707 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.930244 12824 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.931339 12707 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.932281 12707 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:40.932894 12707 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:40.874577 11331 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:12:40.934435 11251 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:40.875283 11933 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:40.823465 12752 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.877800 12815 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:40.935511 12815 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.759878 12780 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.211782 12619 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:40.938752 12815 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.939450 12619 ts_tablet_manager.cc:1434] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 1.766s	user 0.019s	sys 0.019s
W20260709 12:12:40.939917 12619 ts_tablet_manager.cc:1467] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Tablet startup took 1766ms
I20260709 12:12:40.739015 12666 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:40.881888 12674 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.366s	user 0.024s	sys 0.060s
I20260709 12:12:40.811968 12669 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.813172 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:40.942051 12666 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:40.943043 11930 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:40.727428 12745 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:40.947114 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:12:40.861693 11325 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.657s	user 0.021s	sys 0.000s
I20260709 12:12:40.948335 11325 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:40.860807 12652 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:40.860445 12756 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:40.895401 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:40.864701 12762 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3.
I20260709 12:12:40.709111 12731 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.954241 11932 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:40.954052 12756 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:40.955601 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:40.955930 12731 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:40.956302 12756 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:40.942075 12674 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.427s	user 0.024s	sys 0.060s
I20260709 12:12:40.921057 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.957051 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:40.923854 11176 rpcz_store.cc:276] 0709 12:12:39.210977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.211536 (+   559us) service_pool.cc:224] Handling call
0709 12:12:39.211829 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:39.589392 (+377563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:39.589440 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:39.765505 (+176065us) log.cc:844] Serialized 26 byte log entry
0709 12:12:40.133571 (+368066us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:40.133613 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.133731 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.922955 (+789224us) raft_consensus.cc:1685] finished
0709 12:12:40.922987 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.923182 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24535,"raft.queue_time_us":51626,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"thread_start_us":603996,"threads_started":2,"wal-append.queue_time_us":511231}
I20260709 12:12:40.958287 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.958542 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:40.959168 11325 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:40.945083 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:40.960072 11249 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:40.960860 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:40.940315 12619 ts_tablet_manager.cc:1469] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Trace:
0709 12:12:33.408912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.409057 (+   145us) service_pool.cc:224] Handling call
0709 12:12:33.487213 (+ 78156us) ts_tablet_manager.cc:934] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
0709 12:12:34.927120 (+1439907us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:35.506761 (+579641us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:35.506903 (+   142us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:36.072090 (+565187us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:36.072397 (+   307us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:36.072694 (+   297us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:36.072755 (+    61us) inbound_call.cc:177] Queueing success response
0709 12:12:36.612215 (+539460us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b4c00197500. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec49b8a20 00007f3ec49c9a29 00007f3ec49ca13f 00007f3ec498e2ab 00007f3ec49bbf3c 00007f3ec49bc461 00007f3ec49bc768 00007f3eca74266b 00007f3ecb67f15b 00007f3ecb67ef98 00007f3ecb670322
0709 12:12:37.789533 (+1177318us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:12:39.172990 (+1383457us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:12:39.416082 (+243092us) tablet_replica.cc:261] Starting instrumentation
0709 12:12:39.416748 (+   666us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":49402,"raft.queue_time_us":7409,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":150,"spinlock_wait_cycles":220298496,"tablet-copy.queue_time_us":76742,"tablet-download-pool.queue_time_us":1223,"tablet-download-pool.run_cpu_time_us":5234,"tablet-download-pool.run_wall_time_us":805348,"thread_start_us":141596,"threads_started":2}
W20260709 12:12:40.962705 11257 rpcz_store.cc:269] 0709 12:12:39.496991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.795100 (+298109us) service_pool.cc:224] Handling call
0709 12:12:40.826793 (+1031693us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540010b154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:40.826941 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530902400}
I20260709 12:12:40.964543 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.959462 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.965404 11257 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:40.965663 11253 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:40.966699 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.945492 12666 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:40.967003 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:40.971101 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.972800 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:40.868587 12803 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.968370 11932 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:40.968079 11930 raft_consensus.cc:2376] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3. Re-sending same reply.
I20260709 12:12:40.874122 12758 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.978252 11076 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:40.738595 12810 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:40.973555 11253 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:40.982899 11933 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:40.880828 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 343) took 1656 ms. Trace:
I20260709 12:12:40.882715 12662 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.980471 11257 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:40.991245 11076 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:40.893421 12687 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:40.909441 12805 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:40.996767 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:40.742376 12729 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.953418 12652 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:12:41.000077 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53061.5us
W20260709 12:12:40.987952 12643 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:12:41.006454 11257 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:41.011480 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:40.882364 12741 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.020617 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.002285 11076 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 2 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:12:41.036028 11324 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:41.036969 11076 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 2 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:12:41.040484 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:41.036600 12698 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.005277 12747 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:40.874701 12808 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.053996 11328 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:41.040534 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:41.008204 12805 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.007977 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:40.984594 12676 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.007472 12794 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.882910 12817 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.007684 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 114) took 3336 ms. Trace:
I20260709 12:12:40.870054 12804 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:40.612234 11248 rpcz_store.cc:276] 0709 12:12:39.194962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.197596 (+  2634us) service_pool.cc:224] Handling call
0709 12:12:39.207431 (+  9835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:39.207635 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:12:39.575555 (+367920us) log.cc:844] Serialized 14 byte log entry
0709 12:12:40.149096 (+573541us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:40.157228 (+  8132us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:40.170707 (+ 13479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.182306 (+ 11599us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.203259 (+ 20953us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26666,"prepare.queue_time_us":103299,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":6086,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":13695,"thread_start_us":632338,"threads_started":2,"wal-append.queue_time_us":610252}
I20260709 12:12:41.008494 12786 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:41.020557 12756 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:41.013123 12728 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.001788 12723 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.056847 12609 raft_consensus.cc:2764] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:12:41.050794 12681 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:40.893913 11928 rpcz_store.cc:276] 0709 12:12:39.251262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.254103 (+  2841us) service_pool.cc:224] Handling call
0709 12:12:39.307744 (+ 53641us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7eee 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:40.719822 (+1412078us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7f8c 00007f3ec35c7ef6 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:40.726487 (+  6665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:40.726765 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:40.726806 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:40.726976 (+   170us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:40.727021 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.727098 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.727263 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134181888}
I20260709 12:12:40.999006 12758 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:41.008752 12662 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.075331 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:41.012778 12810 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.085150 12698 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:41.114178 12652 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.082738 12651 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:40.999570 11926 rpcz_store.cc:276] 0709 12:12:39.214590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.218149 (+  3559us) service_pool.cc:224] Handling call
0709 12:12:40.870668 (+1652519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.139006 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.082284 12795 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:41.045325 12707 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.106447 12745 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:12:41.118445 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:41.167004 11328 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:41.142580 12698 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.087658 11931 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:41.081204 12818 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.206712 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 337) took 3036 ms. Trace:
I20260709 12:12:41.207428 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:41.136305 12711 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.981s	user 0.008s	sys 0.004s
I20260709 12:12:41.105494 12821 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.094893 12786 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:41.133965 12763 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.164783 12681 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:41.166738 12795 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:41.160930 12666 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.137362 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:41.135452 12737 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:41.162946 12758 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:41.129300 12736 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.234627 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1635) took 2308 ms. Trace:
I20260709 12:12:41.206493 12818 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.161712 12817 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.216398 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:41.220602 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.220842 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.221208 12780 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:40.849653 11929 rpcz_store.cc:276] 0709 12:12:39.293531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.915307 (+621776us) service_pool.cc:224] Handling call
0709 12:12:39.946143 (+ 30836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:39.946667 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:39.946713 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:39.947278 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:12:40.347535 (+400257us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:40.366382 (+ 18847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.405565 (+ 39183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.848860 (+443295us) raft_consensus.cc:1685] finished
0709 12:12:40.848889 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.849052 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4469,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7535,"thread_start_us":330829,"threads_started":1,"wal-append.queue_time_us":664122}
I20260709 12:12:41.205718 12698 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.100315 12803 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:41.206931 12676 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.093225 12794 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.230983 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.106719 12756 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.083449 11252 rpcz_store.cc:276] 0709 12:12:37.663921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.665886 (+  1965us) service_pool.cc:224] Handling call
0709 12:12:37.688602 (+ 22716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:39.029131 (+1340529us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54001b2c20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6663dc0 00007f3ec66ee1c0 00007f3ec66ef231 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c
0709 12:12:40.903843 (+1874712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.996884 (+ 93041us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527395584}
I20260709 12:12:41.224404 11323 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:41.235107 11331 rpcz_store.cc:276] 0709 12:12:38.922763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.923236 (+   473us) service_pool.cc:224] Handling call
0709 12:12:39.300399 (+377163us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:39.300594 (+   195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:40.933878 (+1633284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.230804 (+296926us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400285ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:41.231030 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3643666944}
I20260709 12:12:41.203862 12745 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.223946 12681 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.243773 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:41.223958 12786 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.230145 12795 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:41.246021 12817 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:41.205185 12728 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:41.213163 12612 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:12:41.261861 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 345996us
I20260709 12:12:41.263726 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:41.232548 12741 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.249336 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1048) took 1221 ms. Trace:
I20260709 12:12:41.212951 12711 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.236805 12804 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:41.273383 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.274050 11324 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:41.276144 12808 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.278595 12818 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:41.282662 12803 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:41.208709 11931 rpcz_store.cc:276] 0709 12:12:38.170330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.174684 (+  4354us) service_pool.cc:224] Handling call
0709 12:12:39.301046 (+1126362us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7eee 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:41.022400 (+1721354us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7f8c 00007f3ec35c7ef6 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:41.044914 (+ 22514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.206376 (+161462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.206618 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94254208}
W20260709 12:12:41.286675 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1150) took 2093 ms (client timeout 2199 ms). Trace:
I20260709 12:12:41.286609 12819 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.292754 12756 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:41.293923 11327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1626) took 2438 ms (client timeout 2715 ms). Trace:
W20260709 12:12:41.294497 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 446973us
I20260709 12:12:41.232494 12666 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:41.304646 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60953.5us
I20260709 12:12:41.264971 11248 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:41.310046 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:41.303222 12677 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.004s
W20260709 12:12:41.303464 12827 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 12:12:41.314416 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50774.4us
I20260709 12:12:41.352833 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:41.354058 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:41.317871 12747 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:41.265661 12780 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.315220 11329 rpcz_store.cc:276] 0709 12:12:40.028215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.248435 (+1220220us) service_pool.cc:224] Handling call
0709 12:12:41.248781 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.248984 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.249012 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.249058 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:41.249091 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.249162 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.249276 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.315460 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-4a634acd9ff543cca551c8a28e72d6c4 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:41.329255 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.360920 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:41.361548 11324 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.307507 12821 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:41.305387 12817 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.312425 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1046) took 1290 ms. Trace:
I20260709 12:12:41.360740 12796 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:41.364900 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.365139 12821 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:12:41.328729 11250 rpcz_store.cc:269] 0709 12:12:39.193273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.197522 (+  4249us) service_pool.cc:224] Handling call
0709 12:12:41.286437 (+2088915us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400068454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:41.286605 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3036204928}
I20260709 12:12:41.369366 12817 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.382861 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:41.383608 11253 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:41.349036 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 455355us
I20260709 12:12:41.317575 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.348454 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:41.374578 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.365494 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.368402 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.328840 12803 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.434237 11329 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
W20260709 12:12:41.436939 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51347.2us
W20260709 12:12:41.377939 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1458) took 1950 ms (client timeout 2094 ms). Trace:
I20260709 12:12:41.376562 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.439556 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:41.382033 11147 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.199s (SENT) [suppressed 19 similar messages]
I20260709 12:12:41.385646 11325 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:41.441026 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:41.354928 12756 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:12:41.439317 11254 rpcz_store.cc:269] 0709 12:12:39.427121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.818308 (+391187us) service_pool.cc:224] Handling call
0709 12:12:41.377746 (+1559438us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001dd3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:41.377883 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441300864}
I20260709 12:12:41.445592 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.449712 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:41.451314 11325 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:41.448066 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.446185 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:41.456496 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:41.458284 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:41.453300 12711 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.359579 12762 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:41.444648 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:41.357890 12677 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.369415 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-4a634acd9ff543cca551c8a28e72d6c4 on tablet 4a634acd9ff543cca551c8a28e72d6c4 with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:41.459698 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:41.461146 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.455421 11330 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:41.465102 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.373389 12821 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.467211 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.367769 12662 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.433213 12794 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.466320 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:41.466758 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:41.367151 12835 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.385462 12810 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.465862 11254 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
W20260709 12:12:41.463333 12742 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.380s	user 0.014s	sys 0.000s
I20260709 12:12:41.367472 12793 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:12:41.342010 11327 rpcz_store.cc:269] 0709 12:12:38.855424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.908780 (+ 53356us) service_pool.cc:224] Handling call
0709 12:12:41.293569 (+2384789us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54004290d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:41.293829 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3353742208}
I20260709 12:12:41.481242 11222 leader_election.cc:400] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3.
I20260709 12:12:41.373888 12819 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:41.375478 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.455024 12756 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:41.456687 12818 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:41.470909 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:41.499430 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:41.456717 12804 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:12:41.506497 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 124484us
I20260709 12:12:41.506735 12804 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:41.507030 12814 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:41.507365 12804 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.464504 12736 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:41.482671 12747 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:41.503373 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1637) took 1401 ms. Trace:
I20260709 12:12:41.437467 12796 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:12:41.513978 11932 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:12:41.461898 12830 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:41.515808 12830 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:41.516311 12830 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:41.516419 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45568.4us
I20260709 12:12:41.520038 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:41.520116 12729 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:41.481009 12669 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:41.521334 12794 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:41.501011 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:41.367442 12612 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.521867 12794 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:41.522759 12794 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.489012 12762 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:41.455073 12765 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:12:41.486438 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.521231 12669 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.509017 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:41.369998 11323 rpcz_store.cc:276] 0709 12:12:40.015935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.016810 (+   875us) service_pool.cc:224] Handling call
0709 12:12:41.045237 (+1028427us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:41.306646 (+261409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11645184}
I20260709 12:12:41.526160 12736 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:12:41.531708 12831 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:12:41.505386 12793 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:41.532812 12787 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.311s	user 0.006s	sys 0.012s
I20260709 12:12:41.512163 11330 rpcz_store.cc:276] 0709 12:12:40.101953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.502538 (+1400585us) service_pool.cc:224] Handling call
0709 12:12:41.502852 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.503023 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.503047 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.503085 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:41.503125 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.503178 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.503319 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.533807 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.556514 11257 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:41.508035 12756 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:41.558935 12736 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:41.505682 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:41.512181 12747 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:41.551592 12728 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.551956 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.564402 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:41.504217 12742 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.508457 12737 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:12:41.528225 12745 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:41.561305 12780 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.546147 12669 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.507822 12819 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.565689 11326 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:41.540539 12711 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:12:41.495988 12835 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.575533 11932 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:41.559371 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.570943 12786 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.600867 11250 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:41.582406 12794 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:41.602442 12786 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.603286 11180 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:12:41.593047 12838 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:12:41.565704 12747 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.569204 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1644) took 1465 ms. Trace:
I20260709 12:12:41.606047 11330 rpcz_store.cc:276] 0709 12:12:40.103716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.563864 (+1460148us) service_pool.cc:224] Handling call
0709 12:12:41.568508 (+  4644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.568761 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.568802 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.568864 (+    62us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:41.568913 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.568995 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.569128 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.608058 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1645) took 1504 ms. Trace:
I20260709 12:12:41.608670 11253 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:41.608546 11330 rpcz_store.cc:276] 0709 12:12:40.103951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.607085 (+1503134us) service_pool.cc:224] Handling call
0709 12:12:41.607417 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.607615 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.607641 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.607705 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:41.607753 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.607835 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.607989 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.612905 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1643) took 1509 ms. Trace:
I20260709 12:12:41.612802 12506 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.613400 11323 rpcz_store.cc:276] 0709 12:12:40.103487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.566179 (+1462692us) service_pool.cc:224] Handling call
0709 12:12:41.609433 (+ 43254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.612324 (+  2891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.612362 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.612460 (+    98us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:41.612497 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.612579 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.612844 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.583559 12803 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.624500 12742 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:41.625526 12805 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.006s
I20260709 12:12:41.625981 12677 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:41.625976 12805 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.626627 12748 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:41.627666 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1449) took 2303 ms (client timeout 1842 ms). Trace:
I20260709 12:12:41.628104 12805 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.622920 12793 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:41.628077 11251 rpcz_store.cc:269] 0709 12:12:39.324006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.325765 (+  1759us) service_pool.cc:224] Handling call
0709 12:12:41.627425 (+2301660us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400087ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:41.627565 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2628881280}
I20260709 12:12:41.620792 12817 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.629880 11329 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.623142 12804 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.614382 12506 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.627874 12795 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.623564 12812 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:41.639410 11330 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:41.658214 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1647) took 1076 ms. Trace:
I20260709 12:12:41.658704 11330 rpcz_store.cc:276] 0709 12:12:40.581502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.638477 (+1056975us) service_pool.cc:224] Handling call
0709 12:12:41.638931 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.657874 (+ 18943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.658142 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.655812 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:41.623797 12701 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.758s	user 0.009s	sys 0.000s
I20260709 12:12:41.653337 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:41.660769 11324 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:41.561902 12787 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.662142 11329 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:41.679371 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:41.664094 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1648) took 1081 ms. Trace:
W20260709 12:12:41.716372 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 196376us
I20260709 12:12:41.663373 11932 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.698746 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1639) took 1595 ms. Trace:
I20260709 12:12:41.674793 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:41.716259 11330 rpcz_store.cc:276] 0709 12:12:40.582217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.663085 (+1080868us) service_pool.cc:224] Handling call
0709 12:12:41.663465 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.663685 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.663723 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.663779 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:41.663821 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.663893 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.664025 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.605232 12698 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.654295 12812 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:41.753921 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.721794 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.607404 12681 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.663424 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:41.720149 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45478.9us
I20260709 12:12:41.679976 12786 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:41.760442 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:41.760708 11177 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:41.761431 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:41.765997 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.769330 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.770563 11147 leader_election.cc:400] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:41.766881 11179 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:41.771064 11147 leader_election.cc:403] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:41.775277 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:41.783202 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 192298us
I20260709 12:12:41.784763 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.719053 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:41.722568 12676 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.785097 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:41.756104 12763 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:12:41.794229 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4389713999999003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 12:12:41.795248 11174 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:41.680403 12701 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.784384 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.779139 12595 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:41.797564 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:41.810601 11322 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:41.784560 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1646) took 1674 ms. Trace:
I20260709 12:12:41.818152 12595 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.818643 12595 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.819661 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.820516 11178 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:41.801010 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:41.770336 12786 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.808220 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:41.804230 12812 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:41.831212 12812 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.785392 11181 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:41.832966 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.778915 12788 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:41.808120 12763 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.795123 12669 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:41.825917 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1042) took 1828 ms (client timeout 2199 ms). Trace:
I20260709 12:12:41.808652 12729 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.843191 12818 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.843695 12818 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:41.808699 12752 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:41.827399 12786 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.806949 12676 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:41.795276 12762 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.834510 11181 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:41.850222 11324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:41.850703 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.844375 12618 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:12:41.810891 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:41.829356 11927 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.963s	user 0.005s	sys 0.011s
I20260709 12:12:41.869259 12669 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.813141 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1640) took 1710 ms. Trace:
I20260709 12:12:41.836678 12788 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.824158 12711 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:41.818670 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:12:41.840350 11322 rpcz_store.cc:269] 0709 12:12:39.997589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.016320 (+ 18731us) service_pool.cc:224] Handling call
0709 12:12:41.654500 (+1638180us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7e9c 00007f3ec35c7a5b 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:41.825830 (+171330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11536640}
I20260709 12:12:41.828452 12819 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.851532 11181 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:41.884897 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.838346 12763 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.822846 11323 rpcz_store.cc:276] 0709 12:12:40.104182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.614474 (+1510292us) service_pool.cc:224] Handling call
0709 12:12:41.657272 (+ 42798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.698052 (+ 40780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.718645 (+ 20593us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.718728 (+    83us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:41.722107 (+  3379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.759398 (+ 37291us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.778072 (+ 18674us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:41.838873 12835 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.849879 11926 raft_consensus.cc:2368] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:41.830204 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:41.872576 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:41.852305 11931 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 2: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:41.873756 12846 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:41.839100 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:41.845955 12752 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:41.881887 11932 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:41.800906 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:41.889000 11181 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:41.870556 11927 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:41.883380 12788 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.879492 12804 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:41.903318 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64323us
I20260709 12:12:41.866501 12762 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:41.909719 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 108896us
I20260709 12:12:41.888960 11928 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:41.912755 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:41.915468 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1154) took 1304 ms. Trace:
I20260709 12:12:41.918761 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:41.932410 11255 rpcz_store.cc:276] 0709 12:12:40.611516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.974857 (+363341us) service_pool.cc:224] Handling call
0709 12:12:40.975164 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.035367 (+ 60203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.035394 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.035433 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:41.035461 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.915231 (+879770us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b5400109854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:41.915275 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.915408 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110133504}
I20260709 12:12:41.942106 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:41.906036 11178 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:41.943570 11932 raft_consensus.cc:2368] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:41.909350 12752 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:41.949724 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:41.954234 11181 raft_consensus.cc:2368] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:41.908977 12698 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:41.963594 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:41.964725 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 351) took 1332 ms. Trace:
I20260709 12:12:41.916975 12681 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:41.965183 11935 rpcz_store.cc:276] 0709 12:12:40.631895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.632815 (+   920us) service_pool.cc:224] Handling call
0709 12:12:41.963392 (+1330577us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:41.964641 (+  1249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17095424}
I20260709 12:12:41.923909 12796 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:12:41.923990 11927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 428) took 1740 ms (client timeout 1578 ms). Trace:
I20260709 12:12:41.904424 12747 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:41.973615 12853 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:12:41.979535 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:41.997778 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1463) took 1823 ms. Trace:
I20260709 12:12:41.925102 12846 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.000912 11249 rpcz_store.cc:276] 0709 12:12:40.173850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.180113 (+  6263us) service_pool.cc:224] Handling call
0709 12:12:41.997553 (+1817440us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54002a6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:41.997704 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874873600}
W20260709 12:12:42.013393 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 250645us
I20260709 12:12:42.019169 12506 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.019694 12506 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:41.979681 11927 rpcz_store.cc:269] 0709 12:12:40.176703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.181792 (+  5089us) service_pool.cc:224] Handling call
0709 12:12:41.911969 (+1730177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:42.021780 12506 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:42.021715 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 102321us
I20260709 12:12:42.023247 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.035514 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.021209 11929 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:42.044272 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:42.015213 12848 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:12:42.021915 12821 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.533s	user 0.003s	sys 0.013s
I20260709 12:12:42.048447 12506 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:42.044947 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.069307 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:42.069659 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:42.036947 12830 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.003s
I20260709 12:12:42.106356 12506 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:42.046000 12829 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:42.049580 12677 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:42.066918 12780 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:42.177155 12506 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:42.175804 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 104714us
I20260709 12:12:42.072077 12742 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:42.070271 12848 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:42.044442 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:42.086242 12803 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:42.172333 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:42.196911 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 166) took 1291 ms. Trace:
I20260709 12:12:42.067370 12674 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
W20260709 12:12:42.175740 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 83943.1us
I20260709 12:12:42.174705 12829 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:42.086400 12796 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:42.077878 12821 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.199978 12729 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:42.179883 12677 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:42.200821 12747 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.185115 12780 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:42.178581 12830 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.188759 12609 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:42.198661 12848 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:42.201835 11180 rpcz_store.cc:276] 0709 12:12:40.905205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.906938 (+  1733us) service_pool.cc:224] Handling call
0709 12:12:42.196617 (+1289679us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54000d9b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.196825 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377758592}
I20260709 12:12:41.878189 11327 rpcz_store.cc:276] 0709 12:12:40.102732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.528963 (+1426231us) service_pool.cc:224] Handling call
0709 12:12:41.555486 (+ 26523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.582873 (+ 27387us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:12:41.652875 (+ 70002us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.783428 (+130553us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.793830 (+ 10402us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:41.798501 (+  4671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.805579 (+  7078us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.811300 (+  5721us) inbound_call.cc:177] Queueing success response
0709 12:12:41.877240 (+ 65940us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b180002d880. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
Metrics: {"prepare.queue_time_us":31314,"prepare.run_wall_time_us":52,"raft.queue_time_us":23683,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"spinlock_wait_cycles":40710400,"thread_start_us":73401,"threads_started":1,"wal-append.queue_time_us":129755}
I20260709 12:12:42.212467 12786 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.213830 11178 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:42.194520 12742 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:42.198366 12803 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:12:42.225759 11933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 427) took 2049 ms (client timeout 1545 ms). Trace:
I20260709 12:12:42.226441 11934 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:12:42.208321 12829 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.210268 12762 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.225903 12677 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:42.228408 12780 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:42.252637 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:42.236007 11933 rpcz_store.cc:269] 0709 12:12:40.176302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.176864 (+   562us) service_pool.cc:224] Handling call
0709 12:12:42.225537 (+2048673us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400667bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.225690 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508400384}
I20260709 12:12:42.238173 11296 leader_election.cc:400] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:42.253788 11296 leader_election.cc:403] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:12:42.232291 11927 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:42.237669 12742 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.259068 11296 leader_election.cc:400] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:42.260160 11296 leader_election.cc:403] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:12:42.238440 12803 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.261443 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:42.231927 12609 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.264595 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:42.267516 12609 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:42.268425 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:42.268748 12595 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:42.269155 12742 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.270849 11182 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:42.272154 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:42.272948 11933 raft_consensus.cc:2376] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4. Re-sending same reply.
I20260709 12:12:42.270630 12612 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:42.274017 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:42.274557 12846 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.274714 11926 raft_consensus.cc:2376] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4. Re-sending same reply.
I20260709 12:12:42.263501 12859 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.275632 12859 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.276604 12786 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.278467 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:42.279820 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:42.281280 12635 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:42.281746 12595 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.282300 12804 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:42.269160 11178 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:42.282903 12804 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:42.283880 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:42.283983 12804 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.285924 12865 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:42.287760 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:42.288420 11177 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.288558 11928 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:12:42.257818 12737 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:42.285282 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:42.290896 11178 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:42.291152 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:42.290258 12857 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.291939 11179 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
W20260709 12:12:42.238757 12756 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.004s
I20260709 12:12:42.294924 12821 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:42.269256 12796 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.341023 12742 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.353446 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:42.356303 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 160) took 1455 ms. Trace:
W20260709 12:12:42.357945 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 78216.3us
I20260709 12:12:42.361353 11177 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:42.365232 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:42.366248 11932 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.294317 12846 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:12:42.350605 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 166) took 1051 ms. Trace:
I20260709 12:12:42.315025 12756 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:42.351343 12794 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.002s
I20260709 12:12:42.315392 12788 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.263280 12805 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:42.346750 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:42.352167 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:42.373705 12506 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.385360 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 124) took 1022 ms. Trace:
I20260709 12:12:42.385562 12506 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.385851 11256 rpcz_store.cc:276] 0709 12:12:41.362666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.467974 (+105308us) service_pool.cc:224] Handling call
0709 12:12:42.346564 (+878590us) spinlock_profiling.cc:243] Waited 7.29 ms on lock 0x7b5800196898. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:42.385292 (+ 38728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16766720}
I20260709 12:12:42.374181 11296 leader_election.cc:400] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:42.376670 11180 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:42.378614 11176 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:42.386868 11296 leader_election.cc:403] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:12:42.388952 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.369004 11329 rpcz_store.cc:276] 0709 12:12:41.298933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.445934 (+147001us) service_pool.cc:224] Handling call
0709 12:12:42.350462 (+904528us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002855d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.350552 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396011264}
I20260709 12:12:42.374029 11933 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:42.368718 12793 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.390084 11179 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:42.382424 12865 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:42.392035 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.392215 12506 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.398975 11932 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:42.378551 12830 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.398591 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.391111 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:42.401846 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:42.402100 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.402643 11181 raft_consensus.cc:2368] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:42.403014 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.403729 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:42.404318 11180 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.404151 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.406067 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:42.352377 12857 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.398257 12742 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:42.384964 12848 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.407178 12612 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:42.405731 12635 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:42.382910 12677 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.399793 12807 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:42.378974 12763 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.418152 12612 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:42.419131 12763 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.401361 12868 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:42.403359 12805 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:42.407420 12737 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:42.408289 12794 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:42.410579 12666 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.414956 12830 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:42.430876 11180 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:42.403050 12793 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.419457 12612 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.416636 12635 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:42.436676 11181 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:42.436836 11296 leader_election.cc:400] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:42.420006 12807 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:42.407542 12752 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.437692 11296 leader_election.cc:403] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:12:42.426090 12805 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:42.428983 12814 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.356769 11175 rpcz_store.cc:276] 0709 12:12:40.900767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.974400 (+ 73633us) service_pool.cc:224] Handling call
0709 12:12:41.913882 (+939482us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:41.949269 (+ 35387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.967723 (+ 18454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.967766 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.972601 (+  4835us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:41.982031 (+  9430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.988474 (+  6443us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.356076 (+367602us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b54002e4ad0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.356231 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80723584}
I20260709 12:12:42.411813 12865 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.431517 12737 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:42.436200 11178 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:42.444010 12677 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.423249 12868 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:42.437750 12635 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:42.439682 12807 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.446938 12788 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.436589 12794 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:42.459717 12794 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.459821 12737 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.438408 12830 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:42.446041 12805 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:42.466607 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 342) took 3266 ms. Trace:
I20260709 12:12:42.467036 11934 rpcz_store.cc:276] 0709 12:12:39.200186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.200330 (+   144us) service_pool.cc:224] Handling call
0709 12:12:39.305823 (+105493us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:40.656124 (+1350301us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7e9c 00007f3ec35c7a5b 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:42.226204 (+1570080us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:42.236754 (+ 10550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.466406 (+229652us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400667bd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.466547 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3717646336}
I20260709 12:12:42.469683 12651 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.473534 12758 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:12:42.465353 11174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 156) took 2867 ms (client timeout 2312 ms). Trace:
I20260709 12:12:42.452838 12846 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.469340 12830 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.461918 12756 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.451711 12814 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:42.474310 11174 rpcz_store.cc:269] 0709 12:12:39.597076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.863560 (+266484us) service_pool.cc:224] Handling call
0709 12:12:42.463760 (+2600200us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b540016eed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.463933 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4419172096}
I20260709 12:12:42.475677 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1050) took 2446 ms. Trace:
I20260709 12:12:42.474057 12752 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.479017 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.488731 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.490048 11324 raft_consensus.cc:606] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:42.490456 11324 raft_consensus.cc:614] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:42.486665 11331 rpcz_store.cc:276] 0709 12:12:40.028805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.288920 (+1260115us) service_pool.cc:224] Handling call
0709 12:12:41.289351 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.289621 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:12:41.362064 (+ 72443us) log.cc:844] Serialized 14 byte log entry
0709 12:12:42.472903 (+1110839us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b18001599a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a
0709 12:12:42.473179 (+   276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:42.473249 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:42.473313 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.473438 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.475599 (+  2161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14515,"prepare.queue_time_us":202363,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":994,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":133,"spinlock_wait_cycles":3656960,"thread_start_us":192037,"threads_started":2,"wal-append.queue_time_us":192555}
I20260709 12:12:42.491730 12848 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.492167 12814 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.491956 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.480827 12803 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.482697 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.494071 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.501276 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.495862 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:42.458623 12868 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.504462 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.505267 11256 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:12:42.505781 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.507694 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.507963 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:42.508862 11331 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:42.511265 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-86c5246d860d4d559ea665671cc46b5c received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:42.511765 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-86c5246d860d4d559ea665671cc46b5c on tablet 86c5246d860d4d559ea665671cc46b5c with peer 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:12:42.504400 12818 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.011s
I20260709 12:12:42.515550 11249 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:42.517138 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.517323 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:42.518100 11249 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:42.520449 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:42.515162 12818 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.527232 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:42.528235 11249 raft_consensus.cc:2376] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4. Re-sending same reply.
W20260709 12:12:42.534574 11326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1642) took 2429 ms (client timeout 2000 ms). Trace:
W20260709 12:12:42.535094 11326 rpcz_store.cc:269] 0709 12:12:40.103231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.679353 (+576122us) service_pool.cc:224] Handling call
0709 12:12:42.532537 (+1853184us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5400459cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.532749 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624162176}
I20260709 12:12:42.536331 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.575017 11331 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:42.474586 12829 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:42.636471 11322 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:42.637578 12701 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.644088 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:42.644819 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:42.644011 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:42.732316 12857 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.734618 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.698845 12859 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.762219 12803 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:42.751149 12635 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:42.801419 12780 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.006s
I20260709 12:12:42.807860 12817 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.812857 12818 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.813159 12780 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.815105 12742 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:42.815930 12780 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.812814 12635 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.815977 12805 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:42.821440 12846 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:42.829175 12805 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.832965 12868 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.846025 12807 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.849120 12807 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:42.938198 12651 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.928524 12506 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.921142 11330 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:42.913143 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:42.905424 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1515) took 1096 ms. Trace:
I20260709 12:12:42.904134 11327 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:42.850857 12868 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:42.924795 11935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.009s	sys 0.004s
W20260709 12:12:42.985988 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.003s	sys 0.021s
W20260709 12:12:42.987785 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.004s	sys 0.021s
I20260709 12:12:42.960932 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37422 (request call id 11) took 1126 ms. Trace:
I20260709 12:12:42.961971 12865 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.966253 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1513) took 1166 ms. Trace:
I20260709 12:12:42.975922 12805 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:42.986109 11935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.009s	sys 0.017s
I20260709 12:12:42.919849 11328 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:43.001601 12805 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:42.989794 11179 rpcz_store.cc:276] 0709 12:12:41.799403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.295097 (+495694us) service_pool.cc:224] Handling call
0709 12:12:42.356175 (+ 61078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:42.398934 (+ 42759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.965667 (+566733us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b54001f3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:42.966066 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77499136}
I20260709 12:12:42.988962 11324 rpcz_store.cc:276] 0709 12:12:41.835254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.837451 (+  2197us) service_pool.cc:224] Handling call
0709 12:12:42.960374 (+1122923us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400331dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e6853 00007f3ecb6b460a 00007f3ec4c3a7d4 00007f3ec4c3a762 00007f3ec4c3a691 00007f3ec4c3a60c 00007f3ec4c398c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:42.960585 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469690624}
I20260709 12:12:43.019238 11927 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Advancing to term 2
I20260709 12:12:43.020062 11174 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:43.008154 12651 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.014480 12506 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.026219 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-6d225f8a873c4abe992018eb9353664f received from {username='slave'} at 127.0.0.1:59670
I20260709 12:12:43.026399 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.027020 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-6d225f8a873c4abe992018eb9353664f on tablet 6d225f8a873c4abe992018eb9353664f with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:43.027850 11322 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:43.028347 12506 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:42.981563 11177 rpcz_store.cc:276] 0709 12:12:41.808523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.374593 (+566070us) service_pool.cc:224] Handling call
0709 12:12:42.375236 (+   643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:42.375442 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:12:42.414609 (+ 39167us) log.cc:844] Serialized 14 byte log entry
0709 12:12:42.904155 (+489546us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:42.904271 (+   116us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:42.904490 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.904662 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.905076 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48539,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":21470,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":347,"thread_start_us":518094,"threads_started":2,"wal-append.queue_time_us":444093}
I20260709 12:12:43.025286 12651 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.008067 12865 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.013192 12805 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:43.035140 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.045291 12859 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.046541 12859 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.048986 12865 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:43.053675 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.058568 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:43.058869 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.060676 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.070986 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.072039 11173 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:43.072582 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1502) took 1291 ms. Trace:
I20260709 12:12:43.072891 11173 rpcz_store.cc:276] 0709 12:12:41.781009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.781189 (+   180us) service_pool.cc:224] Handling call
0709 12:12:43.070559 (+1289370us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:43.072484 (+  1925us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7278976}
I20260709 12:12:43.076753 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 172) took 1028 ms. Trace:
I20260709 12:12:43.080336 12859 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.081394 12651 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.083201 12787 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:43.085934 12885 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.002s
I20260709 12:12:43.087797 12795 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:43.035828 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 599085us
I20260709 12:12:43.089591 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:43.091552 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:43.095193 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.096730 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.102392 12865 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:43.103235 12865 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:43.103564 12805 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:43.104200 12805 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:43.104698 12805 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:43.106276 12793 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.106937 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:43.107858 11933 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:43.110230 11324 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.110695 11928 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:43.054309 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.117327 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:43.118059 11322 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.099295 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1511) took 1303 ms. Trace:
I20260709 12:12:41.718371 11325 rpcz_store.cc:276] 0709 12:12:40.102471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.503109 (+1400638us) service_pool.cc:224] Handling call
0709 12:12:41.524389 (+ 21280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.524578 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:41.656488 (+131910us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.697237 (+ 40749us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.697299 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:41.697340 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.697404 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.697535 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24856,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":8886,"thread_start_us":148445,"threads_started":2}
I20260709 12:12:43.120086 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.120468 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:43.120951 11330 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:43.121451 11929 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:43.122047 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.123044 11325 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:43.124293 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.125792 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.077322 11181 rpcz_store.cc:276] 0709 12:12:42.047832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.420473 (+372641us) service_pool.cc:224] Handling call
0709 12:12:42.432164 (+ 11691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:42.437203 (+  5039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:43.076177 (+638974us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b5400256cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:43.076520 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9514880}
I20260709 12:12:43.116343 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:43.119621 11928 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:43.114024 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:43.104871 12865 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.127187 12873 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:43.128141 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:43.082659 12651 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.138144 11325 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.140184 11327 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:43.145931 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.146449 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.156005 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:43.159544 11935 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:43.120106 11178 rpcz_store.cc:276] 0709 12:12:41.795658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.299410 (+503752us) service_pool.cc:224] Handling call
0709 12:12:42.390390 (+ 90980us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:42.453461 (+ 63071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:43.098601 (+645140us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b54002571d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:43.099044 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68473088}
I20260709 12:12:43.154870 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.165972 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.167809 11175 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:43.174086 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:43.166368 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.162386 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.186268 11327 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:43.187529 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.198406 11330 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:43.199199 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:43.199489 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.200623 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.189363 12676 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.182178 12793 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:43.190358 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.192659 11322 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:43.183101 12795 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:43.194278 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:43.185312 12868 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:43.195104 12816 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
W20260709 12:12:43.196494 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 552417us
I20260709 12:12:43.197549 12618 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:43.206314 11324 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:43.207073 11327 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:43.206984 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.208205 11176 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:43.208096 11249 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:43.208907 11175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45550
W20260709 12:12:43.211001 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 1.7098e+06us
I20260709 12:12:43.211076 12805 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:43.212729 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 99123.7us
I20260709 12:12:43.218353 11325 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:43.194090 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.216043 11255 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:43.236203 12664 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:43.236580 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.237197 12664 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.238377 11173 raft_consensus.cc:2368] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:43.238480 12586 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:43.238166 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:43.217397 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.227953 12795 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.243986 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.249348 11330 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:43.250758 12643 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:43.250983 12859 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:43.251076 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:43.251500 12859 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.252076 12859 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.251581 11252 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.253575 12664 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.254459 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.251237 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.255254 11176 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:43.255362 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:43.256201 12805 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.257520 12618 tablet_bootstrap.cc:492] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:12:43.269523 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.270175 12618 ts_tablet_manager.cc:1403] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 1.446s	user 0.053s	sys 0.055s
I20260709 12:12:43.270660 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.271570 11253 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.271636 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.272747 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.273155 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.251500 11175 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:12:43.274559 12889 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:12:43.274674 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.276880 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.278196 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.279162 11248 raft_consensus.cc:2368] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:43.280022 11928 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:43.280377 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.280701 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:43.280022 11931 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:43.281965 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.282793 11257 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.278573 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.283052 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:43.283548 11328 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:43.284297 12687 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:43.284577 12793 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:43.284919 12687 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:43.285892 11935 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Advancing to term 3
I20260709 12:12:43.286626 12798 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:43.287182 12798 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.287623 12798 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.289430 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1493) took 1568 ms. Trace:
I20260709 12:12:43.288203 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:43.248024 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74954.6us
I20260709 12:12:43.284929 11328 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:43.218135 12787 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:12:43.310137 12793 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:43.289907 11182 rpcz_store.cc:276] 0709 12:12:41.720539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.720746 (+   207us) service_pool.cc:224] Handling call
0709 12:12:41.795613 (+ 74867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:42.271488 (+475875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:43.288937 (+1017449us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54002e3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:43.289233 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092461440}
W20260709 12:12:43.312256 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57049.9us
I20260709 12:12:43.313249 12664 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:43.317909 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.318122 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.318344 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.318907 12651 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.318841 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.292634 12643 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:43.320020 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.344062 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.320847 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.344461 12643 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.321508 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.323014 11327 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:43.325982 12669 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:43.345860 12669 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.346251 12669 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.348420 12829 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.326874 12805 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:43.349728 12805 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.350601 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.351868 12805 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:43.258630 12795 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.311065 12832 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:12:43.332654 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.357649 11179 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.358464 11180 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:43.332986 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.361931 11249 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.338045 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:43.363188 11329 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:43.364502 11181 raft_consensus.cc:2368] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:43.365005 12737 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:43.365410 12737 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:43.366392 12737 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.367928 12742 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:43.368201 12737 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.340528 12664 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.370357 12664 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.344179 11175 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.372140 11173 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:43.368909 12742 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.338362 12687 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.377894 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:43.378507 11182 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:12:43.386361 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:43.387883 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:43.322994 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.391623 11935 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:43.393858 12742 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:43.389279 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46845.4us
W20260709 12:12:43.398016 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 68302.3us
W20260709 12:12:43.342931 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 107392us
I20260709 12:12:43.351953 11323 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:12:43.397162 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70119.5us
I20260709 12:12:43.407927 11257 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:43.389211 11175 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:43.412853 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.415222 11177 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.416328 11330 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:43.415786 11180 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.426250 11179 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:43.428351 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:43.413546 12586 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:43.454854 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.446352 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:43.430636 11178 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:43.461541 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 173884us
I20260709 12:12:43.441567 12586 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:43.471380 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55200.4us
I20260709 12:12:43.445004 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:43.462822 11176 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.475549 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.477275 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.485126 11257 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.463217 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:43.497117 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 75864.4us
I20260709 12:12:43.470881 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 368) took 1102 ms. Trace:
I20260709 12:12:43.473351 12586 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.507032 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:43.512840 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.515471 11930 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:43.514092 11181 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:43.518424 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:43.520944 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 126003us
I20260709 12:12:43.484899 12662 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:43.526660 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.496477 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.533435 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.512905 12687 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:43.512864 12742 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:43.521540 12804 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:43.532591 12662 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:43.539250 12643 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:12:43.544003 11253 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:12:43.544610 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 127051us
I20260709 12:12:43.549803 12788 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:43.558323 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.485340 12798 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.542430 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:43.519320 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:43.569938 11173 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.568349 12865 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 12:12:43.577224 12893 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:12:43.578476 11076 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 256) took 5199 ms. Trace:
I20260709 12:12:43.578868 11254 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:43.579257 12643 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:43.579613 12643 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:43.579924 12859 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.569072 12829 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
W20260709 12:12:43.568909 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 44101.6us
I20260709 12:12:43.582324 12896 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:43.583065 12896 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:43.584380 12896 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.586165 11175 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:43.589120 12662 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.591826 11173 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:43.593513 11935 raft_consensus.cc:2368] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:43.593886 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.595122 11175 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:43.595659 12859 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.596976 11255 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:43.598616 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.573546 11257 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.528527 12805 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:43.575205 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.511680 11927 rpcz_store.cc:276] 0709 12:12:42.359266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.368085 (+  8819us) service_pool.cc:224] Handling call
0709 12:12:43.017979 (+649894us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:43.018576 (+   597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:43.107819 (+ 89243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:43.107859 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:43.108556 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 12:12:43.200996 (+ 92440us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:12:43.257871 (+ 56875us) log.cc:844] Serialized 14 byte log entry
0709 12:12:43.420186 (+162315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:43.424324 (+  4138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:43.427493 (+  3169us) raft_consensus.cc:1685] finished
0709 12:12:43.446433 (+ 18940us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:43.458979 (+ 12546us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37599,"prepare.queue_time_us":40753,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":1508,"raft.queue_time_us":50398,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":302,"spinlock_wait_cycles":44063488,"thread_start_us":213945,"threads_started":2,"wal-append.queue_time_us":76511}
W20260709 12:12:43.603389 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.601198 12805 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.605896 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.606541 12798 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.606743 12805 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.607194 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.606767 11254 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.611413 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:43.612257 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.506748 12618 raft_consensus.cc:359] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:43.613902 11252 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.615655 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.615787 12857 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:43.616328 12857 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:43.616954 11254 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:43.573004 12669 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.617939 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.618623 11254 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:43.619431 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.619606 12669 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.613818 12618 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:12:43.620433 12618 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:43.622875 12618 ts_tablet_manager.cc:1434] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.352s	user 0.007s	sys 0.002s
I20260709 12:12:43.624199 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.624902 11250 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.604303 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:43.630200 12859 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:43.630707 12859 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.631108 12859 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:43.631134 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:43.631770 11173 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:43.632102 12889 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:43.632685 12889 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:43.633869 12889 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:43.637130 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 151482us
I20260709 12:12:43.637956 12865 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:12:43.638999 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 82228.7us
I20260709 12:12:43.639202 12742 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.640008 12873 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:43.640230 12859 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.642509 11296 leader_election.cc:400] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:43.642048 12804 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.642922 11296 leader_election.cc:403] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:12:43.644675 12804 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.645442 11249 raft_consensus.cc:2368] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:43.646085 12873 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.647748 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.651136 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.652514 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.653934 12832 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:43.617491 12857 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.658625 12895 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:43.613400 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.649221 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.642640 12859 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.559500 11177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45550
I20260709 12:12:43.697260 11177 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:12:43.666429 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 131089us
I20260709 12:12:43.667152 11253 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.669617 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.671063 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:43.674129 12742 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:43.580869 12643 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.676291 12664 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.540841 12687 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.639031 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:43.684839 11178 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.578872 11076 rpcz_store.cc:276] 0709 12:12:38.329723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.342500 (+ 12777us) service_pool.cc:224] Handling call
0709 12:12:38.425154 (+ 82654us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:38.888742 (+463588us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5400260110. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c8640 00007f3ecc49f846 00007f3ecc5a990e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:43.524357 (+4635615us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:41.440521 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:41.570148 (+129627us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:41.570189 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:12:41.570207 (+    18us) tablet.cc:662] Decoding operations
0709 12:12:41.612930 (+ 42723us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:41.612995 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:41.613025 (+    30us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:12:41.613709 (+   684us) tablet.cc:701] Row locks acquired
0709 12:12:41.613725 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:12:41.613873 (+   148us) write_op.cc:270] Start()
0709 12:12:41.613999 (+   126us) write_op.cc:276] Timestamp: P: 1783599161613828 usec, L: 0
0709 12:12:41.614037 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:12:41.618474 (+  4437us) log.cc:844] Serialized 2918 byte log entry
0709 12:12:42.450691 (+832217us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b34000b26e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3eca056161 00007f3eca12328e 00007f3eca12571a 00007f3eca1256a8 00007f3eca125628 00007f3eca1255e3 00007f3eca1247fc 00007f3ecc5572f6 00007f3ecc551aa8 00007f3ec66f8310 00007f3ec66cf95d
0709 12:12:42.450845 (+   154us) op_driver.cc:464] REPLICATION: finished
0709 12:12:42.854465 (+403620us) write_op.cc:301] APPLY: starting
0709 12:12:42.854595 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:42.854877 (+   282us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:42.854922 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:42.856319 (+  1397us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:42.856619 (+   300us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:12:42.856705 (+    86us) write_op.cc:312] APPLY: finished
0709 12:12:42.857090 (+   385us) log.cc:844] Serialized 59 byte log entry
0709 12:12:42.864864 (+  7774us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:42.865048 (+   184us) write_op.cc:454] Released schema lock
0709 12:12:42.865133 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:42.866366 (+  1233us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:42.866428 (+    62us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":353885,"reported_tablets":8,"spinlock_wait_cycles":51367936,"child_traces":[["op",{"apply.queue_time_us":403296,"apply.run_cpu_time_us":3720,"apply.run_wall_time_us":11141,"num_ops":7,"prepare.queue_time_us":318822,"prepare.run_cpu_time_us":15845,"prepare.run_wall_time_us":315649,"raft.queue_time_us":305064,"raft.run_cpu_time_us":33114,"raft.run_wall_time_us":885985,"replication_time_us":379693,"spinlock_wait_cycles":4060032,"thread_start_us":950293,"threads_started":4,"wal-append.queue_time_us":47896}]]}
I20260709 12:12:43.690513 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:43.691425 12669 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:43.693624 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 297296us
I20260709 12:12:43.694664 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.695607 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.693931 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.700136 11181 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:43.726804 12859 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:12:43.719830 12687 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.727259 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:43.730017 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.774655 11255 raft_consensus.cc:2368] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:43.731135 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:43.731334 12795 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.731743 12804 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:43.732288 12586 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:43.737219 12742 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:12:43.738087 12643 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.738394 12857 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.720912 11927 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:43.741742 11254 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.743446 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.740226 12664 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.744822 12788 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:43.748163 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.790364 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.749965 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.720732 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.753696 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:43.759361 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 120554us
I20260709 12:12:43.775529 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.796803 11248 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:43.797467 11179 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:43.798434 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.798740 11179 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:43.799657 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.800312 12795 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.790148 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:43.801563 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.803736 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.806387 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.807126 11248 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.810216 11174 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:43.811220 11180 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:43.812088 11177 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:43.814487 11173 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:43.780818 12787 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:12:43.783558 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 93144.6us
W20260709 12:12:43.784143 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64886.4us
I20260709 12:12:43.776854 11253 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:43.785724 12865 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:43.819430 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.819453 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.801142 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.779086 12669 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:43.825433 11181 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:43.822754 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:43.821061 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 90056.8us
I20260709 12:12:43.831552 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:43.790563 11250 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:43.839931 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:43.841025 11249 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.843194 11252 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:43.822649 12865 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:43.849902 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:43.853196 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 216811us
I20260709 12:12:43.856483 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.836594 12798 raft_consensus.cc:2764] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:12:43.858403 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 133) took 1213 ms. Trace:
I20260709 12:12:43.842875 11175 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:43.879276 12669 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:43.881063 12586 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:43.882457 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:43.889715 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.855288 12865 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.906280 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.858428 12666 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:43.855463 12787 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:12:43.918671 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:43.919364 11325 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:43.867070 12895 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:43.864050 11256 rpcz_store.cc:276] 0709 12:12:42.510325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.510462 (+   137us) service_pool.cc:224] Handling call
0709 12:12:42.510751 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:43.213539 (+702788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:43.213604 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:43.214734 (+  1130us) log.cc:844] Serialized 26 byte log entry
0709 12:12:43.309574 (+ 94840us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:43.714225 (+404651us) log.cc:844] Serialized 14 byte log entry
0709 12:12:43.714642 (+   417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:43.716795 (+  2153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:43.716849 (+    54us) raft_consensus.cc:1685] finished
0709 12:12:43.716868 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:43.724043 (+  7175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":745760,"prepare.queue_time_us":460163,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":704582,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":17396,"thread_start_us":373690,"threads_started":2,"wal-append.queue_time_us":242168}
I20260709 12:12:43.903086 12651 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:43.857173 11078 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 1 to 2, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 2 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:43.932446 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95073.4us
I20260709 12:12:43.932477 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:43.917810 12666 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.920264 12895 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:12:43.954879 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 130058us
I20260709 12:12:43.957167 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:43.941226 11078 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 3, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 3 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:43.959270 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:43.935317 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.970702 11931 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Advancing to term 2
I20260709 12:12:43.942008 12816 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:43.972826 11173 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:43.973452 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.942251 12666 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:43.943809 12873 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:43.986888 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.938318 11927 raft_consensus.cc:1217] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:12:43.986544 12742 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:43.989719 11179 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:43.989974 11177 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:43.979986 12816 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:43.994445 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:43.997423 12586 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:43.998080 12664 raft_consensus.cc:2764] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:12:43.985610 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:44.001685 12906 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:12:43.994508 11296 leader_election.cc:400] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:44.014045 11249 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:44.014652 11181 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:44.018182 11252 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:44.047091 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:44.056186 11256 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:44.047340 11078 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 3, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 3 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:44.047645 11175 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:43.950511 12788 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:12:44.051405 11173 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:44.061136 11177 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:43.998667 12687 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.016460 12816 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:44.050040 11147 leader_election.cc:400] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
W20260709 12:12:44.061674 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64711.9us
I20260709 12:12:44.078595 11257 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:44.070920 11324 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:44.015797 12586 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.077224 11078 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 3, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 3 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:44.054903 12873 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.077422 11253 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:12:44.084578 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.084928 11253 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.068084 12795 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:44.087749 11251 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:44.088866 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:44.059351 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.045152 12742 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.096439 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:44.100749 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 141544us
W20260709 12:12:44.102988 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 108635us
W20260709 12:12:44.096629 12904 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:12:44.110164 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.110913 11175 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.111188 12663 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:44.113121 11930 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:44.109005 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:44.117296 11325 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
W20260709 12:12:44.114387 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 134585us
I20260709 12:12:44.122432 11934 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:44.127575 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.068948 12895 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.128819 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.129443 12895 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.130666 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.132027 12807 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.132368 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.132798 12805 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.094174 12873 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.135962 12873 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:44.136579 12795 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:44.136955 11175 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:44.137130 12795 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:44.138233 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:44.138897 12742 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:44.138999 12651 raft_consensus.cc:993] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:44.139089 12807 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:44.138645 12795 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:44.139614 12807 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:44.140110 12807 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.141571 12896 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:44.142040 12896 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:44.142452 11176 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:44.142982 12896 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.143100 12865 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:44.145925 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.146044 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.146759 12586 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:44.149677 11250 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:44.108179 11253 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.159421 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:44.159782 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:44.160629 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.195282 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.195904 11177 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:44.196089 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:44.196703 12917 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:44.197154 12918 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:44.160972 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.197394 12917 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:44.197814 12918 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:44.198109 11182 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:44.197958 12807 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.161782 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.199352 11179 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:44.198727 12917 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.198911 12918 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.160662 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.163314 11929 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:44.200392 12807 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:44.166170 12687 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:44.107563 11178 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:44.202243 12911 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.205122 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.170346 11176 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:44.170150 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:12:44.170697 11900 leader_election.cc:336] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.520s (SENT)
I20260709 12:12:44.170797 12742 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.157735 12788 raft_consensus.cc:1081] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:44.169983 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.180043 11931 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:44.182482 11253 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:44.220696 12807 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.220348 12666 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.222916 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.223855 12742 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.189939 12865 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:44.193413 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:44.227854 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 91756.2us
I20260709 12:12:44.226950 11296 leader_election.cc:400] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:44.205948 12873 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:44.206045 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:44.218832 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 48173.9us
I20260709 12:12:44.219159 11177 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:44.194183 11248 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:44.228996 11254 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:44.234665 11253 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:44.237309 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:44.238095 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:44.219199 12896 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.244989 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.245740 11177 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.252951 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.238590 12787 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.241494 12859 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:12:44.270326 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 191340us
I20260709 12:12:44.275738 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:44.243835 12911 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:44.287341 11177 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:44.290287 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.245779 12816 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.295352 12586 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:44.296185 12816 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:44.298045 12919 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:44.298657 12919 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.299441 12859 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:44.299955 12919 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.300661 12859 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.305410 11325 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:44.298329 12911 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:44.301946 11323 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:44.293439 12669 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.307852 12816 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:44.321493 12816 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:44.322583 12816 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.323036 12910 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:44.312058 12919 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.317647 12788 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.337239 11176 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:44.310261 12909 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:12:44.335055 12651 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:44.356792 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.359731 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.364121 11176 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:44.366393 11254 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:44.366508 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.366972 11254 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.367566 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.367945 11254 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.368577 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.369980 12925 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:44.371057 12925 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.369380 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.371775 12925 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:44.373685 12925 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.374732 12925 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:44.376209 12925 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.371773 12666 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:44.377971 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:44.378177 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.379318 11254 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.343696 12865 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:12:44.385239 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.387310 11249 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:44.379733 11178 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:12:44.393723 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.381829 12586 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.360136 12910 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:12:44.403393 12924 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:12:44.404119 12807 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.380848 12651 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:44.399166 11249 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:44.372618 12911 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.400139 11176 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:44.387072 12923 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:12:44.397583 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 152523us
W20260709 12:12:44.416901 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 173014us
I20260709 12:12:44.425105 12651 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:44.448887 11325 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:44.425889 12896 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:44.427093 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:44.432461 12910 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:12:44.437700 12913 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:44.419435 12924 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:44.439349 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.442222 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.450666 11324 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:44.469812 12663 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:44.470890 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.471057 12687 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:44.471534 12687 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.471652 11249 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:44.472687 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:44.474045 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.474888 12787 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:44.476141 12932 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:44.476547 12932 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:44.477306 12666 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:44.477653 12932 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.477950 12666 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.478366 12895 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:44.478787 12895 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:44.479746 12895 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.480386 12666 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:44.481341 12895 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.455240 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.486019 11254 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:44.458482 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:44.461697 11256 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:12:44.465575 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:44.467422 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.467432 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.472595 12687 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:44.454609 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 82593.2us
I20260709 12:12:44.428560 12586 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:12:44.505558 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.509443 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.510325 11249 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:44.511442 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.508386 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:44.522346 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:44.524796 11324 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:44.526233 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74373.2us
I20260709 12:12:44.468405 12924 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.503388 11250 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.522863 11256 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:44.533407 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:44.470904 12663 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:12:44.507931 12917 raft_consensus.cc:2764] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:12:44.538976 11296 leader_election.cc:400] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
I20260709 12:12:44.541356 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:44.542342 11296 leader_election.cc:403] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:12:44.552703 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 113572us
I20260709 12:12:44.552703 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:44.554049 11256 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:44.554220 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.555365 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:44.556622 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:44.557579 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.558408 11251 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:44.561813 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-5df07764623640ce8ec29ad695812b7c received from {username='slave'} at 127.0.0.1:52330
I20260709 12:12:44.562530 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-5df07764623640ce8ec29ad695812b7c on tablet 5df07764623640ce8ec29ad695812b7c with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:44.555568 12669 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:44.566128 12915 raft_consensus.cc:2764] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:12:44.568491 12788 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.540329 12919 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:12:44.482012 12923 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:12:44.538548 12807 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:44.572634 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.555336 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:44.580219 11250 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:44.580066 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:44.581311 11324 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:44.583786 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.552943 12873 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3.
W20260709 12:12:44.584069 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49761.8us
I20260709 12:12:44.543174 12865 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:12:44.590723 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:44.537256 12931 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:12:44.579090 12807 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:44.598464 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:44.592921 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:44.589671 11254 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:44.599431 11254 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.590912 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.595916 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.602504 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:44.603120 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:44.603519 11250 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:44.582213 12926 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.590797 12805 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.582237 12669 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.609972 11254 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.614301 11330 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:44.587253 12896 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:44.635746 11296 leader_election.cc:400] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
I20260709 12:12:44.663923 11296 leader_election.cc:403] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:12:44.651685 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.652474 11173 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:44.651170 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.604640 12807 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:44.605391 12687 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:44.708039 11254 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:44.708357 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 72771.2us
I20260709 12:12:44.709086 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.634285 12663 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:44.614418 12859 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:44.636330 12816 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:44.654925 11251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:12:44.723961 11251 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:44.724251 11177 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:44.724368 11251 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:44.661531 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:44.739444 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.735391 11254 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:44.738318 11324 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:44.666637 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:44.724591 12925 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:44.751571 11933 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:44.661688 12873 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:12:44.738703 12805 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:44.761343 11324 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:44.746529 12911 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:44.758203 11250 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
W20260709 12:12:44.764521 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 158068us
I20260709 12:12:44.765296 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:44.746802 12663 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.747316 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:44.760577 12932 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:44.777282 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:44.769524 12941 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:12:44.769177 12669 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:44.770347 12944 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 12:12:44.772514 12930 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.348s	user 0.005s	sys 0.000s
I20260709 12:12:44.773047 12935 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:44.774148 11322 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:44.599961 12924 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.749249 12816 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:44.753814 12925 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.798677 11330 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:44.735391 12859 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:44.800472 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.777086 11178 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:44.815423 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:44.819294 11324 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:44.779075 12795 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:44.823217 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46048.6us
I20260709 12:12:44.824365 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:44.814884 12798 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.855206 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:44.855463 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 203857us
I20260709 12:12:44.856230 11324 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:44.856999 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:44.816404 12865 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4.
W20260709 12:12:44.852895 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 126309us
I20260709 12:12:44.898168 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:44.901403 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.903735 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:44.904467 11935 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:44.906116 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49223.1us
I20260709 12:12:44.914173 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:12:44.919549 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 95257.6us
W20260709 12:12:44.925606 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54745us
I20260709 12:12:44.926783 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:44.927805 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 101976us
I20260709 12:12:44.871600 12895 raft_consensus.cc:2764] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:12:44.871107 12664 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:44.900785 12873 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:44.941075 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.943286 11148 leader_election.cc:400] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4.
I20260709 12:12:44.949429 11324 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:44.913579 12805 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:44.869853 12911 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:12:44.816056 12935 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.956110 11256 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:44.964067 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:44.972981 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:44.977937 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70844.3us
I20260709 12:12:44.977505 12896 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:44.987282 12896 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:44.987866 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:44.989661 11326 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:12:44.990219 12948 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:12:44.989662 11330 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:44.991225 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:12:44.991873 12947 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:12:44.992689 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:44.993131 12664 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:44.974716 12643 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:44.993957 12816 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:44.994619 12919 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:44.999873 12643 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:45.000618 12805 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:45.001941 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:44.914554 12663 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.004168 12666 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:45.004623 12666 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:44.990108 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:45.006220 11327 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:45.005643 12666 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.007004 12666 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:45.010322 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.012147 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:45.014600 12922 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:44.957429 12959 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:44.989404 12896 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:45.021317 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 48455.4us
W20260709 12:12:45.027086 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 90387.1us
I20260709 12:12:45.029676 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:45.021695 11935 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:45.015152 11251 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:45.031523 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 85829us
I20260709 12:12:45.031774 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:12:45.031857 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 105202us
I20260709 12:12:45.039958 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.042101 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:45.045910 11324 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:12:44.954245 12925 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.053445 11251 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.019919 12959 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.044607 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:12:45.057466 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78582.1us
I20260709 12:12:45.049759 12664 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:45.061483 12664 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:45.052165 12922 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:45.062107 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.062361 12922 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:45.063366 12922 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.063879 12909 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:45.065011 12961 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:45.065462 12961 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:45.045306 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:45.078126 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:45.060918 12959 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:45.084923 11935 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:45.069648 12859 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.054414 12643 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:12:45.069353 12950 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:45.071281 12664 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:45.079608 12919 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:45.070647 12873 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:45.073191 12787 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:12:45.100786 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47085.9us
I20260709 12:12:45.088640 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:45.083302 12795 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:45.077154 12663 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:12:45.089437 12909 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:12:45.107465 11251 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:45.066480 12961 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:45.083725 12922 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:12:45.111101 12961 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:45.104142 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:45.104470 12964 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:45.104856 12936 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.370s	user 0.002s	sys 0.000s
I20260709 12:12:45.105322 12958 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:45.105686 12950 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:45.079376 12915 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:12:45.096385 11323 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:45.094700 12664 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:45.111150 12967 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:45.118633 12935 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.130862 12816 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.124086 12663 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:45.146476 12816 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:45.148007 11248 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:12:45.149016 12816 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.130502 12807 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.150535 11328 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:45.131424 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.135293 12586 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:45.155143 12586 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:45.156147 12586 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.159386 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.160744 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:45.161947 12918 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:45.162814 12918 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:45.136564 11176 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:45.118139 12795 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:12:45.123641 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 83796.6us
I20260709 12:12:45.153386 12968 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:12:45.154374 12807 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.172816 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.154506 11178 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:45.157392 12586 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:45.167065 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 98114.6us
I20260709 12:12:45.170897 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:45.171865 11176 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
W20260709 12:12:45.174136 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.265913 11182 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:45.266502 11182 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:45.267241 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:45.163249 12918 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.271303 12918 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:45.274144 12955 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:45.275074 12970 raft_consensus.cc:2764] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:12:45.276511 12663 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:45.278611 12974 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:45.279376 12974 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.281908 12805 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:45.284080 12971 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:12:45.286585 12976 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:12:45.289448 12807 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.174525 12950 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:45.299803 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:45.254408 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 81676.8us
I20260709 12:12:45.151968 12664 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:45.300989 11325 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:45.301199 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.174162 12958 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:12:45.217617 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52006.5us
I20260709 12:12:45.313380 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.176430 12967 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:45.270665 12662 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:45.290221 12807 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.233860 12932 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:45.233481 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:45.328293 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:45.329456 11329 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:45.225705 12832 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:12:45.406211 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:45.333563 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:45.414644 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87344.7us
I20260709 12:12:45.415030 12788 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:45.415645 12788 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:45.415793 12974 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.417021 12662 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.416663 12788 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.417693 12662 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.420164 12662 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:45.420632 12958 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.285145 11182 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:45.422601 11182 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:45.414801 12969 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 12:12:45.412930 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 111821us
I20260709 12:12:45.424826 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:45.427997 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.428085 11929 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:45.295948 12873 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.429875 12805 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:45.430577 12805 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.431067 12805 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:45.429500 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:45.433314 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:45.316020 12787 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:45.440223 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:45.327070 12586 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.112s
W20260709 12:12:45.432989 12687 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.002s
W20260709 12:12:45.450059 11331 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.007s	sys 0.021s
I20260709 12:12:45.413486 12931 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:12:45.458614 11331 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.008s	sys 0.021s
I20260709 12:12:45.460682 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.445502 12666 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:45.446035 12662 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:45.406399 12932 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.449347 12970 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:45.471659 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.460215 12663 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:45.468247 12666 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.478852 12666 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:45.479256 12925 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:45.467449 11248 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:45.481156 12984 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:45.466293 12918 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.475426 12981 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:45.486693 12981 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.459856 12931 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.488183 12931 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:45.488848 12981 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.489519 11173 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
W20260709 12:12:45.495793 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60507.1us
I20260709 12:12:45.496224 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.335999 12950 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.438143 12795 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.506739 12979 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
W20260709 12:12:45.510671 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.502360 12982 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:45.514346 12977 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:12:45.502837 12975 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:45.503813 12807 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:45.470698 12959 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.504333 12984 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:45.504982 12974 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:45.506072 11331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:37422
I20260709 12:12:45.505784 12972 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:12:45.523428 11182 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:45.498677 12958 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:45.553352 12742 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:45.558177 12895 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:45.562660 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.563566 12980 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:45.570042 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:45.577427 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:45.605947 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:45.570298 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:45.533914 12982 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:45.619552 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:45.629024 11180 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
W20260709 12:12:45.574074 12669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.019s	sys 0.011s
I20260709 12:12:45.612599 11175 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:45.576776 12742 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:45.562651 12911 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:45.635671 12669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.019s	sys 0.033s
I20260709 12:12:45.619761 12984 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:45.610235 12895 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:45.608484 12975 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:45.610823 12980 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:45.612341 12662 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:45.641899 12982 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.649760 12795 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:45.581943 12931 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.638252 12979 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:12:45.534097 12787 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:45.700397 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 181103us
W20260709 12:12:45.696213 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 256140us
I20260709 12:12:45.713299 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:45.716403 11250 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:45.721442 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:45.725201 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:45.729058 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:45.654884 12984 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:45.501075 12967 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.765547 11255 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:45.699337 12980 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:45.769322 11331 raft_consensus.cc:606] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:12:45.709935 12662 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:45.719754 12910 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.655951 12911 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.740105 12795 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:45.792037 12805 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.796972 12950 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:45.797132 12980 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.798053 12959 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:45.755735 12742 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.799292 11929 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:45.802740 12664 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.803256 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:45.806264 11176 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:45.807552 12924 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:45.758373 12994 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.768836 12586 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:12:45.831696 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 269097us
I20260709 12:12:45.838408 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:45.830462 12981 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.845134 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:45.849586 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:45.800143 12662 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:45.766398 12895 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.839844 12950 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.858419 12896 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.858949 12999 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.824965 12958 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:45.859388 12931 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.862423 11181 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:45.872390 12950 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.879186 12999 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:45.820205 12910 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.864216 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:45.841866 12984 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.859936 12959 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:45.865840 11176 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:12:45.866074 12980 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.866868 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:45.867251 12961 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.000s
I20260709 12:12:45.855264 12742 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:45.867475 12955 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:45.868670 12666 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.877029 12895 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:45.889405 12992 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:12:45.870993 12935 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:45.852142 12981 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.889815 12911 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.897881 12935 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.898303 12950 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.900305 12981 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.902165 13009 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:45.902810 13009 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.904279 12999 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.905141 12982 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.854808 12975 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.891481 12795 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.900110 12935 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.885987 12964 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:45.914749 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.877262 12896 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.893112 12931 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:45.892987 12664 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.905045 13009 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:45.895967 12984 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:45.940940 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:45.936508 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:45.941663 11326 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:12:45.900629 13003 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.848374 12994 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.950486 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:45.951987 12994 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.952072 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:45.954797 13003 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.955574 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:45.957695 13003 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:45.958549 12917 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:45.959188 12917 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:45.959671 12987 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 12:12:45.961808 12917 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:45.962581 12919 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:45.966266 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:45.967311 11329 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:45.965842 12919 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:45.974887 12910 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:45.982229 11252 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:45.983888 11329 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:12:45.984372 11329 pending_rounds.cc:85] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:12:45.988260 11328 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:12:45.988718 11328 pending_rounds.cc:85] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:12:46.003677 11322 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:45.989851 11328 pending_rounds.cc:107] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:12:46.008214 11328 raft_consensus.cc:2889] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:45.989955 11329 pending_rounds.cc:107] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:12:46.010918 11329 raft_consensus.cc:2889] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:46.004022 12958 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:45.937821 12964 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:45.991657 12805 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:46.035827 12666 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:46.046782 11326 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
W20260709 12:12:46.053849 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.069504 12921 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.070212 12921 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.072475 12921 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.072985 12918 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.102427 12664 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:46.106498 11900 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.088s (SENT)
I20260709 12:12:46.104105 12664 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:46.132911 12995 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.426s	user 0.002s	sys 0.000s
W20260709 12:12:46.151991 12953 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.149s	user 0.003s	sys 0.013s
I20260709 12:12:46.132632 12896 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.066473 12662 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.166147 12662 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.169474 12662 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:46.170826 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64356.2us
I20260709 12:12:46.159330 12586 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.174188 12586 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:46.189179 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1214) took 1595 ms. Trace:
I20260709 12:12:46.196210 12664 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:46.197343 12586 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.204608 12932 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.219451 12931 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:46.220996 12964 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.232075 11323 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:12:46.232689 11323 pending_rounds.cc:85] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 1
I20260709 12:12:46.233034 11323 pending_rounds.cc:107] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:12:46.233367 11323 raft_consensus.cc:2889] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:46.237758 11249 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:46.190202 11257 rpcz_store.cc:276] 0709 12:12:44.594068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:44.594913 (+   845us) service_pool.cc:224] Handling call
0709 12:12:44.635266 (+ 40353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:44.738281 (+103015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:44.738338 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:44.752989 (+ 14651us) log.cc:844] Serialized 26 byte log entry
0709 12:12:44.824267 (+ 71278us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:45.017070 (+192803us) log.cc:844] Serialized 14 byte log entry
0709 12:12:45.017452 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:45.017556 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:46.151080 (+1133524us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:12:46.181714 (+ 30634us) raft_consensus.cc:1685] finished
0709 12:12:46.188855 (+  7141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:46.189082 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69453,"prepare.queue_time_us":240838,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":86461,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":16684,"spinlock_wait_cycles":32652928,"thread_start_us":203716,"threads_started":2,"wal-append.queue_time_us":243551}
W20260709 12:12:46.264451 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.270972 11248 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 12:12:46.271622 11248 pending_rounds.cc:85] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 2
I20260709 12:12:46.271996 11248 pending_rounds.cc:107] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 12:12:46.272401 11248 raft_consensus.cc:2889] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:46.274000 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:46.278231 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.282181 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:46.286602 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.283756 12968 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.288322 12968 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.288709 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:46.289498 11935 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:46.290419 12968 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.290970 12816 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.291461 12816 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.293418 12816 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.295639 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:46.297152 13006 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.297794 13006 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.298272 13010 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.298895 13010 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.300454 13006 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.302690 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:46.304244 13010 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:46.304986 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:46.318763 11927 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:46.320278 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.353951 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:46.283845 12919 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.334141 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:46.326690 11935 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:12:46.364888 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.045s	sys 0.043s
W20260709 12:12:46.368798 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.045s	sys 0.043s
I20260709 12:12:46.320544 12816 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:12:46.390962 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.371735 13017 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:46.426278 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.426208 13017 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:46.428263 11255 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:12:46.428328 11325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.014s	sys 0.016s
W20260709 12:12:46.429698 11325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.014s	sys 0.016s
I20260709 12:12:46.430696 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.429111 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.437345 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.438987 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:46.441512 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 107485us
I20260709 12:12:46.367048 12924 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:46.445847 12924 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:46.435809 12915 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.446385 12924 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:46.446931 11254 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:12:46.446918 12915 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:46.448048 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.448455 11250 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:46.448732 11322 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:46.443392 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.449927 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.450237 11326 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:46.450563 11328 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:46.451347 12968 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:46.451783 12968 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.452189 12968 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:46.452960 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.455314 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.341289 12918 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 12:12:46.464193 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:46.474072 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:46.386363 13006 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.476383 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:46.481940 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.482782 11257 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:46.479544 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.491150 11255 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:46.449447 12915 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:46.496408 11250 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:46.497112 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:46.529472 12795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.010s	sys 0.041s
I20260709 12:12:46.536334 12957 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:46.536839 12957 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.537199 12957 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:46.537895 12795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.011s	sys 0.041s
I20260709 12:12:46.480481 12959 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
W20260709 12:12:46.443472 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.014s	sys 0.061s
W20260709 12:12:46.552184 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.015s	sys 0.061s
I20260709 12:12:46.492695 13007 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:46.513263 12964 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:46.480907 12910 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:46.558125 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:46.581566 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:46.581830 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:46.588089 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.485905 13006 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:46.583392 11251 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:12:46.583760 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.021s	sys 0.055s
W20260709 12:12:46.593806 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.021s	sys 0.055s
I20260709 12:12:46.582300 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.563102 13007 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:46.587450 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.595623 11329 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:46.557742 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:46.597389 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:46.580349 12964 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.591545 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:46.606499 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:46.588836 13010 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:46.611348 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:46.592501 13002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.037s	sys 0.072s
I20260709 12:12:46.631022 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:46.596221 12915 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:46.607065 12964 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:46.626065 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:46.619028 12975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.001s	sys 0.027s
W20260709 12:12:46.655660 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49266.5us
I20260709 12:12:46.594777 12910 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:46.618745 12908 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.012s	sys 0.015s
I20260709 12:12:46.597664 12643 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:12:46.609163 12923 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.608779 12960 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:46.739318 13007 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:46.882798 12908 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.012s	sys 0.015s
W20260709 12:12:46.663131 13002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.038s	sys 0.072s
I20260709 12:12:46.662956 12924 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:46.886721 12923 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.663345 12932 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 12:12:46.739271 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 120048us
I20260709 12:12:46.739435 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:46.895057 11327 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
W20260709 12:12:46.629925 12926 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.630s	user 0.023s	sys 0.018s
I20260709 12:12:46.633446 12959 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.093s
W20260709 12:12:46.663066 12975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.002s	sys 0.027s
I20260709 12:12:46.664399 12742 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:46.750982 12895 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:46.826215 13020 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:46.826613 12662 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.838219 12919 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:46.840245 12798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.016s	sys 0.041s
I20260709 12:12:46.841097 13017 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:46.866070 12997 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:46.896615 12926 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.897s	user 0.031s	sys 0.018s
I20260709 12:12:46.895368 11223 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:46.894055 13030 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:46.969906 13030 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:46.970211 12957 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:46.971104 13030 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:46.905799 12923 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.972728 12957 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:46.906491 13022 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:12:46.907444 13023 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:12:46.913064 11327 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:12:46.720225 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:46.975803 11327 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:46.976212 13019 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:46.976766 13019 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:46.977276 13019 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:46.972764 13030 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:46.982065 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 261927us
I20260709 12:12:46.983698 12964 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:46.986047 12964 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:46.961149 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1686) took 1141 ms. Trace:
I20260709 12:12:47.004401 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:47.003463 11330 rpcz_store.cc:276] 0709 12:12:45.819525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.819651 (+   126us) service_pool.cc:224] Handling call
0709 12:12:45.832726 (+ 13075us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:45.856516 (+ 23790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:45.856563 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:45.857190 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:12:46.594195 (+737005us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:46.594239 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:46.594309 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:46.960899 (+366590us) raft_consensus.cc:1685] finished
0709 12:12:46.960921 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:46.961059 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68480,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":147,"spinlock_wait_cycles":543104,"thread_start_us":719478,"threads_started":2,"wal-append.queue_time_us":992049}
I20260709 12:12:46.961750 12975 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:47.006125 12975 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:47.008915 12975 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:46.962389 13024 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:12:47.011276 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 207) took 1199 ms. Trace:
I20260709 12:12:47.011722 11325 rpcz_store.cc:276] 0709 12:12:45.811963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.812395 (+   432us) service_pool.cc:224] Handling call
0709 12:12:45.812777 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:45.858579 (+ 45802us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:45.858621 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:45.870376 (+ 11755us) log.cc:844] Serialized 26 byte log entry
0709 12:12:46.430127 (+559751us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:46.430188 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:46.430291 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:47.011023 (+580732us) raft_consensus.cc:1685] finished
0709 12:12:47.011053 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:47.011211 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25486,"raft.queue_time_us":88786,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":90,"thread_start_us":559952,"threads_started":2,"wal-append.queue_time_us":953592}
W20260709 12:12:46.922721 13021 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 12:12:46.963266 13027 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:12:46.963557 13020 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.015636 13020 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:46.964493 13017 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:47.018189 13017 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:47.020895 13017 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:46.913853 12798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.017s	sys 0.041s
W20260709 12:12:46.968914 12968 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.005s
I20260709 12:12:47.023166 12968 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:46.969208 12919 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.026060 12919 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:46.937844 12924 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:46.952174 13006 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:46.914719 12662 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:47.031785 12662 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:46.888698 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.986322 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1675) took 1816 ms. Trace:
I20260709 12:12:47.034055 11173 rpcz_store.cc:276] 0709 12:12:45.170139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.170270 (+   131us) service_pool.cc:224] Handling call
0709 12:12:45.170579 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:45.490010 (+319431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:46.986079 (+1496069us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b5400363654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:46.986257 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733053824}
I20260709 12:12:46.987239 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:47.036162 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49004.5us
I20260709 12:12:47.037508 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:47.038632 13010 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:47.039336 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:47.039474 13010 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:46.934170 12997 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:46.988623 11249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 146) took 2406 ms (client timeout 2088 ms). Trace:
W20260709 12:12:46.989159 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 93879.1us
I20260709 12:12:46.993106 13019 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:47.049110 12873 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:47.051841 13019 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:47.060240 11174 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:47.002496 12742 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:47.005254 11257 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
W20260709 12:12:47.044198 11249 rpcz_store.cc:269] 0709 12:12:44.582505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:44.582788 (+   283us) service_pool.cc:224] Handling call
0709 12:12:46.988426 (+2405638us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:46.988539 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3762263936}
W20260709 12:12:47.048520 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:46.912786 12896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.007s	sys 0.020s
W20260709 12:12:47.065258 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:47.066345 11933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.048s	sys 0.111s
W20260709 12:12:47.075153 11933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.048s	sys 0.112s
W20260709 12:12:47.076095 11331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.009s	sys 0.020s
I20260709 12:12:46.915220 12910 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:47.079583 11331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.009s	sys 0.020s
W20260709 12:12:47.086968 11253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.041s	sys 0.063s
W20260709 12:12:47.087499 11253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.041s	sys 0.063s
I20260709 12:12:47.028704 12919 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:47.062775 12975 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:12:47.074204 12896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.007s	sys 0.021s
W20260709 12:12:47.113978 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74738.7us
I20260709 12:12:47.115796 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:47.116124 11250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.029s	sys 0.011s
W20260709 12:12:47.088563 12931 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.026s
W20260709 12:12:47.125110 11250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.030s	sys 0.011s
I20260709 12:12:47.112861 12959 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.107465 12968 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:47.117674 12742 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:47.140749 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.018s	sys 0.022s
I20260709 12:12:47.140560 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:47.144112 11173 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
W20260709 12:12:47.126487 12931 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.006s	sys 0.026s
W20260709 12:12:47.144389 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.018s	sys 0.023s
W20260709 12:12:47.219527 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 618018us
I20260709 12:12:47.261474 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:47.126039 12975 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:47.300283 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:47.302253 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:47.303437 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:47.302564 12957 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:47.302821 12964 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:47.127703 13010 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.232s
I20260709 12:12:47.261925 12972 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:12:47.375111 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.608s (SENT) [suppressed 13 similar messages]
I20260709 12:12:47.288802 12968 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:12:47.396692 11179 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.049s	sys 0.077s
I20260709 12:12:47.371346 13020 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:47.139664 12959 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:47.406096 11179 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.050s	sys 0.077s
I20260709 12:12:47.303874 12909 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:12:47.496675 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:47.386013 13017 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:47.497478 11173 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:47.497934 11251 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.032s	sys 0.092s
W20260709 12:12:47.503552 11251 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.032s	sys 0.092s
I20260709 12:12:47.373243 12957 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:47.501482 13029 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:12:47.497576 12873 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:47.513386 12959 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:47.514030 13017 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:47.376107 12964 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:47.524327 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:47.276160 12742 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:47.473277 12918 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:12:47.125909 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.021s	sys 0.014s
I20260709 12:12:47.441951 12968 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:47.696003 12957 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:47.741042 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.260s	user 0.022s	sys 0.014s
W20260709 12:12:47.734809 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.015s	sys 0.029s
W20260709 12:12:47.730650 13002 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.042s	sys 0.071s
I20260709 12:12:47.768662 12918 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:47.777720 11254 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.038s	user 0.048s	sys 0.075s
W20260709 12:12:47.780328 11254 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.048s	sys 0.075s
W20260709 12:12:47.780953 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.29955e+06us
I20260709 12:12:47.719084 12997 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:47.784164 11173 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
W20260709 12:12:47.771075 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.015s	sys 0.030s
W20260709 12:12:47.751053 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.040s	sys 0.083s
W20260709 12:12:47.790409 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.040s	sys 0.083s
I20260709 12:12:47.717341 12873 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:47.778278 11324 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.000s	sys 0.022s
I20260709 12:12:47.801956 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:47.802894 11175 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
W20260709 12:12:47.785763 12926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.021s	sys 0.039s
I20260709 12:12:47.712769 12975 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:47.800594 12896 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.007s	sys 0.011s
W20260709 12:12:47.811422 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.004s	sys 0.014s
W20260709 12:12:47.792047 13002 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.042s	sys 0.073s
W20260709 12:12:47.842751 11220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.012s	sys 0.007s
W20260709 12:12:47.869987 11220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.012s	sys 0.007s
W20260709 12:12:47.870232 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:47.802004 11324 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.000s	sys 0.022s
W20260709 12:12:47.817219 11253 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.013s	sys 0.025s
W20260709 12:12:47.849118 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53230.2us
W20260709 12:12:47.871537 11253 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.014s	sys 0.031s
W20260709 12:12:47.852886 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.004s	sys 0.017s
I20260709 12:12:47.804873 13020 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:47.882167 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:47.806910 12982 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:47.806715 12926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.021s	sys 0.040s
W20260709 12:12:47.808377 12896 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.008s	sys 0.011s
I20260709 12:12:47.858359 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:47.866168 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:47.903923 11173 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
W20260709 12:12:47.878283 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 743412us
W20260709 12:12:47.810509 12908 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.001s	sys 0.017s
I20260709 12:12:47.885442 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1240) took 1323 ms. Trace:
I20260709 12:12:47.854629 12895 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:47.906353 11250 rpcz_store.cc:276] 0709 12:12:46.562499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.563013 (+   514us) service_pool.cc:224] Handling call
0709 12:12:46.563376 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:46.581717 (+ 18341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:46.581754 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:46.582266 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:12:47.127833 (+545567us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:47.127891 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:47.142880 (+ 14989us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:47.885119 (+742239us) raft_consensus.cc:1685] finished
0709 12:12:47.885150 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:47.885359 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22275,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":7325,"thread_start_us":545137,"threads_started":1,"wal-append.queue_time_us":1189788}
I20260709 12:12:47.910349 12982 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:47.929584 11934 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 1 LEARNER]: Advancing to term 2
I20260709 12:12:47.930893 12982 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:47.934260 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:47.935187 12873 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:47.935341 11927 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:47.929658 11175 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:47.939692 13018 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.276s	user 0.017s	sys 0.018s
I20260709 12:12:47.941566 13042 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.941876 11928 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:47.942502 13042 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:47.946264 12968 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:47.948186 13002 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:47.951814 12586 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:47.862401 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:47.967146 11175 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:47.886528 13037 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:47.908442 12908 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.002s	sys 0.018s
W20260709 12:12:47.987828 11900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.009s	sys 0.024s
I20260709 12:12:47.972734 11934 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:12:47.879927 13045 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:47.945312 13042 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:47.962920 13048 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:47.927628 13038 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:12:47.985958 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:47.993688 11295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.008s	sys 0.039s
W20260709 12:12:47.995764 11900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.010s	sys 0.024s
I20260709 12:12:47.998227 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1522) took 2281 ms. Trace:
I20260709 12:12:48.000960 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:48.010321 11295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.008s	sys 0.039s
W20260709 12:12:48.010450 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 635329us
W20260709 12:12:48.011466 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.011622 11173 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:12:48.012720 11256 rpcz_store.cc:276] 0709 12:12:45.712279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.712448 (+   169us) service_pool.cc:224] Handling call
0709 12:12:45.757191 (+ 44743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:45.830723 (+ 73532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:45.830775 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:45.839914 (+  9139us) log.cc:844] Serialized 26 byte log entry
0709 12:12:46.354348 (+514434us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:46.354386 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:46.354470 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:47.938923 (+1584453us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:12:47.980973 (+ 42050us) raft_consensus.cc:1685] finished
0709 12:12:47.981011 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:47.993283 (+ 12272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28928,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":23262,"spinlock_wait_cycles":129197440,"thread_start_us":557527,"threads_started":2,"wal-append.queue_time_us":726815}
I20260709 12:12:48.035382 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:48.036240 11330 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:47.961359 12895 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.042205 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:48.054255 11328 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:48.056670 11330 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:47.908691 13020 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.058563 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.006686 12742 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:47.992690 13037 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:48.041659 12931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.004s	sys 0.047s
I20260709 12:12:48.030514 13054 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:48.055261 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:48.044319 13046 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.042517 12918 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.056680 12964 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:47.999034 13045 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.056928 13017 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:48.110357 13036 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:12:48.123698 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1688) took 1588 ms. Trace:
I20260709 12:12:48.129021 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:48.110123 12742 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:48.111560 12931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.008s	sys 0.047s
I20260709 12:12:48.003597 13048 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:48.137674 11331 rpcz_store.cc:276] 0709 12:12:46.535603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.535798 (+   195us) service_pool.cc:224] Handling call
0709 12:12:46.536249 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:46.536731 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:46.536771 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:46.537394 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:12:47.112655 (+575261us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:47.112701 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:47.112782 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:48.123394 (+1010612us) raft_consensus.cc:1685] finished
0709 12:12:48.123420 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:48.123620 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1420,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":146,"thread_start_us":548474,"threads_started":1,"wal-append.queue_time_us":964808}
I20260709 12:12:48.141698 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.142553 11173 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:48.143692 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:48.157179 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 334955us
I20260709 12:12:48.164324 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.140851 13055 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.167204 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.141374 13017 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.174724 11331 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:48.128057 13054 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.179045 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1689) took 1625 ms. Trace:
I20260709 12:12:48.178869 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:48.145181 12742 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.180656 13065 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:48.181054 13065 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:48.182127 13065 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.182528 12742 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:48.183432 13065 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:48.177700 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.183786 11926 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Advancing to term 3
W20260709 12:12:48.184194 13050 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:12:48.134567 12957 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.181177 11324 rpcz_store.cc:276] 0709 12:12:46.554012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.554218 (+   206us) service_pool.cc:224] Handling call
0709 12:12:47.016766 (+462548us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:47.017049 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:47.017311 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:12:47.879930 (+862619us) log.cc:844] Serialized 14 byte log entry
0709 12:12:48.178721 (+298791us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:48.178777 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:12:48.178820 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.178881 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:48.178988 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":842905,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":15792,"raft.queue_time_us":4569,"spinlock_wait_cycles":15244544,"thread_start_us":1153532,"threads_started":2}
I20260709 12:12:48.187063 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1692) took 1045 ms. Trace:
I20260709 12:12:48.189327 11173 rpcz_store.cc:276] 0709 12:12:47.142000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.177351 (+1035351us) service_pool.cc:224] Handling call
0709 12:12:48.186984 (+  9633us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:48.185981 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:48.164029 12918 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.185460 13048 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.190830 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 192) took 2615 ms. Trace:
I20260709 12:12:48.170567 12982 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.163791 12968 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:48.191252 11174 rpcz_store.cc:276] 0709 12:12:45.571871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.572017 (+   146us) service_pool.cc:224] Handling call
0709 12:12:45.609009 (+ 36992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:47.060912 (+1451903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.187356 (+1126444us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400256cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:48.187530 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3336823552}
I20260709 12:12:48.135562 13046 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:48.179162 13002 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:48.199193 11173 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:48.200023 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1691) took 1073 ms. Trace:
I20260709 12:12:48.200418 11173 rpcz_store.cc:276] 0709 12:12:47.126410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.198349 (+1071939us) service_pool.cc:224] Handling call
0709 12:12:48.198693 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.199700 (+  1007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.199955 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:48.202582 11935 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:48.191449 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:48.176512 13017 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.218183 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 155) took 1827 ms. Trace:
W20260709 12:12:48.215332 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55262us
I20260709 12:12:48.215338 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1241) took 1552 ms. Trace:
I20260709 12:12:48.218619 11253 rpcz_store.cc:276] 0709 12:12:46.390774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.427430 (+ 36656us) service_pool.cc:224] Handling call
0709 12:12:46.445669 (+ 18239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:46.445992 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:46.446018 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:46.446500 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:12:47.087981 (+641481us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:47.872080 (+784099us) log.cc:844] Serialized 14 byte log entry
0709 12:12:48.134779 (+262699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.134861 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:48.217956 (+ 83095us) raft_consensus.cc:1685] finished
0709 12:12:48.217979 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:48.218116 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":798275,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":309761,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":204,"thread_start_us":1650605,"threads_started":3,"wal-append.queue_time_us":1455646}
I20260709 12:12:48.219601 11251 rpcz_store.cc:276] 0709 12:12:46.664205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.719431 (+ 55226us) service_pool.cc:224] Handling call
0709 12:12:46.719795 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:46.720008 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:47.695991 (+975983us) log.cc:844] Serialized 14 byte log entry
0709 12:12:48.214973 (+518982us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:48.215024 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:48.215067 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.215128 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:48.215268 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1088991,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":134360,"spinlock_wait_cycles":18089728,"thread_start_us":1195589,"threads_started":3,"wal-append.queue_time_us":490679}
I20260709 12:12:48.227298 13066 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:48.237277 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:48.207472 13045 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.212881 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.177702 13055 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.250644 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:48.231220 12908 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.193531 12918 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.221947 13068 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:48.204278 12968 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.255439 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:48.255374 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54393us
I20260709 12:12:48.204208 12982 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:48.210232 13002 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.265697 13020 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.266345 13020 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:48.266528 13057 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:12:48.266880 13020 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.269096 13020 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.248113 13010 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:12:48.271219 12979 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:48.272454 12979 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:48.272900 12979 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.274866 12663 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:48.275740 12663 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:48.276355 12663 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.277417 12643 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:48.277912 12643 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:12:48.278153 13058 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:12:48.250005 12895 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:48.273356 12979 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:48.278882 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1670) took 3666 ms. Trace:
I20260709 12:12:48.285586 12895 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.286423 12663 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.286212 12895 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.287667 12643 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.288877 13055 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:48.289072 12643 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:48.288271 13059 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:12:48.290692 13054 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.291567 13046 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.292076 12895 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:48.293031 12910 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.179s	user 0.009s	sys 0.002s
I20260709 12:12:48.296031 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:48.260545 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 61116.3us
I20260709 12:12:48.302109 11173 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:12:48.305467 11328 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:48.282426 11179 rpcz_store.cc:276] 0709 12:12:44.612915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:44.613038 (+   123us) service_pool.cc:224] Handling call
0709 12:12:45.768749 (+1155711us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:45.768955 (+   206us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:45.771015 (+  2060us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:12:46.542845 (+771830us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b18002ef060. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:12:47.472885 (+930040us) log.cc:844] Serialized 14 byte log entry
0709 12:12:48.040441 (+567556us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:48.041184 (+   743us) log.cc:844] Serialized 26 byte log entry
0709 12:12:48.041346 (+   162us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:48.041393 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.041492 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:48.278666 (+237174us) raft_consensus.cc:1685] finished
0709 12:12:48.278687 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:48.278822 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1678,"prepare.queue_time_us":1114617,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2737,"raft.queue_time_us":2343,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":7187,"spinlock_wait_cycles":88509056,"thread_start_us":1231852,"threads_started":2,"wal-append.queue_time_us":648822}
I20260709 12:12:48.279404 12742 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:48.261319 13068 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:48.281121 12931 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.311614 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:48.312379 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.316764 11250 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:48.293051 13017 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:48.301481 13010 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:12:48.317929 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.258692 12908 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.328019 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.313552 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.341461 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:48.340678 11252 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:48.312456 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:48.318925 11248 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:48.345574 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:48.345602 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.293408 13037 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.319454 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.319804 11928 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:48.348809 11249 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:48.319177 12957 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.314428 13076 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:48.320510 11253 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:48.320565 13084 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.321292 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.321491 13068 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.321969 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.310835 12742 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.322543 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:48.301083 13064 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.307442 13046 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.323179 12968 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.323659 12918 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.307673 13045 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:48.329887 12908 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:48.331090 11181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.403s	user 0.020s	sys 0.018s
I20260709 12:12:48.313042 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:48.293483 12910 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.333701 13085 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:48.337054 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.294179 13071 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:12:48.343822 13086 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:12:48.358256 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 102884us
W20260709 12:12:48.567265 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:48.568221 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.344470 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:48.319996 12982 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.349871 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.282876 12975 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:12:48.317909 12931 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.343947 13002 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:48.578184 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 265211us
I20260709 12:12:48.350428 13079 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:48.365530 13076 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.579555 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:48.580302 13020 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.580900 13079 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:48.581745 13075 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.582405 13075 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.582700 13079 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.585224 13075 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:48.533762 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 289956us
W20260709 12:12:48.572429 11181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.644s	user 0.024s	sys 0.026s
I20260709 12:12:48.593981 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1552) took 2741 ms. Trace:
I20260709 12:12:48.594254 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:48.594506 11181 rpcz_store.cc:276] 0709 12:12:45.853057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.879220 (+ 26163us) service_pool.cc:224] Handling call
0709 12:12:45.879577 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:45.879939 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:45.879969 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:45.885363 (+  5394us) log.cc:844] Serialized 26 byte log entry
0709 12:12:48.593473 (+2708110us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:48.593530 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.593669 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:48.593731 (+    62us) raft_consensus.cc:1685] finished
0709 12:12:48.593750 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:48.593908 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":365,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":7989,"thread_start_us":2665170,"threads_started":1,"wal-append.queue_time_us":710840,"wal-append.run_cpu_time_us":19208,"wal-append.run_wall_time_us":1420784}
W20260709 12:12:48.433676 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74677.5us
I20260709 12:12:48.598199 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:48.566462 11255 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:48.586127 13045 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.586480 13076 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.520610 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:48.333757 11926 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:12:48.566112 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 111285us
I20260709 12:12:48.580842 13020 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.571240 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:48.576943 12643 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:48.600962 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 165074us
I20260709 12:12:48.566073 13086 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.595759 12742 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:48.588903 11900 connection.cc:256] RPC call timeout handler was delayed by -1.2712984100000995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 12:12:48.350572 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.578018 13072 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.580339 12896 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:48.615460 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.617865 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.617805 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:48.617865 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.654361 11255 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:48.655628 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.604786 13075 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.605494 11179 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:48.599143 13064 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.648583 11252 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:48.590780 13078 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.652002 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.588958 13071 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:48.661434 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:48.609261 13055 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:48.576752 13084 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.633303 13086 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:48.672492 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.609050 12975 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:48.684365 11249 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
I20260709 12:12:48.678834 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:48.671264 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.602056 13046 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.589175 13085 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.691685 13054 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:48.650985 12957 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.695670 13084 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.696987 12931 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.698197 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.699128 13020 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.699928 12910 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.700551 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.700799 12923 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:48.701371 12896 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:48.761027 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.702574 11182 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:48.762769 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1683) took 1654 ms. Trace:
I20260709 12:12:48.763221 11182 rpcz_store.cc:276] 0709 12:12:47.108261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.701689 (+1593428us) service_pool.cc:224] Handling call
0709 12:12:48.702158 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.762405 (+ 60247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.762679 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:48.673972 13072 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.705612 13045 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.689739 13078 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.707290 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.690469 13002 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:48.714386 12979 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.772997 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.774914 12979 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.634519 12643 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.741320 13064 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.744719 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.668967 11929 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:48.749040 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:48.753885 12957 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.609187 13083 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.772023 11252 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:48.772840 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.688045 13075 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.790514 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.795644 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.760094 12896 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:12:48.946837 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 197922us
I20260709 12:12:48.942992 11181 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:48.945895 11249 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:48.759016 12923 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:48.949544 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:48.794188 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.945295 11330 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:48.950891 11249 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:48.951205 11253 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:48.845800 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1693) took 1703 ms. Trace:
W20260709 12:12:48.945583 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 205423us
I20260709 12:12:48.789574 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.937896 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:48.952535 11182 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:48.952749 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:48.941824 13019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.556s	user 0.006s	sys 0.005s
I20260709 12:12:48.790788 13085 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.948978 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:48.946033 11932 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:12:48.952849 13099 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:48.796042 11250 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
I20260709 12:12:48.949671 12896 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:48.953315 11930 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:48.943842 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.775596 13078 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:48.797374 13072 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.970044 11180 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:48.970145 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:48.970649 11180 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 197) took 1865 ms (client timeout 2170 ms). Trace:
I20260709 12:12:48.882797 13010 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.955791 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:48.956781 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.971937 11178 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:48.972035 13078 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:12:48.972446 11178 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 198) took 1866 ms (client timeout 2126 ms). Trace:
I20260709 12:12:48.972465 13078 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.972999 13078 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:48.972868 11178 rpcz_store.cc:269] 0709 12:12:47.105546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.946344 (+1840798us) service_pool.cc:224] Handling call
0709 12:12:48.972391 (+ 26047us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:48.974046 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.974335 13086 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.952131 11173 rpcz_store.cc:276] 0709 12:12:47.142339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.211647 (+1069308us) service_pool.cc:224] Handling call
0709 12:12:48.211979 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.308653 (+ 96674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.806139 (+497486us) spinlock_profiling.cc:243] Waited 89.7 ms on lock 0x7b5400364f54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:48.845721 (+ 39582us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206209920}
I20260709 12:12:48.959784 13071 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:48.939675 13046 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.801669 12972 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:48.976079 12972 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:48.977092 13086 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:48.978080 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.978695 11256 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:48.795514 13037 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:48.979460 13037 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:48.979511 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.980294 11256 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 2: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:48.979944 12742 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.981139 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.791908 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1093) took 2880 ms. Trace:
I20260709 12:12:48.981794 11323 rpcz_store.cc:276] 0709 12:12:45.911291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.956959 (+ 45668us) service_pool.cc:224] Handling call
0709 12:12:45.957341 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:46.234351 (+277010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.791562 (+2557211us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54001c43d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:48.791791 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626365696}
W20260709 12:12:48.950928 11929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 399) took 2626 ms (client timeout 1993 ms). Trace:
I20260709 12:12:48.791355 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.966768 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:48.955657 13019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.572s	user 0.007s	sys 0.005s
I20260709 12:12:48.966157 13102 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:48.986212 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.986742 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.987391 11926 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:48.986075 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.988642 11256 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:48.982560 12742 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:48.989395 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:48.989984 11256 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:48.990437 11932 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:48.990964 13078 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:48.960707 12923 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:48.966365 13099 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:48.992588 13034 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.224s	user 0.001s	sys 0.002s
I20260709 12:12:48.992864 13099 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:48.961881 11928 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:48.974668 11178 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:48.975936 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:48.964196 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:48.957073 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.997287 11182 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:48.997383 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:48.957199 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:48.983295 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:48.971154 11180 rpcz_store.cc:269] 0709 12:12:47.105261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.804119 (+1698858us) service_pool.cc:224] Handling call
0709 12:12:48.970542 (+166423us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:48.984160 11929 rpcz_store.cc:269] 0709 12:12:46.318429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.318815 (+   386us) service_pool.cc:224] Handling call
0709 12:12:48.340688 (+2021873us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:48.945131 (+604443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20312320}
I20260709 12:12:48.999274 12742 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:48.999449 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:48.999748 12742 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.000166 12742 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:49.000201 11929 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Advancing to term 3
I20260709 12:12:48.983559 13010 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:49.001770 11328 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:48.980414 13037 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:49.002473 13037 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:49.002650 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.003103 13102 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:49.003599 13102 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.003971 13102 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:48.967643 13083 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:49.005342 13055 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:49.008119 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 394) took 2719 ms. Trace:
I20260709 12:12:49.008634 11933 rpcz_store.cc:276] 0709 12:12:46.275221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.275380 (+   159us) service_pool.cc:224] Handling call
0709 12:12:46.275712 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:46.276099 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:46.276138 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:46.276784 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 12:12:47.075745 (+798961us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:47.075805 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:47.075897 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:48.991982 (+1916085us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:12:48.994562 (+  2580us) raft_consensus.cc:1685] finished
0709 12:12:48.994583 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:48.994776 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7840,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":180,"spinlock_wait_cycles":8558848,"thread_start_us":798477,"threads_started":1,"wal-append.queue_time_us":1207862}
I20260709 12:12:48.994598 11249 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:48.990806 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:49.012933 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:49.013795 11256 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:49.014896 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:49.015291 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 444116us
W20260709 12:12:49.029743 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46543us
I20260709 12:12:49.028797 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.031424 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:49.035288 11182 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:49.033671 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:49.037354 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.037791 11173 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:49.039492 11328 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:49.030800 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:49.041109 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:49.041770 11249 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:49.043658 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.045296 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.032286 13076 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:49.046443 11181 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:49.051164 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.032138 12908 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:49.070209 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:49.032758 12975 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:49.068279 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:49.070924 11934 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:49.071918 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:49.049582 13076 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.024333 13019 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:12:49.088132 11182 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:49.091082 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.093982 11173 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:49.083844 13048 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:49.070257 13061 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:49.094765 11249 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:49.073994 12923 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:49.096594 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:49.070708 12908 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:49.046486 12742 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:49.074693 12975 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.083463 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:49.087988 12663 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:49.089869 13076 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:49.102017 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.124174 11934 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
W20260709 12:12:49.124276 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:49.125942 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 83895.5us
I20260709 12:12:49.126401 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.129415 11249 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:49.092798 13046 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:12:49.103001 12816 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:49.127597 11248 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:49.104562 12908 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:49.137118 13083 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:49.104393 12975 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:49.100587 12979 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:12:49.136550 13099 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:49.127846 12663 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:49.158246 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.158577 12663 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:49.160394 12896 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:49.145223 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.161520 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:49.162899 12896 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:49.164228 11930 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:49.164546 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.165068 11173 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:49.165943 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.166378 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:49.167068 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.167632 11173 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:49.167693 11926 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:49.168588 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.144345 13010 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:49.103147 13045 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.170085 13045 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:49.170799 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:49.152271 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.171952 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:49.173503 13045 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:49.173911 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:49.174350 11249 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:12:49.147158 12816 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:49.175815 11249 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:49.176040 12816 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:49.174567 11934 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 1.
I20260709 12:12:49.178021 13076 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:49.179203 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:49.180353 13076 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:49.183233 13099 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:49.171996 11182 raft_consensus.cc:2376] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4. Re-sending same reply.
W20260709 12:12:49.154707 13097 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:12:49.155382 11929 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:49.103242 13072 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:12:49.155956 12742 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:49.150233 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:49.184752 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:49.142136 12643 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:49.741333 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:49.742642 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 581161us
I20260709 12:12:49.742419 11182 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
W20260709 12:12:49.538951 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.745363 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1701) took 1175 ms. Trace:
W20260709 12:12:49.432972 13093 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.374s	user 0.000s	sys 0.002s
I20260709 12:12:49.296247 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:49.657804 13110 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.775414 13110 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:49.679931 12663 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 12:12:49.773449 11182 rpcz_store.cc:276] 0709 12:12:48.569523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.184595 (+615072us) service_pool.cc:224] Handling call
0709 12:12:49.745307 (+560712us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.525581 11926 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:12:49.772276 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 622082us
I20260709 12:12:49.779397 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1252) took 1130 ms. Trace:
W20260709 12:12:49.301695 13103 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:12:49.706244 13102 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.702s	user 0.005s	sys 0.011s
W20260709 12:12:49.592765 12972 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.001s
I20260709 12:12:49.185499 13019 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.781217 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1542) took 1461 ms. Trace:
W20260709 12:12:49.572230 13105 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 12:12:49.779873 11255 rpcz_store.cc:276] 0709 12:12:48.649270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.778526 (+1129256us) service_pool.cc:224] Handling call
0709 12:12:49.778849 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:49.779033 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:49.779060 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:49.779111 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:49.779141 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.779190 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.779307 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.780735 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:49.253036 13084 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.579s
I20260709 12:12:49.260183 13046 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:49.788240 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.793126 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1574) took 1004 ms. Trace:
I20260709 12:12:49.790179 11252 rpcz_store.cc:276] 0709 12:12:48.316643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.344184 (+ 27541us) service_pool.cc:224] Handling call
0709 12:12:49.777578 (+1433394us) spinlock_profiling.cc:243] Waited 78.6 ms on lock 0x7b5400277254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:49.777722 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180778752}
I20260709 12:12:49.794636 11182 rpcz_store.cc:276] 0709 12:12:48.831125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.777594 (+946469us) service_pool.cc:224] Handling call
0709 12:12:49.793044 (+ 15450us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:49.795562 11176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 195) took 2691 ms (client timeout 1567 ms). Trace:
I20260709 12:12:49.794193 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:49.795689 11252 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:12:49.785462 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:49.778379 12742 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:49.796571 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1547) took 1137 ms. Trace:
I20260709 12:12:49.797712 11252 rpcz_store.cc:276] 0709 12:12:48.658743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.795117 (+1136374us) service_pool.cc:224] Handling call
0709 12:12:49.795476 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:49.796318 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.796520 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.786460 12643 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:49.797320 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1546) took 1139 ms. Trace:
I20260709 12:12:49.799088 11252 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
W20260709 12:12:49.796030 11176 rpcz_store.cc:269] 0709 12:12:47.104658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.795399 (+2690741us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:49.795508 (+   109us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:49.799351 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.800771 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1253) took 1130 ms. Trace:
I20260709 12:12:49.801605 11252 rpcz_store.cc:276] 0709 12:12:48.670269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.798623 (+1128354us) service_pool.cc:224] Handling call
0709 12:12:49.798940 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:49.800494 (+  1554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.800719 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.800974 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.803064 11255 rpcz_store.cc:276] 0709 12:12:48.658437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.791702 (+1133265us) service_pool.cc:224] Handling call
0709 12:12:49.792187 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:49.797029 (+  4842us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:49.797066 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:49.797107 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:49.797136 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.797199 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.797279 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.805505 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 174) took 1088 ms. Trace:
I20260709 12:12:49.806054 11255 rpcz_store.cc:276] 0709 12:12:48.717152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.804428 (+1087276us) service_pool.cc:224] Handling call
0709 12:12:49.804838 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:49.805048 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:49.805101 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:49.805161 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:49.805206 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.805285 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.805434 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.800830 11173 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:49.810338 11176 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:49.782058 12663 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:49.811573 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.785686 13102 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:49.814483 11255 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:49.815284 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.817080 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1557) took 1481 ms. Trace:
I20260709 12:12:49.817590 11182 rpcz_store.cc:276] 0709 12:12:48.335636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.799910 (+1464274us) service_pool.cc:224] Handling call
0709 12:12:49.817014 (+ 17104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.818024 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1572) took 1052 ms. Trace:
I20260709 12:12:49.819093 11176 rpcz_store.cc:276] 0709 12:12:48.765445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.811360 (+1045915us) service_pool.cc:224] Handling call
0709 12:12:49.817964 (+  6604us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.810943 13046 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:49.818754 11255 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:49.820187 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:49.812922 12742 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:49.820961 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.820886 13110 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:49.789503 12972 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:49.822748 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1558) took 1486 ms. Trace:
I20260709 12:12:49.823036 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 469) took 1152 ms. Trace:
I20260709 12:12:49.827292 11932 rpcz_store.cc:276] 0709 12:12:48.670891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.991456 (+320565us) service_pool.cc:224] Handling call
0709 12:12:48.991845 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:48.992398 (+   553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:48.992454 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:48.993132 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 12:12:49.086771 (+ 93639us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:49.086809 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.086902 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:49.822830 (+735928us) raft_consensus.cc:1685] finished
0709 12:12:49.822853 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.822980 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":93167,"threads_started":1,"wal-append.queue_time_us":137185}
I20260709 12:12:49.829555 11176 rpcz_store.cc:276] 0709 12:12:48.335882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.819947 (+1484065us) service_pool.cc:224] Handling call
0709 12:12:49.822688 (+  2741us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.811483 12643 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:49.824859 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1563) took 1487 ms. Trace:
I20260709 12:12:49.824597 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:49.794863 13019 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:49.831096 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:49.834888 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 205) took 1247 ms. Trace:
I20260709 12:12:49.823477 13046 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:49.833667 11177 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1682) took 2725 ms (client timeout 2231 ms). Trace:
I20260709 12:12:49.833902 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 461) took 1910 ms. Trace:
W20260709 12:12:49.837996 11177 rpcz_store.cc:269] 0709 12:12:47.107934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.833355 (+2725421us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:49.833532 (+   177us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:49.838063 11935 rpcz_store.cc:276] 0709 12:12:47.923252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.923447 (+   195us) service_pool.cc:224] Handling call
0709 12:12:47.923778 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.203153 (+279375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.833660 (+1630507us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b540059a6d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:49.833846 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640590720}
I20260709 12:12:49.838938 13046 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:49.840428 13075 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:49.824656 11255 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:49.835216 11176 raft_consensus.cc:606] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:49.831881 11182 rpcz_store.cc:276] 0709 12:12:48.337127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.818278 (+1481151us) service_pool.cc:224] Handling call
0709 12:12:49.824798 (+  6520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.472013 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1554) took 1432 ms. Trace:
I20260709 12:12:50.472041 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:51810 (request call id 7) took 1539 ms. Trace:
I20260709 12:12:49.836184 11173 rpcz_store.cc:276] 0709 12:12:48.587229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.814212 (+1226983us) service_pool.cc:224] Handling call
0709 12:12:49.834800 (+ 20588us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.472535 11255 rpcz_store.cc:276] 0709 12:12:49.039291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.822838 (+783547us) service_pool.cc:224] Handling call
0709 12:12:49.823139 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.471729 (+648590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.471946 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:49.847811 12908 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.000s
W20260709 12:12:49.885985 13109 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:12:50.475935 12908 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.476812 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1541) took 2161 ms. Trace:
I20260709 12:12:50.478016 12908 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:50.477200 11251 rpcz_store.cc:276] 0709 12:12:48.315578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.351062 (+ 35484us) service_pool.cc:224] Handling call
0709 12:12:48.351387 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:48.351876 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:48.351912 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:49.133453 (+781541us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b18001b9ac0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a
0709 12:12:49.134997 (+  1544us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b2c00000e98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:12:49.135477 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:12:49.500917 (+365440us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:49.500970 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.501071 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:49.958922 (+457851us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b4800042b58. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:12:50.476553 (+517631us) raft_consensus.cc:1685] finished
0709 12:12:50.476578 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.476736 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":43612,"raft.run_wall_time_us":1602917,"spinlock_wait_cycles":64098560,"thread_start_us":365033,"threads_started":1,"wal-append.queue_time_us":650445}
I20260709 12:12:49.885838 13102 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:49.900475 13060 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:12:49.917716 12975 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.732s	user 0.004s	sys 0.010s
I20260709 12:12:49.928247 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1553) took 1593 ms. Trace:
I20260709 12:12:50.563822 12975 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:50.563858 11177 rpcz_store.cc:276] 0709 12:12:48.334512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.838994 (+1504482us) service_pool.cc:224] Handling call
0709 12:12:49.839383 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:49.839613 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:49.839643 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:49.840254 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:12:49.861910 (+ 21656us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:49.861939 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.861995 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:49.927957 (+ 65962us) raft_consensus.cc:1685] finished
0709 12:12:49.927980 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.928170 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":21211,"threads_started":1,"wal-append.queue_time_us":44550}
I20260709 12:12:49.947811 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:49.947674 13115 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:12:50.566077 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 618302us
W20260709 12:12:49.972954 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.010s	user 0.027s	sys 0.043s
W20260709 12:12:50.567457 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.605s	user 0.027s	sys 0.043s
I20260709 12:12:50.567186 12959 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:50.569913 12975 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:50.030643 11175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 200) took 2651 ms (client timeout 1820 ms). Trace:
W20260709 12:12:50.571885 11175 rpcz_store.cc:269] 0709 12:12:47.379498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.842876 (+463378us) service_pool.cc:224] Handling call
0709 12:12:50.030378 (+2187502us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b54002e40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:50.030540 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59564416}
I20260709 12:12:50.572324 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.573729 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.036552 11178 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:50.575034 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1561) took 2238 ms. Trace:
I20260709 12:12:50.050529 11252 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:50.065600 11076 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 3, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:50.078457 12643 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:50.087674 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1555) took 1048 ms. Trace:
I20260709 12:12:50.583045 11253 rpcz_store.cc:276] 0709 12:12:49.039459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.083179 (+1043720us) service_pool.cc:224] Handling call
0709 12:12:50.083591 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.083842 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:12:50.085384 (+  1542us) log.cc:844] Serialized 14 byte log entry
0709 12:12:50.087175 (+  1791us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:50.087281 (+   106us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:50.087370 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.087476 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.087609 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1524,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":35,"raft.queue_time_us":95,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":2204,"threads_started":2,"wal-append.queue_time_us":1312}
W20260709 12:12:50.388218 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.585937 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:50.442401 13078 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:50.448426 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1694) took 3305 ms. Trace:
I20260709 12:12:50.594659 11179 rpcz_store.cc:276] 0709 12:12:47.143546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.309218 (+1165672us) service_pool.cc:224] Handling call
0709 12:12:48.346004 (+ 36786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.669812 (+323808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.448101 (+1778289us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54000cd854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:50.448343 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595690240}
W20260709 12:12:50.449366 12816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.021s	sys 0.014s
I20260709 12:12:49.904999 12972 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:50.465075 11935 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:50.468648 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 344) took 1298 ms. Trace:
I20260709 12:12:50.473080 11182 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:49.827313 13111 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:50.474273 11255 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:12:50.608222 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1556) took 1568 ms. Trace:
I20260709 12:12:50.577343 13115 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:50.609948 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1562) took 2273 ms. Trace:
I20260709 12:12:50.610383 11182 rpcz_store.cc:276] 0709 12:12:48.336895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.472440 (+2135545us) service_pool.cc:224] Handling call
0709 12:12:50.472788 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.609573 (+136785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.609889 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.472628 11176 rpcz_store.cc:276] 0709 12:12:48.933091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.830955 (+897864us) service_pool.cc:224] Handling call
0709 12:12:50.471965 (+641010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.872818 13046 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.479110 13060 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:50.482564 13102 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.484315 11324 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:12:50.484969 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.485929 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.615171 11325 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:50.488137 13074 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.493672 12663 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:50.496677 12957 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:50.501717 13007 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:50.620060 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.621512 11323 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:49.901799 13019 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:50.626379 13060 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.611668 11926 rpcz_store.cc:276] 0709 12:12:49.170082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.170527 (+   445us) service_pool.cc:224] Handling call
0709 12:12:50.468387 (+1297860us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54001bf154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:50.468542 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754336000}
I20260709 12:12:50.627696 13060 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.613665 13102 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.628918 11330 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:12:50.629253 11330 pending_rounds.cc:85] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:12:50.511535 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.629706 11330 pending_rounds.cc:107] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:12:50.629853 13102 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:50.630018 11330 raft_consensus.cc:2889] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:50.630280 11322 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:12:50.514314 11328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 235) took 1567 ms (client timeout 1901 ms). Trace:
W20260709 12:12:50.526098 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.631402 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.631315 13074 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.632318 11322 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 2: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:50.632464 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:50.525238 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.012s	sys 0.029s
I20260709 12:12:50.526966 11180 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:50.634243 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1566) took 2288 ms. Trace:
I20260709 12:12:50.634598 11180 rpcz_store.cc:276] 0709 12:12:48.345846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.526116 (+2180270us) service_pool.cc:224] Handling call
0709 12:12:50.526553 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.633982 (+107429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.634184 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.633615 13074 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:50.532563 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.635847 11331 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.636140 13078 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:50.636664 13078 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.637892 12908 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:50.638312 12908 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:50.638692 12908 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:50.639570 11175 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:50.640396 13128 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:50.544543 13017 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:50.641799 13017 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:12:50.630897 11328 rpcz_store.cc:269] 0709 12:12:48.946598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.946840 (+   242us) service_pool.cc:224] Handling call
0709 12:12:50.514077 (+1567237us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b5400428bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:50.514209 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111220480}
I20260709 12:12:50.641324 11076 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 2 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:50.643059 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.644335 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.644945 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:50.644423 11076 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:50.560914 11248 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:50.646201 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1244) took 2007 ms. Trace:
I20260709 12:12:50.646411 12663 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.001675 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1685) took 2892 ms. Trace:
W20260709 12:12:50.041806 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:50.571905 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 168) took 1958 ms (client timeout 2168 ms). Trace:
W20260709 12:12:50.648043 11250 rpcz_store.cc:269] 0709 12:12:48.613593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.615175 (+  1582us) service_pool.cc:224] Handling call
0709 12:12:50.571721 (+1956546us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540010c054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:50.571844 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329312128}
W20260709 12:12:50.648525 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.023s	sys 0.031s
I20260709 12:12:50.581674 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:50.649099 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.023s	sys 0.031s
I20260709 12:12:50.649292 11331 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:50.648836 11076 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 2 to 3, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 3 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:50.649791 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.650252 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.650832 11331 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:50.650898 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.647572 11078 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 240) took 7061 ms. Trace:
W20260709 12:12:50.438791 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.006s	sys 0.009s
I20260709 12:12:50.589995 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:50.650635 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 879592us
W20260709 12:12:50.652458 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.006s	sys 0.009s
I20260709 12:12:50.652565 11251 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:12:50.652896 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 867426us
I20260709 12:12:50.591058 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:50.653484 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:50.653863 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 62965.5us
I20260709 12:12:50.654495 12981 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.654633 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:50.655094 12981 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.655694 12643 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:50.656186 12643 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.656544 12643 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.657239 12981 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:50.608618 11255 rpcz_store.cc:276] 0709 12:12:49.039621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.473746 (+1434125us) service_pool.cc:224] Handling call
0709 12:12:50.474050 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:50.607875 (+133825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.608136 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.474436 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1564) took 2137 ms. Trace:
I20260709 12:12:50.658084 11173 rpcz_store.cc:276] 0709 12:12:48.337383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.473412 (+2136029us) service_pool.cc:224] Handling call
0709 12:12:50.473808 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.474106 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:50.474135 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:50.474179 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:50.474212 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.474268 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.474378 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":829,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":132}
W20260709 12:12:50.597749 12816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.022s	sys 0.014s
I20260709 12:12:50.660037 11181 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:50.577194 11178 rpcz_store.cc:276] 0709 12:12:48.336665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.035776 (+1699111us) service_pool.cc:224] Handling call
0709 12:12:50.036131 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.574748 (+538617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.574969 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.616919 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.662590 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:50.662995 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.652060 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.663746 11329 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:50.664048 13110 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:50.664114 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.664700 13110 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.618506 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.665830 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.642652 13017 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.629153 13046 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:50.666792 13017 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.666934 13046 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:50.667495 13046 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:50.667045 13110 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:50.667994 13046 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:50.668160 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.668960 11331 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
W20260709 12:12:50.555081 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.667351 12643 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.588016 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:50.634778 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:50.666056 11325 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:50.612185 11327 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:50.671109 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1555) took 2336 ms. Trace:
I20260709 12:12:50.665354 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.671459 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.671942 11324 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:12:50.558740 11927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 404) took 2389 ms (client timeout 1556 ms). Trace:
I20260709 12:12:50.670413 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.672426 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.672642 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:50.672847 11927 rpcz_store.cc:269] 0709 12:12:48.169503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.185712 (+ 16209us) service_pool.cc:224] Handling call
0709 12:12:50.558457 (+2372745us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400650ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:50.558643 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593532672}
I20260709 12:12:50.570358 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.673998 11326 raft_consensus.cc:2376] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5. Re-sending same reply.
I20260709 12:12:50.674189 13111 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.674624 13111 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:50.675124 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.672551 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.671479 11181 rpcz_store.cc:276] 0709 12:12:48.335105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.898499 (+1563394us) service_pool.cc:224] Handling call
0709 12:12:49.898978 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.660420 (+761442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.670924 (+ 10504us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400347bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:50.671056 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749790080}
I20260709 12:12:50.673504 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.676208 11325 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:50.673298 11327 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:50.673276 11324 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:50.677198 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.679296 11325 raft_consensus.cc:2410] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:50.633303 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.679821 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1250) took 2039 ms. Trace:
I20260709 12:12:50.696420 11249 rpcz_store.cc:276] 0709 12:12:48.640090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.453605 (+813515us) service_pool.cc:224] Handling call
0709 12:12:50.575499 (+1121894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.575736 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:50.575773 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:50.575834 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:50.575886 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.575961 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.679648 (+103687us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b540010c050. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:50.679765 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917546752}
W20260709 12:12:50.682723 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.700001 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.701284 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:50.704033 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:50.682966 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.012s	sys 0.031s
W20260709 12:12:50.705332 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 911144us
I20260709 12:12:50.647058 11174 rpcz_store.cc:276] 0709 12:12:47.108825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.196342 (+1087517us) service_pool.cc:224] Handling call
0709 12:12:48.196700 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:48.326516 (+129816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:48.326548 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:48.352433 (+ 25885us) log.cc:844] Serialized 26 byte log entry
0709 12:12:48.776006 (+423573us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:48.776048 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.001160 (+1225112us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b54003d1dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:50.001225 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:50.001280 (+    55us) raft_consensus.cc:1685] finished
0709 12:12:50.001308 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.001568 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42086,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":164,"spinlock_wait_cycles":152067072,"thread_start_us":241193,"threads_started":2,"wal-append.queue_time_us":476857}
I20260709 12:12:50.686839 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1576) took 1661 ms. Trace:
I20260709 12:12:50.679080 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.708033 11175 rpcz_store.cc:276] 0709 12:12:49.024962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.572712 (+1547750us) service_pool.cc:224] Handling call
0709 12:12:50.572986 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.639952 (+ 66966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.686544 (+ 46592us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b54003d22d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:50.686760 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152396032}
I20260709 12:12:50.709415 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.712594 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:50.714741 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:50.700659 11173 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:50.721886 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.721886 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.723920 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1254) took 2021 ms. Trace:
I20260709 12:12:50.724421 11179 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:50.724445 11252 rpcz_store.cc:276] 0709 12:12:48.702901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.802450 (+1099549us) service_pool.cc:224] Handling call
0709 12:12:49.802822 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.577478 (+774656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.695645 (+118167us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54000f87d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:50.723831 (+ 28186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569071872}
I20260709 12:12:50.715757 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:50.727169 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 135026us
I20260709 12:12:50.726851 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:50.731606 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.752038 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.752310 11326 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:50.753194 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 162) took 4310 ms. Trace:
I20260709 12:12:50.650635 11076 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:12:50.752662 11181 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:50.754531 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:50.727669 12972 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:12:50.753665 11254 rpcz_store.cc:276] 0709 12:12:46.442782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.447803 (+  5021us) service_pool.cc:224] Handling call
0709 12:12:46.448080 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:46.610181 (+162101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:46.610246 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:46.610863 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:12:47.781179 (+1170316us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:48.042030 (+260851us) log.cc:844] Serialized 14 byte log entry
0709 12:12:48.042302 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.678633 (+2636331us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54000f82d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:50.678687 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:50.678749 (+    62us) raft_consensus.cc:1685] finished
0709 12:12:50.752955 (+ 74206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.753106 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":162467,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":152301,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":33781,"spinlock_wait_cycles":371580800,"thread_start_us":1233724,"threads_started":2,"wal-append.queue_time_us":1199793,"wal-append.run_cpu_time_us":3003,"wal-append.run_wall_time_us":1262059}
I20260709 12:12:50.721738 12957 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:50.721921 12981 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:50.759277 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:50.758584 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 123866us
I20260709 12:12:50.776949 11181 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:50.821192 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.822268 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.822505 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:50.749720 13110 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:50.749529 13074 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:50.822954 11327 raft_consensus.cc:2410] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:12:50.651684 11078 rpcz_store.cc:276] 0709 12:12:43.586310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.586640 (+   330us) service_pool.cc:224] Handling call
0709 12:12:50.647506 (+7060866us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:44.375850 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:44.376564 (+   714us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:44.376624 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:12:44.376645 (+    21us) tablet.cc:662] Decoding operations
0709 12:12:44.377428 (+   783us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:44.377524 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:44.377555 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:12:44.378201 (+   646us) tablet.cc:701] Row locks acquired
0709 12:12:44.378256 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:12:44.378444 (+   188us) write_op.cc:270] Start()
0709 12:12:44.378591 (+   147us) write_op.cc:276] Timestamp: P: 1783599164378379 usec, L: 0
0709 12:12:44.378630 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:12:44.404976 (+ 26346us) log.cc:844] Serialized 1749 byte log entry
0709 12:12:44.901484 (+496508us) op_driver.cc:464] REPLICATION: finished
0709 12:12:45.155992 (+254508us) write_op.cc:301] APPLY: starting
0709 12:12:45.156095 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:45.156285 (+   190us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:45.156449 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:45.172502 (+ 16053us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:45.405817 (+233315us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:12:45.418593 (+ 12776us) write_op.cc:312] APPLY: finished
0709 12:12:45.418968 (+   375us) log.cc:844] Serialized 41 byte log entry
0709 12:12:45.419176 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:45.419287 (+   111us) write_op.cc:454] Released schema lock
0709 12:12:45.419350 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:45.419767 (+   417us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:45.419804 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1604,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":240052,"apply.run_cpu_time_us":3471,"apply.run_wall_time_us":263739,"num_ops":4,"prepare.queue_time_us":202448,"prepare.run_cpu_time_us":10795,"prepare.run_wall_time_us":89845,"raft.queue_time_us":240670,"raft.run_cpu_time_us":15752,"raft.run_wall_time_us":251803,"replication_time_us":448748,"thread_start_us":474689,"threads_started":4,"wal-append.queue_time_us":121035,"wal-append.run_cpu_time_us":16132,"wal-append.run_wall_time_us":1899791}]]}
W20260709 12:12:50.816660 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 162089us
W20260709 12:12:50.818055 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 163470us
I20260709 12:12:50.822926 11174 raft_consensus.cc:2376] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4. Re-sending same reply.
I20260709 12:12:50.694073 12663 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.826983 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:50.758613 13127 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:50.763937 13102 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:50.762560 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.842968 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:50.764001 13019 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:50.850411 11932 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.840723 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.851888 11181 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:50.853088 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.824337 12981 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.844611 13074 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:50.832075 13137 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:50.827415 13110 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:50.844075 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:50.844997 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 85783.9us
I20260709 12:12:50.854847 11181 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:50.813966 13133 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:50.854909 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.751653 13125 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:12:50.813166 13136 raft_consensus.cc:2764] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:12:50.840060 13129 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:12:50.840013 12816 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:50.860342 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 126759us
I20260709 12:12:50.818310 13134 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.832188 11322 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:12:50.646706 11248 rpcz_store.cc:276] 0709 12:12:48.638413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.175170 (+536757us) service_pool.cc:224] Handling call
0709 12:12:50.560438 (+1385268us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b5800196898. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:50.560690 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.645863 (+ 85173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.646125 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3467136}
I20260709 12:12:50.868481 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.871440 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.872637 13115 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.874522 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.889693 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.890318 11324 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:50.890614 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.845957 13094 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:50.849788 13139 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.854579 12960 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:50.863644 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:50.831910 13132 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:50.849920 13102 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:50.855046 13019 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:50.865473 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.891157 11174 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:50.895669 11329 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:50.898171 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:50.944427 11254 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:50.898779 11934 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:50.944362 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.899140 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:50.945472 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.946095 11255 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:50.899410 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:50.901839 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:50.946125 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.881786 13133 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.866360 13110 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.903646 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.854782 13127 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.882314 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:50.887998 11323 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:50.905699 12816 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.906276 13136 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.886976 13055 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:12:50.907178 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:50.907946 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:50.908927 11929 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:50.863289 13130 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.910656 13074 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.860199 12981 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:50.914785 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.881776 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.938140 12972 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:12:50.929140 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.934698 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.941295 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:50.946555 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.902777 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:50.947273 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:50.948300 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.949538 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.946973 11253 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:50.956315 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.929896 13094 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:50.912906 13115 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:50.912146 12643 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:50.960381 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:51.063086 11254 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.888885 12967 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:12:50.974227 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:50.881523 13099 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:12:50.915695 12957 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:50.954282 13102 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.915069 12958 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
W20260709 12:12:50.920579 11203 debug-util.cc:398] Leaking SignalData structure 0x7b080011ae20 after lost signal to thread 10014
I20260709 12:12:50.955312 13019 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:51.149210 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.988919 11256 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:50.915686 12873 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:12:51.058668 11249 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:51.042073 11255 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:51.059543 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:51.059751 11328 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:12:51.060123 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0061902049999389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
I20260709 12:12:51.060449 11182 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:51.023041 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:51.059805 11175 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 1.
I20260709 12:12:51.061991 11325 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:51.007992 11932 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:51.047089 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:50.973546 13017 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.941795 13139 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.208582 11325 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:51.209100 11255 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
W20260709 12:12:50.957443 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 130524us
I20260709 12:12:50.914860 13134 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:50.976963 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:51.064246 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 139832us
I20260709 12:12:51.177953 11250 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:12:51.097307 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 206) took 2506 ms. Trace:
I20260709 12:12:50.991938 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 165) took 2379 ms. Trace:
I20260709 12:12:50.969643 11324 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:51.218173 11177 rpcz_store.cc:276] 0709 12:12:48.591113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.566253 (+1975140us) service_pool.cc:224] Handling call
0709 12:12:50.566683 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.566966 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:50.686272 (+119306us) log.cc:844] Serialized 14 byte log entry
0709 12:12:51.096737 (+410465us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:51.096817 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:51.096901 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.097032 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:51.097191 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":203545,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":60,"thread_start_us":528915,"threads_started":2}
I20260709 12:12:51.063979 11330 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:51.218362 11257 rpcz_store.cc:276] 0709 12:12:48.612686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.615039 (+  2353us) service_pool.cc:224] Handling call
0709 12:12:48.632939 (+ 17900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:49.033003 (+400064us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:49.092796 (+ 59793us) log.cc:844] Serialized 14 byte log entry
0709 12:12:50.580895 (+1488099us) spinlock_profiling.cc:243] Waited 9.9 ms on lock 0x7b180043fba0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:12:50.661321 (+ 80426us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:50.661810 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:12:50.663001 (+  1191us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:50.731524 (+ 68523us) log.cc:844] Serialized 14 byte log entry
0709 12:12:50.974217 (+242693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.974319 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:50.991681 (+ 17362us) raft_consensus.cc:1685] finished
0709 12:12:50.991704 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.991859 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":415787,"prepare.queue_time_us":216846,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":971,"raft.queue_time_us":491059,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":15021,"spinlock_wait_cycles":22781184,"thread_start_us":434405,"threads_started":4,"wal-append.queue_time_us":1514608}
I20260709 12:12:50.970329 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:51.128820 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:50.977962 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:51.061568 11173 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:51.079814 11331 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:12:51.223507 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:50.959964 13141 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:12:51.228811 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 384854us
I20260709 12:12:51.224802 11255 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:51.225690 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:51.230082 11329 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:51.230265 11181 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:51.231259 11256 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:51.231405 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:51.231973 11253 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:51.236711 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:51.235038 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:51.034363 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:51.210567 12908 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.004s
I20260709 12:12:51.254663 11257 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:12:51.130227 11322 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:51.130897 12959 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:51.245298 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:50.977451 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:50.978034 13078 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:50.978740 12975 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:51.248457 11182 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:51.134097 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:51.249446 11179 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:51.250136 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:51.218067 12643 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:51.263329 11249 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
W20260709 12:12:51.264045 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 377608us
I20260709 12:12:51.270030 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:51.271505 11326 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:51.298492 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:51.299681 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:12:51.273259 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:51.276191 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49369.9us
I20260709 12:12:51.300748 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:51.284034 13010 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:51.245895 12958 consensus_queue.cc:1243] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:12:51.275146 13153 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:50.992733 13136 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:51.267899 11203 debug-util.cc:398] Leaking SignalData structure 0x7b080011a640 after lost signal to thread 11113
W20260709 12:12:51.283890 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.258142 13141 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:51.258576 12908 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.257529 12663 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.227067 13017 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.258318 13133 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:51.379298 13106 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:51.113521 13130 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:51.355453 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57973.1us
W20260709 12:12:51.389422 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.274237 12959 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.433683 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:51.350564 12960 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:51.354454 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:51.296407 13002 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:51.389365 11148 connection.cc:256] RPC call timeout handler was delayed by -3.0875847679999424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 12:12:51.317045 13010 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:51.351227 13096 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:51.317188 13078 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:51.430135 13106 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.389252 12967 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:51.469061 12967 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:51.461714 11178 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:51.470079 12967 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.471802 13106 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:51.472007 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:51.472808 11931 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:51.462873 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:51.476213 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:50.969854 13132 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.388387 13142 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.385622 13153 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:51.399925 11203 debug-util.cc:398] Leaking SignalData structure 0x7b080011a560 after lost signal to thread 11127
W20260709 12:12:51.295820 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45751.8us
I20260709 12:12:51.418097 13141 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.298825 12975 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:51.441715 11148 connection.cc:256] RPC call timeout handler was delayed by -3.0083847680000417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 12:12:51.496165 11148 connection.cc:256] RPC call timeout handler was delayed by -2.9300847679999151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 12:12:51.496556 11148 connection.cc:256] RPC call timeout handler was delayed by -2.9201847679998991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 12:12:51.495148 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 194462us
I20260709 12:12:51.451130 11176 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:12:51.451522 11174 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 12:12:51.296407 12643 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:51.463042 13134 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.425403 13150 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:51.388554 12957 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.450300 12816 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.448046 13010 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:51.455452 13159 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:12:51.456377 13127 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.467630 13164 raft_consensus.cc:2764] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:12:51.468007 12959 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.468526 11927 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:12:51.484489 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:51.553508 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69109.8us
I20260709 12:12:51.492849 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:51.494341 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 139982us
W20260709 12:12:51.496884 11148 connection.cc:256] RPC call timeout handler was delayed by -2.6411847679999028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
I20260709 12:12:51.555018 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:51.555683 11148 connection.cc:256] RPC call timeout handler was delayed by -2.0102847680000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2947s for a response
I20260709 12:12:51.556183 12972 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:51.556504 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 321567us
I20260709 12:12:51.556793 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:51.445864 11328 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:51.557462 13156 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:51.469082 13017 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:51.506299 13111 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.831s	user 0.011s	sys 0.002s
I20260709 12:12:51.558630 13084 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:51.558876 13111 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.463058 13157 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:51.560101 13157 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.560504 13157 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:51.454571 11179 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:51.465025 13110 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.515244 12960 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:12:51.564973 12960 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:51.518560 13150 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:51.565413 12972 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:51.565855 13150 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:51.496243 12663 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:12:51.502051 13046 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.003s
I20260709 12:12:51.490875 13153 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:51.482160 13160 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:51.448904 13002 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.460748 12981 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.500681 13141 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:51.584692 11175 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:51.517035 12643 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:51.497614 11174 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:51.439358 13146 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
W20260709 12:12:51.791864 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 236341us
I20260709 12:12:51.563390 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:51.565359 12960 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:51.801476 13130 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:51.801918 12960 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.802666 13010 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.803211 12960 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:51.572247 13110 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.814415 12981 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:51.815546 13148 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.381s	user 0.002s	sys 0.003s
I20260709 12:12:51.816098 12975 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.816512 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:51.804021 13147 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:51.818308 11257 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:51.818727 13147 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.575335 13157 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.815981 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:51.821172 13147 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.821242 13157 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.566952 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:51.515105 13170 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:12:51.499794 13133 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:12:51.490722 13142 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:51.554939 11203 debug-util.cc:398] Leaking SignalData structure 0x7b08003b75c0 after lost signal to thread 11206
I20260709 12:12:51.567996 13134 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:51.837738 13142 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:51.654366 13154 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:12:51.597225 13139 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.598924 13160 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:51.583206 13046 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:51.656599 13151 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:12:51.545360 12967 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
W20260709 12:12:51.790092 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.499728 13144 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 12:12:51.515614 13001 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 12:12:51.514232 13094 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:51.797806 13102 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.012s
I20260709 12:12:51.791620 13099 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.792160 13155 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:51.798516 13141 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.800001 11926 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:51.800240 13078 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:51.800468 11322 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:12:51.801061 13019 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.006s
I20260709 12:12:51.807365 13010 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.808311 13002 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:51.808574 13163 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:51.809829 13161 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:51.811012 12643 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.811539 13106 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.818770 13153 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:51.821522 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:51.578197 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 85487.4us
W20260709 12:12:51.791033 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 234716us
I20260709 12:12:51.826176 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:51.881057 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:51.883842 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:51.826496 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:51.827045 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 263887us
I20260709 12:12:51.885121 11934 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:51.885538 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:51.885888 13141 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.871368 13162 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:12:51.828045 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:51.829113 13149 raft_consensus.cc:993] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:51.828984 12816 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:51.827364 13125 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:51.828980 13115 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.681s	user 0.015s	sys 0.002s
I20260709 12:12:51.829753 13166 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:51.831089 13111 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.830987 13167 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:51.835479 11255 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:51.827080 13173 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:51.836990 12972 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:12:51.837148 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 270255us
I20260709 12:12:51.948179 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1581) took 1051 ms. Trace:
I20260709 12:12:51.790411 13007 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:51.948946 13007 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:51.949810 12923 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:51.950361 12923 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:51.949923 13007 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.951313 13007 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:51.951195 12923 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.951512 13173 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.952435 12923 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:51.792093 13164 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 12:12:51.839478 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:51.840919 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 324245us
I20260709 12:12:51.852712 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1117) took 1226 ms. Trace:
W20260709 12:12:51.827028 13165 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.228s	user 0.000s	sys 0.016s
I20260709 12:12:51.839061 12975 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.955492 12975 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.956266 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:51.957140 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:51.957306 13111 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:51.957787 13111 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:51.958077 12975 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:51.958137 13131 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:51.958433 13173 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.958721 13131 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:51.860699 13102 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.862252 13160 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.814476 13150 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.858114 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 348) took 1021 ms. Trace:
I20260709 12:12:51.866885 11175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab915731e0347ae8353a7f300c44401"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:51.965973 11175 raft_consensus.cc:606] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:51.966308 11175 raft_consensus.cc:614] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:51.967180 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:51.968029 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:51.860817 13139 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:12:51.870966 13019 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.875142 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1718) took 1038 ms. Trace:
I20260709 12:12:51.874373 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1721) took 1013 ms. Trace:
I20260709 12:12:51.877022 13181 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:51.860813 13155 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:51.976372 13155 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.856869 13136 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:51.869294 11203 debug-util.cc:398] Leaking SignalData structure 0x7b08000f76e0 after lost signal to thread 11281
I20260709 12:12:51.887517 13094 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:51.888182 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:12:51.978191 11203 debug-util.cc:398] Leaking SignalData structure 0x7b0800326ac0 after lost signal to thread 11282
I20260709 12:12:51.888315 11223 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:51.978804 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 90558.8us
I20260709 12:12:51.979243 13094 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:51.979912 13094 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:51.980228 13139 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:51.980683 13139 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:12:51.981168 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 93042.1us
I20260709 12:12:51.888715 12643 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.868724 12908 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.981529 13139 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.873103 11931 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Advancing to term 4
I20260709 12:12:51.982787 13139 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:51.870486 13055 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:51.984046 13055 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:51.984190 13152 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:51.984614 13152 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.984931 13152 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:51.864403 13099 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.889920 13046 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:51.886364 13134 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:51.894294 13168 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:12:51.894713 13060 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:51.896911 13161 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:51.897574 13153 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:51.856654 13130 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:12:51.907359 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:51.909188 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 349) took 1071 ms. Trace:
I20260709 12:12:51.910219 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:51.910997 12959 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:51.912894 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:51.999904 11254 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:12:51.913951 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:51.914942 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:52.000705 11174 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:51.916657 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.002167 11257 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:51.921658 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.002815 11254 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:52.002977 11256 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
W20260709 12:12:51.922739 11900 connection.cc:256] RPC call timeout handler was delayed by -1.3672841729999163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 12:12:52.004168 11900 connection.cc:256] RPC call timeout handler was delayed by -1.1425901249997423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
I20260709 12:12:51.899101 13132 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:51.921792 13163 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.928613 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:51.936213 13125 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.935058 13158 raft_consensus.cc:1081] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:12:51.938283 13166 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:51.941128 13115 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:51.934273 13146 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.948154 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:51.580871 13137 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:51.839721 13156 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:51.960340 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:51.955152 11328 rpcz_store.cc:276] 0709 12:12:50.626578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.206248 (+579670us) service_pool.cc:224] Handling call
0709 12:12:51.206536 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:51.557828 (+351292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.846567 (+288739us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54002f67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:51.852640 (+  6073us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549514624}
W20260709 12:12:51.963079 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:51.963563 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.964968 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 475) took 1068 ms. Trace:
I20260709 12:12:52.036692 11933 rpcz_store.cc:276] 0709 12:12:50.896443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.899165 (+  2722us) service_pool.cc:224] Handling call
0709 12:12:51.963850 (+1064685us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54004312d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:51.964904 (+  1054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2254635904}
I20260709 12:12:51.958161 13111 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:51.970382 11327 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:51.969993 11180 rpcz_store.cc:276] 0709 12:12:50.836707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.235395 (+398688us) service_pool.cc:224] Handling call
0709 12:12:51.275212 (+ 39817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:51.358132 (+ 82920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:51.358185 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:51.358855 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:12:51.495599 (+136744us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:51.495643 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.495712 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:51.874934 (+379222us) raft_consensus.cc:1685] finished
0709 12:12:51.874956 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:51.875080 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":191065,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":63,"thread_start_us":134995,"threads_started":2,"wal-append.queue_time_us":201542}
I20260709 12:12:51.970109 11182 rpcz_store.cc:276] 0709 12:12:50.860707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.379444 (+518737us) service_pool.cc:224] Handling call
0709 12:12:51.379894 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:51.380284 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:51.380325 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:51.380954 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:12:51.860489 (+479535us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:12:51.860517 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.860951 (+   434us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:51.874127 (+ 13176us) raft_consensus.cc:1685] finished
0709 12:12:51.874150 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:51.874308 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87182,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":64,"thread_start_us":424718,"threads_started":1,"wal-append.queue_time_us":463247}
I20260709 12:12:51.965291 13131 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:51.979384 11203 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd540 after lost signal to thread 11355
I20260709 12:12:51.827798 13127 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:51.980492 12972 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:51.890872 13106 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:52.030534 13166 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:51.990212 11251 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:51.980093 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:51.995971 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:52.023741 13147 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:12:52.000785 11173 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
W20260709 12:12:52.044974 11203 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd520 after lost signal to thread 11356
I20260709 12:12:52.005412 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.008531 13060 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:12:52.009207 13186 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:12:51.984979 13055 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:52.069314 11326 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:52.063944 13060 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:52.072525 13060 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:52.074178 13060 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.074920 13102 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:52.077224 13019 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:51.998739 13167 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:12:52.009922 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 216) took 1307 ms. Trace:
I20260709 12:12:51.974721 13181 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:51.981014 13094 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:51.998734 13161 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:52.023653 11249 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:51.997659 13155 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:51.991624 13099 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:12:51.965667 11927 rpcz_store.cc:276] 0709 12:12:50.836429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.839929 (+  3500us) service_pool.cc:224] Handling call
0709 12:12:51.848661 (+1008732us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54005afd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:51.858036 (+  9375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057210624}
I20260709 12:12:52.035216 11181 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:52.037611 11930 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 12:12:52.035800 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 75525.9us
I20260709 12:12:52.038017 13153 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:12:52.042668 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 94608.4us
I20260709 12:12:52.032310 12959 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:12:52.044921 13160 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:52.097280 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:52.028077 13163 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.093070 11931 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:12:52.098191 11931 pending_rounds.cc:85] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 2
I20260709 12:12:52.098481 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:52.039719 13146 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.103400 13146 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:52.106142 13131 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:52.088349 13127 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:52.030831 13132 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:51.963277 12960 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:52.108529 13158 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:52.108920 12960 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.109031 13158 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:52.109453 13153 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:52.110162 13055 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:52.110128 13158 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.110574 13153 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:52.111708 13158 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.060004 11174 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:52.035379 13156 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:52.112973 13156 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:51.916069 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.108525 13127 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:52.114439 13181 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.114856 13127 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.115762 13181 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:52.064628 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 84614.1us
I20260709 12:12:52.117411 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:52.002000 11926 rpcz_store.cc:276] 0709 12:12:50.837548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.841023 (+  3475us) service_pool.cc:224] Handling call
0709 12:12:51.908833 (+1067810us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b540034f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:51.909091 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1602425088}
W20260709 12:12:52.066129 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60828.9us
I20260709 12:12:52.123986 13161 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:51.954013 11178 rpcz_store.cc:276] 0709 12:12:50.896315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.471168 (+574853us) service_pool.cc:224] Handling call
0709 12:12:51.471462 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:51.539605 (+ 68143us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:12:51.804454 (+264849us) log.cc:844] Serialized 14 byte log entry
0709 12:12:51.876487 (+ 72033us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:51.880201 (+  3714us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:12:51.902500 (+ 22299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.947933 (+ 45433us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b54003485d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:51.947981 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:51.948104 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39151,"prepare.queue_time_us":259358,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":10676,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":1854,"spinlock_wait_cycles":86549248,"thread_start_us":201889,"threads_started":2,"wal-append.queue_time_us":96539}
I20260709 12:12:52.067909 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1115) took 1442 ms. Trace:
I20260709 12:12:52.067351 13137 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:52.068985 13046 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:52.082398 11177 rpcz_store.cc:276] 0709 12:12:50.702615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.219854 (+517239us) service_pool.cc:224] Handling call
0709 12:12:51.220166 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:51.295392 (+ 75226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:51.295429 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:51.317804 (+ 22375us) log.cc:844] Serialized 26 byte log entry
0709 12:12:51.540278 (+222474us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:51.552967 (+ 12689us) log.cc:844] Serialized 14 byte log entry
0709 12:12:51.582893 (+ 29926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.582970 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:51.868223 (+285253us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b1800137080. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:12:52.009539 (+141316us) raft_consensus.cc:1685] finished
0709 12:12:52.009721 (+   182us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:52.009854 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46965,"prepare.queue_time_us":56735,"prepare.run_cpu_time_us":19579,"prepare.run_wall_time_us":148815,"raft.queue_time_us":257777,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":19869,"spinlock_wait_cycles":65192320,"thread_start_us":185339,"threads_started":2,"wal-append.queue_time_us":245861}
I20260709 12:12:52.026734 13149 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:12:52.091856 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:52.128000 13137 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:52.128994 13137 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.130478 13141 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:52.132663 13176 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:52.129081 13156 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.132802 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:52.133307 13176 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:52.133672 13019 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:52.093418 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.134227 13019 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:52.134382 13176 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.135067 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:52.135202 13019 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.095019 13115 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:52.095150 13150 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:52.136401 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:52.136713 13146 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:52.138943 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:51.997116 11324 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
W20260709 12:12:52.061810 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51629.3us
I20260709 12:12:52.123775 12923 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:52.126209 11327 rpcz_store.cc:276] 0709 12:12:50.625912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.858011 (+232099us) service_pool.cc:224] Handling call
0709 12:12:52.067667 (+1209656us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002855d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:52.067828 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2484032256}
I20260709 12:12:52.085943 12908 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:12:52.137428 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:52.129257 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.115989 13127 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:52.138738 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 231499us
I20260709 12:12:52.145834 13173 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:52.146219 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:52.090858 13166 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.140136 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:52.098483 11931 pending_rounds.cc:107] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:12:52.147084 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:52.140365 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:52.136888 13185 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:12:52.157745 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:52.144517 13046 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.150602 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:52.158334 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:52.136811 13150 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:52.183307 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:52.185413 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.146907 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.177521 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:52.189891 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:52.189563 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.191419 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:52.191432 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:52.144553 11203 debug-util.cc:398] Leaking SignalData structure 0x7b0800394920 after lost signal to thread 11959
W20260709 12:12:52.180218 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:52.198625 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52138.9us
I20260709 12:12:52.180816 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:52.185497 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:52.175071 11330 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:52.199821 11257 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:52.149725 13160 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:52.195364 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.205862 11252 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:52.144325 13146 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:52.171531 13134 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.174697 13102 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:52.193286 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:52.204854 11254 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:52.171286 12972 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.207203 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:52.207257 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.215487 11251 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
I20260709 12:12:52.172389 13162 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:52.210363 13160 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.189585 13136 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:52.209120 13159 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:52.203958 11203 debug-util.cc:398] Leaking SignalData structure 0x7b0800394a40 after lost signal to thread 11960
I20260709 12:12:52.178311 11931 raft_consensus.cc:2889] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:52.200665 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:52.214668 11329 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:12:52.218290 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:52.209393 13084 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:52.177804 12981 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.181155 13078 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:52.182487 13060 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:52.222453 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:52.277772 11331 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:12:52.245211 13190 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:12:52.223496 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:52.181200 12908 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:52.189477 11248 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:52.226620 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.211743 13146 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.229360 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:52.109458 12960 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:52.208917 13166 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:52.206940 13133 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:52.203640 13115 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 12:12:52.242224 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.257985 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:52.258738 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:52.222131 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:52.223245 13162 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:52.269958 11330 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:52.272360 11329 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:52.175300 12923 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:52.231003 13136 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:52.250552 11203 debug-util.cc:398] Leaking SignalData structure 0x7b0800394a00 after lost signal to thread 12586
I20260709 12:12:52.207919 12975 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:12:52.234385 13160 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:52.287892 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 58244us
W20260709 12:12:52.408944 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.226027 13134 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.226784 12972 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.270802 13078 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:52.340831 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.228433 13010 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:12:52.281497 13060 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:52.231379 12967 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
W20260709 12:12:52.385555 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 167341us
W20260709 12:12:52.395342 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 171933us
I20260709 12:12:52.436801 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.310890 12908 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:52.398221 13125 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:52.402513 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:52.277222 12981 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.409439 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:52.311435 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 127909us
I20260709 12:12:52.446280 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.446982 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:52.413136 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:52.416402 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.405294 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:52.386405 13007 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:52.312767 13128 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:12:52.390282 11929 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.396607 12663 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:12:52.421458 11203 debug-util.cc:398] Leaking SignalData structure 0x7b08003dc6e0 after lost signal to thread 12669
I20260709 12:12:52.406190 13193 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:52.408385 13136 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:52.440232 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.428390 13150 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:52.434221 11930 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:52.447788 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:52.496934 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49218.4us
I20260709 12:12:52.497725 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:52.498929 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.451469 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.500452 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.427982 13078 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.454591 13159 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:52.458835 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.441606 11322 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:52.434811 13060 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.445912 11248 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:12:52.456416 13125 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.459611 13147 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.502416 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 243) took 1849 ms. Trace:
I20260709 12:12:52.460084 13007 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:52.453810 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:52.480005 13193 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:52.467468 12663 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.523137 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.525203 11324 rpcz_store.cc:276] 0709 12:12:50.653231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.219523 (+566292us) service_pool.cc:224] Handling call
0709 12:12:51.219825 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.139871 (+920046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.502046 (+362175us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b5400429ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:52.502331 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787040640}
I20260709 12:12:52.529309 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:52.551873 11173 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:12:52.531004 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:52.532403 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:52.533702 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:52.536774 11248 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:52.480451 13133 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:52.472409 13163 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:52.515128 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63806.9us
W20260709 12:12:52.570695 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47613.1us
I20260709 12:12:52.580942 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.555279 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:52.559126 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.468910 13139 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:12:52.563745 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:52.584619 11174 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:52.537976 13167 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:52.564699 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:52.450263 13162 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.523952 13164 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:52.564565 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:52.583171 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.597381 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.516188 12923 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.603812 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:52.567971 11248 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:52.612062 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.562623 12663 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:52.636180 11326 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:12:52.634838 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.623970 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.584658 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:52.642037 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.573279 13209 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:52.650302 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69471.2us
I20260709 12:12:52.588392 13125 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:52.591284 13046 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:52.590283 13167 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:52.668115 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:52.631494 13159 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:52.669988 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:52.639879 13137 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:52.650249 13164 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:52.601670 13115 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:12:52.578193 12908 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.654582 12981 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:52.671813 13167 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.667516 13046 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:52.723879 13144 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:52.658062 11178 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:52.670351 13191 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:52.670483 13125 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:52.713302 13142 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:52.741433 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:52.724473 13165 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:12:52.680567 13150 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:12:52.744290 13131 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.000s
I20260709 12:12:52.651979 13193 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:52.750097 13201 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.287s	user 0.000s	sys 0.003s
I20260709 12:12:52.766950 11176 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.753515 13163 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:52.735143 13134 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:52.744174 13046 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.711258 13133 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:52.764766 13152 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.005s
I20260709 12:12:52.741122 13209 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.744164 13194 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:12:52.760807 13192 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.255s	user 0.000s	sys 0.002s
W20260709 12:12:52.781731 13111 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.728s	user 0.011s	sys 0.002s
W20260709 12:12:52.788935 13183 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:12:52.789306 13196 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.287s	user 0.000s	sys 0.002s
W20260709 12:12:52.789945 13202 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:12:52.800691 13127 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.629s	user 0.004s	sys 0.007s
I20260709 12:12:52.801164 13152 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:52.822517 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.769068 13215 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.218s
I20260709 12:12:52.829658 11176 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:52.805135 13193 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:52.803889 13209 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:52.812683 13203 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.251s	user 0.002s	sys 0.000s
I20260709 12:12:52.804410 13194 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:12:52.843098 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 201277us
I20260709 12:12:52.807535 13146 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.807606 11248 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:52.850478 13204 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:12:52.850876 13208 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:52.851859 13195 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.305s	user 0.000s	sys 0.002s
W20260709 12:12:52.854343 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 295330us
I20260709 12:12:52.854708 11179 raft_consensus.cc:2368] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5.
I20260709 12:12:52.859512 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:52.884047 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.886873 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:52.853335 13209 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.813145 13111 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.892320 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:52.891276 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 221526us
I20260709 12:12:52.897384 13228 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:52.907904 11255 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:52.858719 13194 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:52.911912 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:52.837168 13150 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.912751 11251 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3.
I20260709 12:12:52.854769 11929 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:52.917861 13171 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:12:52.921775 13200 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 12:12:52.918253 11127 auto_rebalancer.cc:893] Removing replace marker failed: Replica 016411114c6d42039bd04025378dfc98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:12:52.935367 11127 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 86c5246d860d4d559ea665671cc46b5c already in progress
I20260709 12:12:52.891228 13131 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
W20260709 12:12:52.937250 12960 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.006s
W20260709 12:12:52.935949 11127 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 86c5246d860d4d559ea665671cc46b5c already in progress
I20260709 12:12:52.915781 13147 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.944455 11900 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3.
I20260709 12:12:52.944957 11900 leader_election.cc:403] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:52.905009 13125 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:12:52.914614 13060 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:52.945930 13166 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.004s
I20260709 12:12:52.950889 13211 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.941176 12960 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:52.939400 13171 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:52.955938 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:52.934358 12981 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:12:52.959923 11181 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:12:52.952991 13225 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.976562 11325 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:52.975857 12960 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:52.963304 13223 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 12:12:52.991724 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.950320 13147 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:52.976130 13146 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:52.941066 13219 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:12:52.955179 13222 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.029893 11248 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:52.991196 13127 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:53.030848 13060 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.030678 13127 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.032078 13111 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:53.033349 13127 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.035315 13217 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:53.037580 13147 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:52.965209 13132 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.975852 12957 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:53.036933 13167 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:53.043586 13152 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:52.962589 13211 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.045868 13132 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.052371 13211 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:53.057965 13232 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:12:53.063900 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:53.059666 13160 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.004s
I20260709 12:12:53.065490 13132 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.025154 12663 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:53.066535 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 122207us
I20260709 12:12:52.992970 13225 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.055130 13171 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:53.270248 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 1.04364e+06us
I20260709 12:12:53.272569 11251 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
W20260709 12:12:53.133234 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.275437 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:53.276181 11256 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:53.277464 11328 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
W20260709 12:12:53.254376 13231 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
W20260709 12:12:53.158759 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:53.279597 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.280361 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.281397 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:53.281646 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:53.282043 11256 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:53.330092 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:53.286347 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:53.337740 11257 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:53.339257 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:53.368805 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 82550.2us
W20260709 12:12:53.280843 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:53.391819 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 185) took 1206 ms (client timeout 1561 ms). Trace:
I20260709 12:12:53.396668 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1737) took 1241 ms. Trace:
I20260709 12:12:53.310035 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:53.404088 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:53.404202 11325 rpcz_store.cc:276] 0709 12:12:52.155535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.158048 (+  2513us) service_pool.cc:224] Handling call
0709 12:12:53.385497 (+1227449us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54000e6354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:53.396557 (+ 11060us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826505472}
W20260709 12:12:53.256224 13078 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.004s
W20260709 12:12:53.256985 13136 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.743s	user 0.009s	sys 0.004s
I20260709 12:12:53.409520 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.418087 11325 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:53.419418 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.336246 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.285151 11329 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:53.421347 11327 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:12:53.099788 13158 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.988s	user 0.016s	sys 0.000s
I20260709 12:12:53.181011 13153 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:53.421897 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:53.158161 13055 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:53.133719 13102 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:53.155702 12975 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:53.401139 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:53.157863 13017 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:53.269378 13228 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:53.156066 13181 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:53.430779 13055 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.424667 13078 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.431314 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:53.080188 12663 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.427099 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:53.280002 13167 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:53.413712 13209 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.001s
I20260709 12:12:53.389912 13173 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:53.399878 13161 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.180646 13156 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.398021 13141 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:53.415243 13002 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:53.274391 13046 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.424872 13136 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.377481 13194 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.070745 13160 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.430186 13153 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:53.437179 12975 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:53.440721 13017 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:53.442147 13225 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.459619 13207 raft_consensus.cc:1064] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:53.442289 13147 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:12:53.458792 13173 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:53.462143 13141 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.467162 13130 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:53.473575 13130 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:53.473239 13102 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.473685 13173 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.468421 13211 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:53.475596 13141 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:53.476485 13211 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.479370 13173 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.463335 13209 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.487111 13194 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:53.487824 13209 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:53.488371 13222 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:53.494024 13207 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.485531 13211 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:53.496132 13160 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.502964 13211 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.505383 13211 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:53.538077 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:53.539567 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.554342 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:53.557422 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:53.559952 11173 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:53.560643 11174 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:12:53.562355 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:53.562808 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:53.564190 11173 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:53.564670 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:53.565591 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.566150 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.567075 11173 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:53.567589 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.568250 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.569078 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.572614 11176 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:53.572750 11181 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:12:53.071545 11178 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:53.574062 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.574999 11181 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.577919 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.578581 11177 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:53.579464 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:53.579563 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:53.580415 11177 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:53.580415 11176 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:53.581487 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:53.584158 11249 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:12:53.402004 11254 rpcz_store.cc:269] 0709 12:12:52.184963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.185104 (+   141us) service_pool.cc:224] Handling call
0709 12:12:53.391585 (+1206481us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b540010b654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:53.391743 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42662656}
I20260709 12:12:53.424247 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.589309 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:53.591259 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:53.592993 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.592941 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:53.131793 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 258) took 1025 ms. Trace:
I20260709 12:12:53.594573 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.595708 11324 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:53.594079 11323 rpcz_store.cc:276] 0709 12:12:52.106676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.106922 (+   246us) service_pool.cc:224] Handling call
0709 12:12:52.107287 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.107514 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:52.205912 (+ 98398us) log.cc:844] Serialized 14 byte log entry
0709 12:12:53.130977 (+925065us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b1800306e80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:12:53.131215 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:53.131304 (+    89us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:53.131382 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:53.131493 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:53.131700 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13279,"prepare.queue_time_us":120977,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":915826,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":5550464,"thread_start_us":78160,"threads_started":1}
I20260709 12:12:53.596841 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.427285 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:53.598397 11253 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:53.598563 11325 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:53.416270 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:53.602501 11323 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:53.376127 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.606305 11250 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:53.607138 11322 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:53.607577 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.608371 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:53.609263 11325 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:53.432431 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:53.425607 11329 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.622711 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:53.623827 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:53.626423 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:53.627385 11250 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.631011 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:53.632194 11251 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:53.609321 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:53.610848 11254 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:53.610963 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1746) took 1439 ms. Trace:
W20260709 12:12:53.603302 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 186955us
I20260709 12:12:53.617864 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:53.593508 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.582319 11176 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:53.455760 11934 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:12:53.442463 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:53.632153 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:53.634822 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70584.3us
I20260709 12:12:53.666601 11929 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:53.668123 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:53.638005 11325 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:53.668608 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:53.670974 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:53.680868 13150 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:53.682231 13132 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:53.465083 13002 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.672873 12958 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:53.614413 11174 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:53.673767 12967 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.674620 12957 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3.
I20260709 12:12:53.675405 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:53.674889 13017 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.677546 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:53.677273 12975 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.678346 13046 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:53.678092 13094 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:53.679083 13060 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.679283 13127 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.680042 13078 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:53.664299 11928 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.637073 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:53.681371 11929 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.682093 13010 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:53.620810 11181 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:53.653014 11250 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:53.633803 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:53.669704 11331 rpcz_store.cc:276] 0709 12:12:52.171524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.282171 (+110647us) service_pool.cc:224] Handling call
0709 12:12:52.310882 (+ 28711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:53.134315 (+823433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:53.134366 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:53.275244 (+140878us) log.cc:844] Serialized 26 byte log entry
0709 12:12:53.610389 (+335145us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:53.610495 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:53.610618 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:53.610657 (+    39us) raft_consensus.cc:1685] finished
0709 12:12:53.610667 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:53.610795 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":822983,"raft.queue_time_us":823805,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"thread_start_us":334539,"threads_started":1,"wal-append.queue_time_us":215292}
I20260709 12:12:53.633220 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:53.701532 11177 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:53.703648 11173 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
W20260709 12:12:53.471839 13084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.020s	sys 0.015s
I20260709 12:12:53.672029 11323 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:53.711385 11329 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:53.682183 13055 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:53.789698 11181 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:53.784205 11175 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:53.712016 13158 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:53.793738 11325 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 2. Current term is 3. Ops: []
W20260709 12:12:53.792896 13084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.390s	user 0.020s	sys 0.016s
I20260709 12:12:53.795768 13193 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:53.796124 12873 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:53.796280 13193 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:53.790500 13146 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:53.797240 13146 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.797773 13173 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.798573 13173 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.799865 13146 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:53.800282 13222 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.786130 13017 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:53.801007 13222 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.804088 13222 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:53.804708 13222 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.784627 12958 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.806059 12967 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:53.806967 13084 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:53.807857 13156 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:53.808425 13156 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:53.784349 13010 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.808482 13084 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.809497 13176 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:53.793131 11327 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.808755 12967 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:53.808040 12958 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:53.815304 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46618.1us
I20260709 12:12:53.801332 13173 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:53.817535 13225 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:53.837965 13060 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:53.838516 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 161250us
I20260709 12:12:53.838999 13225 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.839838 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:53.840998 13060 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.842427 12975 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.843436 13136 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:53.844440 13136 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.841019 13046 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:53.845659 13046 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:53.819608 13010 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:53.823642 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46354us
I20260709 12:12:53.812906 13176 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:53.826309 13002 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:53.855840 13046 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.704408 13153 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:53.859189 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.827015 13166 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:53.477831 13130 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.828231 13209 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:53.814994 13156 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.830587 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:53.810695 13193 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:53.833165 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 196389us
I20260709 12:12:53.860960 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:53.837002 11249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 181) took 1666 ms (client timeout 2128 ms). Trace:
I20260709 12:12:53.853403 11174 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:12:53.805344 13222 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:53.835772 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 221) took 1930 ms (client timeout 2059 ms). Trace:
I20260709 12:12:53.843019 13225 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:53.870114 13152 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.870589 13130 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:53.871503 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:53.870857 13125 consensus_queue.cc:1175] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 4 status { last_received { term: 4 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:12:53.873724 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:53.873860 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.875133 11928 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEARNER]: Advancing to term 3
I20260709 12:12:53.877869 13150 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:53.878021 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:53.878795 13226 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.879575 11331 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:53.880802 12923 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.870628 13193 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:53.865685 11249 rpcz_store.cc:269] 0709 12:12:52.170564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.178050 (+  7486us) service_pool.cc:224] Handling call
0709 12:12:53.836719 (+1658669us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:53.836921 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3146481920}
I20260709 12:12:53.851388 13127 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:53.882441 13209 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:53.859926 13176 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:53.894052 11932 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
W20260709 12:12:54.144011 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 468878us
W20260709 12:12:53.868912 11179 rpcz_store.cc:269] 0709 12:12:51.911536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.912543 (+  1007us) service_pool.cc:224] Handling call
0709 12:12:53.835199 (+1922656us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54002b7d54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:53.835529 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964547328}
W20260709 12:12:54.074376 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.142467 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 311982us
I20260709 12:12:53.908218 11256 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:53.936890 11325 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
W20260709 12:12:54.002177 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.006153 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.897130 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:53.910009 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:53.898764 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:53.931705 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:53.930096 13171 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:53.896392 13166 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:12:54.134835 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 595217us
I20260709 12:12:54.118700 11926 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.894991 13156 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:54.170867 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:54.073478 13136 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:54.174057 13225 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:54.174686 13225 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.175248 13225 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:54.175359 13209 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:54.176333 11254 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:54.177824 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 279252us
W20260709 12:12:54.178406 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.179122 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.179826 13153 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:54.179939 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:54.181877 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.940053 13222 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:54.184825 11256 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:54.128106 13159 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:53.919348 13162 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:54.047473 13099 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.905148 11929 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
W20260709 12:12:54.196727 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.197499 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.195953 13060 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:54.199595 11256 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:12:54.199752 13127 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:54.202661 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:54.204636 13209 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:54.205087 13209 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:54.132877 13141 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:54.205487 13209 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:54.206315 11933 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:12:54.206202 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:54.207074 11927 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:54.208829 11933 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:54.211208 13141 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:54.212014 13150 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:54.212379 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.219129 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 347821us
I20260709 12:12:53.921253 13163 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:54.224279 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:54.142957 13226 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:54.221010 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:54.222465 13136 raft_consensus.cc:2764] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:12:54.227705 12972 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:54.230464 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:54.230715 11254 raft_consensus.cc:2410] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:54.187749 13159 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:54.218696 11223 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.618s (SENT)
I20260709 12:12:54.235255 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:54.241426 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:54.242856 13159 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:54.257817 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:54.259060 11252 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:54.260061 13102 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.260488 11252 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:54.260931 13102 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:54.261966 11331 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:54.263597 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:54.263849 13102 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:54.264014 12873 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:54.264582 12873 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:54.264945 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:54.267354 11175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1734) took 1734 ms (client timeout 2240 ms). Trace:
W20260709 12:12:54.267822 11175 rpcz_store.cc:269] 0709 12:12:52.533233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.533377 (+   144us) service_pool.cc:224] Handling call
0709 12:12:54.267099 (+1733722us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000c0b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:54.267261 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2339891968}
I20260709 12:12:54.240428 13099 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:54.279803 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1740) took 1722 ms. Trace:
I20260709 12:12:54.282397 11178 rpcz_store.cc:276] 0709 12:12:52.557703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.770298 (+212595us) service_pool.cc:224] Handling call
0709 12:12:52.771581 (+  1283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:53.574349 (+802768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.279159 (+704810us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400364f54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:54.279531 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676964096}
I20260709 12:12:54.285537 13099 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:53.901368 13007 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:54.286938 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:54.290314 13007 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:54.291401 13252 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.484s	user 0.001s	sys 0.000s
I20260709 12:12:54.293344 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:54.296324 11253 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:54.280923 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:54.185320 12923 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:54.297883 11254 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:12:54.237411 13163 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:54.242120 13166 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:54.318225 11252 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:54.319444 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:54.320865 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 138466us
I20260709 12:12:54.328958 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1132) took 1035 ms. Trace:
I20260709 12:12:54.329445 11327 rpcz_store.cc:276] 0709 12:12:53.293411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.625251 (+331840us) service_pool.cc:224] Handling call
0709 12:12:53.625767 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:53.810640 (+184873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.328608 (+517968us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54001c48d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:54.328846 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381052672}
I20260709 12:12:54.332983 11257 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:54.333868 11257 raft_consensus.cc:606] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Received request to transfer leadership
I20260709 12:12:54.326969 11322 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:54.335443 11249 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:12:54.334273 11331 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:12:54.337292 13241 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.938s	user 0.002s	sys 0.003s
I20260709 12:12:54.338466 11329 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:12:54.342670 11326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1733) took 2194 ms (client timeout 1618 ms). Trace:
W20260709 12:12:54.343185 11326 rpcz_store.cc:269] 0709 12:12:52.148358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.148552 (+   194us) service_pool.cc:224] Handling call
0709 12:12:54.342224 (+2193672us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54002f71d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:54.342447 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024981248}
I20260709 12:12:54.345788 13153 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.346238 11328 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 12:12:54.346673 11328 pending_rounds.cc:85] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:12:54.346992 11328 pending_rounds.cc:107] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 12:12:54.346518 13153 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:54.347321 11328 raft_consensus.cc:2889] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:54.349205 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1126) took 1067 ms. Trace:
I20260709 12:12:54.349292 13153 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:54.345549 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.350390 11256 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: []
W20260709 12:12:54.351294 11934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 355) took 2131 ms (client timeout 1932 ms). Trace:
I20260709 12:12:54.351814 13150 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:54.352440 12923 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:54.352535 13141 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.352936 13099 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:54.353621 13142 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:54.388159 13141 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:54.353914 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 165878us
I20260709 12:12:54.298504 12873 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:54.391621 11324 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:54.355326 11251 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:54.312211 13166 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.359485 12981 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:54.349592 11328 rpcz_store.cc:276] 0709 12:12:53.281824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.281926 (+   102us) service_pool.cc:224] Handling call
0709 12:12:54.345617 (+1063691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.347967 (+  2350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.348990 (+  1023us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002f71d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:54.349147 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445433216}
W20260709 12:12:54.356683 11934 rpcz_store.cc:269] 0709 12:12:52.219996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.220151 (+   155us) service_pool.cc:224] Handling call
0709 12:12:52.453594 (+233443us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:54.350910 (+1897316us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b540053a7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:54.351077 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2124682240}
W20260709 12:12:54.387063 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 490025us
W20260709 12:12:54.387660 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.695966 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.621973 11182 raft_consensus.cc:2410] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:12:54.942812 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.470279 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 114405us
W20260709 12:12:54.584527 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 342618us
W20260709 12:12:54.604975 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.548827 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.802934 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.616875 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.949313 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.470647 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:54.626271 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.583958 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1748) took 2411 ms. Trace:
W20260709 12:12:54.952435 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.469998 11325 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
W20260709 12:12:54.953703 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.952579 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:54.801086 11250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.013s	sys 0.018s
W20260709 12:12:54.956163 11250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.013s	sys 0.028s
I20260709 12:12:54.516330 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:54.471133 11254 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
W20260709 12:12:54.962592 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.687487 13193 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.000s
I20260709 12:12:54.403992 13099 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:12:54.943575 13225 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.768s	user 0.004s	sys 0.011s
W20260709 12:12:54.470026 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 2.059s (SENT) [suppressed 19 similar messages]
W20260709 12:12:54.959239 13130 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.800s	user 0.003s	sys 0.010s
W20260709 12:12:54.967347 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 785253us
I20260709 12:12:54.490741 12923 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:54.426774 13166 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:54.515252 13246 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.963424 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1576) took 1047 ms. Trace:
W20260709 12:12:54.957296 13176 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.780s	user 0.012s	sys 0.003s
I20260709 12:12:54.958721 13153 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.922799 13226 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:54.985502 11253 rpcz_store.cc:276] 0709 12:12:53.906250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.931004 (+ 24754us) service_pool.cc:224] Handling call
0709 12:12:54.947620 (+1016616us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540010a754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:54.953236 (+  5616us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311223680}
I20260709 12:12:54.953352 11330 rpcz_store.cc:276] 0709 12:12:52.172137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.406145 (+234008us) service_pool.cc:224] Handling call
0709 12:12:52.406504 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:53.151761 (+745257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:53.151810 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:53.152490 (+   680us) log.cc:844] Serialized 26 byte log entry
0709 12:12:53.274719 (+122229us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:53.274765 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:53.274831 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:54.273052 (+998221us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:12:54.563765 (+290713us) raft_consensus.cc:1685] finished
0709 12:12:54.563796 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:54.583736 (+ 19940us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":744730,"raft.queue_time_us":748549,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"spinlock_wait_cycles":5430144,"thread_start_us":117388,"threads_started":1,"wal-append.queue_time_us":241683}
I20260709 12:12:54.973820 13099 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:54.764295 13159 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:54.974076 13162 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:54.997787 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 529598us
I20260709 12:12:55.012620 13226 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:55.013558 13226 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:55.016516 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.017823 13226 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.018682 13226 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.019387 13226 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.023458 13226 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:54.979523 13254 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.027015 11932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 360) took 1576 ms (client timeout 2016 ms). Trace:
I20260709 12:12:54.979277 13142 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.032226 11932 rpcz_store.cc:269] 0709 12:12:53.426660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.426825 (+   165us) service_pool.cc:224] Handling call
0709 12:12:54.980338 (+1553513us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54003b5454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:55.002828 (+ 22490us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766475008}
I20260709 12:12:54.985697 13193 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:55.025853 11927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.018s	sys 0.058s
I20260709 12:12:54.982488 13046 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.996529 12873 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:55.036755 13209 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.824s	user 0.014s	sys 0.000s
I20260709 12:12:54.979947 13225 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:55.054713 11251 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
I20260709 12:12:55.055425 11251 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:12:55.056269 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1587) took 1049 ms. Trace:
I20260709 12:12:55.009868 13153 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.056818 11251 rpcz_store.cc:276] 0709 12:12:54.006773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.185582 (+178809us) service_pool.cc:224] Handling call
0709 12:12:55.056155 (+870573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.055670 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:55.044626 11927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.018s	sys 0.058s
I20260709 12:12:55.042927 13141 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:55.010285 13176 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.066025 13219 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.009709 13246 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:55.062172 13133 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.007s	sys 0.008s
W20260709 12:12:55.072182 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.070076 11931 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.000s	sys 0.016s
W20260709 12:12:55.076134 11931 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.000s	sys 0.017s
I20260709 12:12:55.039573 13142 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.072357 12923 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:55.054358 13209 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.043095 13254 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:55.073938 13133 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.008s	sys 0.008s
I20260709 12:12:55.047125 13046 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:55.078174 13156 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.889s	user 0.012s	sys 0.001s
W20260709 12:12:55.093262 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54222.1us
W20260709 12:12:55.087456 11222 connection.cc:256] RPC call timeout handler was delayed by -1.4372662829998717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
W20260709 12:12:55.095517 11222 connection.cc:256] RPC call timeout handler was delayed by -1.288766282999859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 12:12:55.096400 11222 connection.cc:256] RPC call timeout handler was delayed by -1.1222662829998171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 12:12:55.087168 13007 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.796s	user 0.010s	sys 0.005s
I20260709 12:12:55.079748 12923 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.071200 13219 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.015407 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:55.086431 13226 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:12:55.101967 12923 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:55.104081 13156 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.122824 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:55.110462 13133 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:55.137565 12873 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:55.137681 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.141903 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.121479 13007 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:12:55.125311 13156 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.145920 11331 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:12:55.143754 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.132400 13162 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
W20260709 12:12:55.149729 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.136652 13159 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.150823 11325 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:55.149698 11330 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:55.128304 13115 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:12:55.138866 13142 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.146131 13130 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.149446 13163 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.004s
W20260709 12:12:55.143469 13167 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.052s	sys 0.060s
I20260709 12:12:55.162042 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1578) took 1253 ms. Trace:
I20260709 12:12:55.144065 13153 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.162492 11250 rpcz_store.cc:276] 0709 12:12:53.907272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.907891 (+   619us) service_pool.cc:224] Handling call
0709 12:12:53.908429 (+   538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.143641 (+235212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.143692 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.170020 (+ 26328us) log.cc:844] Serialized 26 byte log entry
0709 12:12:54.956797 (+786777us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:54.956847 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.956926 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.160823 (+203897us) raft_consensus.cc:1685] finished
0709 12:12:55.160842 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.160972 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4455,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":159,"thread_start_us":683094,"threads_started":1,"wal-append.queue_time_us":891784}
I20260709 12:12:55.133980 13133 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.163995 13099 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:55.163913 13167 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.053s	sys 0.061s
I20260709 12:12:55.172477 13130 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.173135 13144 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:55.154963 13007 consensus_queue.cc:1243] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:12:55.168269 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 432) took 1541 ms. Trace:
I20260709 12:12:55.173909 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 267) took 1575 ms. Trace:
I20260709 12:12:55.174274 13046 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.177443 13144 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:55.165875 13163 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.215332 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1757) took 1609 ms. Trace:
I20260709 12:12:55.235858 11324 rpcz_store.cc:276] 0709 12:12:53.605872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.295454 (+689582us) service_pool.cc:224] Handling call
0709 12:12:54.314322 (+ 18868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.392150 (+ 77828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.214941 (+822791us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b54002f76d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:55.215242 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177133952}
W20260709 12:12:55.201468 11323 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.001s	sys 0.015s
I20260709 12:12:55.209563 13226 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.251684 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 231210us
W20260709 12:12:55.249213 11323 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.002s	sys 0.015s
W20260709 12:12:55.258267 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.223388 13246 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:55.278538 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 808487us
I20260709 12:12:55.204071 11322 rpcz_store.cc:276] 0709 12:12:53.598455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.609464 (+ 11009us) service_pool.cc:224] Handling call
0709 12:12:55.173696 (+1564232us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54001c4dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:55.173820 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486934912}
I20260709 12:12:55.282706 11250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:55.283303 11250 raft_consensus.cc:606] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Received request to transfer leadership
W20260709 12:12:55.287412 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 336628us
W20260709 12:12:55.277129 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1732) took 2742 ms (client timeout 2629 ms). Trace:
W20260709 12:12:55.289620 11182 rpcz_store.cc:269] 0709 12:12:52.530607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.530722 (+   115us) service_pool.cc:224] Handling call
0709 12:12:55.261782 (+2731060us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54002571d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:55.272966 (+ 11184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4674433792}
I20260709 12:12:55.299198 13219 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.293166 13153 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.306474 13153 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.309094 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:55.273047 13226 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:55.301736 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:55.293192 13156 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.201592 11927 rpcz_store.cc:276] 0709 12:12:53.627132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.208025 (+580893us) service_pool.cc:224] Handling call
0709 12:12:54.336733 (+128708us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:54.336960 (+   227us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.362213 (+ 25253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.362265 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.363015 (+   750us) log.cc:844] Serialized 26 byte log entry
0709 12:12:55.066426 (+703411us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:55.071387 (+  4961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.075819 (+  4432us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.149661 (+ 73842us) raft_consensus.cc:1685] finished
0709 12:12:55.149686 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.166403 (+ 16717us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24491,"raft.queue_time_us":24703,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":118,"spinlock_wait_cycles":11294976,"thread_start_us":697580,"threads_started":1,"wal-append.queue_time_us":760224}
I20260709 12:12:55.314744 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1583) took 1387 ms. Trace:
I20260709 12:12:55.313689 13271 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:55.299520 13163 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.322914 13226 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.325520 13156 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:55.327646 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:55.318282 11256 rpcz_store.cc:276] 0709 12:12:53.967918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.200458 (+232540us) service_pool.cc:224] Handling call
0709 12:12:54.200775 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.362343 (+161568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.314491 (+952148us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54001ce8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:55.314678 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343529344}
I20260709 12:12:55.311996 13099 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:55.322957 11928 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.160s	user 0.007s	sys 0.008s
I20260709 12:12:55.334756 13173 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:55.301116 13219 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:12:55.330312 13133 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.330452 13254 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:55.358587 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.337924 13156 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:55.305727 13144 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.363595 13007 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:55.323297 13271 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.365885 11256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33784
I20260709 12:12:55.375092 11256 raft_consensus.cc:606] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Received request to transfer leadership
W20260709 12:12:55.367241 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.371495 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.377091 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:55.357308 13246 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:12:55.378371 13272 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:55.339442 13267 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:55.383312 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.379820 11175 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:55.341207 13173 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:12:55.384262 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.356385 13163 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.394004 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 148660us
I20260709 12:12:55.341190 13274 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:12:55.382899 11928 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 12:12:55.374729 13141 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.403733 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.404541 11175 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:55.398680 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:55.414976 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47259.2us
I20260709 12:12:55.411559 11330 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:55.404981 11928 pending_rounds.cc:85] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 2
I20260709 12:12:55.417557 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.431809 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.399148 13267 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:55.424507 13226 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.436374 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.437127 11331 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:55.436897 13226 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.438000 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.438376 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:55.438716 11934 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:55.439111 11256 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:12:55.438954 12923 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.438632 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.440212 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:55.440367 11932 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:55.404553 13007 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:55.394184 13046 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:12:55.443173 11324 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:55.443720 11182 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:55.461890 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:55.424676 13115 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.401223 13163 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.458231 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:55.461367 11251 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:55.469002 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.442864 11928 pending_rounds.cc:107] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 12:12:55.467020 11253 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:12:55.462478 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 100393us
I20260709 12:12:55.469907 11325 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:55.474675 13245 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:55.475128 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 190) took 2203 ms. Trace:
I20260709 12:12:55.442335 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1135) took 2096 ms. Trace:
I20260709 12:12:55.487212 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.481524 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.486796 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:55.491984 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 96122us
I20260709 12:12:55.466238 13279 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.498899 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.499670 11324 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:55.433265 13141 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.477101 11928 raft_consensus.cc:2889] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:55.475822 13142 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:55.506325 11249 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:55.480983 13046 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:55.512210 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:55.516084 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.498057 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.525919 11322 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:12:55.526387 11930 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
W20260709 12:12:55.528833 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.521029 11182 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:55.521350 12923 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.521935 13142 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:55.529472 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.492797 13007 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.526165 13267 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.520412 13287 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:12:55.531427 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.533413 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 486) took 2096 ms. Trace:
I20260709 12:12:55.533170 13288 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.501834 13173 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.528168 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 429) took 1902 ms. Trace:
I20260709 12:12:55.566037 11331 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:12:55.551631 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.550995 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:55.565404 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.550783 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.553803 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:55.573242 11182 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:55.502429 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.578186 11255 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:55.571038 11331 pending_rounds.cc:85] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:12:55.581740 11179 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:55.582342 11331 pending_rounds.cc:107] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:12:55.582723 11331 raft_consensus.cc:2889] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:55.564333 11079 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:12:55.584415 11076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 166) took 11787 ms (client timeout 15000 ms). Trace:
I20260709 12:12:55.586066 11079 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:55.587829 11329 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:55.546988 13153 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.588291 11079 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 4, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 4 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:55.573277 11928 rpcz_store.cc:276] 0709 12:12:53.625803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.874311 (+248508us) service_pool.cc:224] Handling call
0709 12:12:53.874841 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.527908 (+1653067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.528084 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.590548 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:55.548233 11931 rpcz_store.cc:276] 0709 12:12:53.433350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.436852 (+  3502us) service_pool.cc:224] Handling call
0709 12:12:54.322867 (+886015us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:54.323177 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.323504 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:12:55.077847 (+754343us) log.cc:844] Serialized 14 byte log entry
0709 12:12:55.487578 (+409731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:55.490997 (+  3419us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:55.494414 (+  3417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.503800 (+  9386us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.529960 (+ 26160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16354,"prepare.queue_time_us":745208,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":18708,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":19069,"spinlock_wait_cycles":3092864,"thread_start_us":1093565,"threads_started":3,"wal-append.queue_time_us":252479}
I20260709 12:12:55.588974 13226 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:55.591825 13166 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.006s
I20260709 12:12:55.588006 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.592360 13166 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.593492 11326 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:55.594410 13271 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.591743 11328 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:55.593526 11176 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:12:55.538610 13278 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 12:12:55.556768 13263 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.384s	user 0.003s	sys 0.000s
I20260709 12:12:55.596946 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.597625 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.589001 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.542564 11930 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Advancing to term 4
I20260709 12:12:55.558890 13173 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:55.564314 13141 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.546300 13269 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.601281 11329 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:55.547807 13282 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:55.563158 13285 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.603106 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68474.2us
I20260709 12:12:55.547104 13142 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.508198 13279 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.562909 13163 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.549480 13288 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.549409 13159 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:55.546244 13046 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.487371 13156 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.605937 11324 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:55.537671 13287 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:55.600281 11178 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:55.610015 13282 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.632648 13285 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.608057 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.638075 11931 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:55.639174 13285 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.640066 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.611558 13007 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:55.546371 13277 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:55.640709 11180 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:55.641184 13277 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.641773 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:55.620090 13193 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.006s
I20260709 12:12:55.618264 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.642375 11180 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:55.621781 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.492249 13115 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.645838 11182 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:55.594794 13166 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.634991 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.635270 11255 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:55.636160 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.487438 11248 rpcz_store.cc:276] 0709 12:12:53.271681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.271879 (+   198us) service_pool.cc:224] Handling call
0709 12:12:53.272320 (+   441us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:53.272763 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:53.272795 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:53.273484 (+   689us) log.cc:844] Serialized 52 byte log entry
0709 12:12:54.173940 (+900456us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b18000c8680. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:12:54.389145 (+215205us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:55.401279 (+1012134us) log.cc:844] Serialized 14 byte log entry
0709 12:12:55.474292 (+ 73013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.474468 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.474534 (+    66us) raft_consensus.cc:1685] finished
0709 12:12:55.474653 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.474857 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27855,"prepare.queue_time_us":503565,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":36184,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":73188,"spinlock_wait_cycles":9174144,"thread_start_us":611141,"threads_started":3,"wal-append.queue_time_us":1803298}
I20260709 12:12:55.635746 13287 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.636943 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1759) took 1209 ms. Trace:
I20260709 12:12:55.610550 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:55.635010 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:55.620880 13159 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:12:55.643332 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.614192 13163 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:55.675238 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:55.676146 11255 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:55.675539 13163 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.611197 13271 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:55.678072 13271 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:55.680692 13176 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:55.678078 13287 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:55.682555 13209 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:55.683104 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.602062 11079 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 3, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:55.647437 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.649820 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.487651 11323 rpcz_store.cc:276] 0709 12:12:53.345834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.846563 (+500729us) service_pool.cc:224] Handling call
0709 12:12:53.883741 (+ 37178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.005647 (+121906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.005714 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.006498 (+   784us) log.cc:844] Serialized 26 byte log entry
0709 12:12:54.193785 (+187287us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:55.271109 (+1077324us) log.cc:844] Serialized 14 byte log entry
0709 12:12:55.400925 (+129816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.408895 (+  7970us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.412284 (+  3389us) raft_consensus.cc:1685] finished
0709 12:12:55.415592 (+  3308us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.433528 (+ 17936us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123603,"prepare.queue_time_us":33442,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":97,"raft.queue_time_us":191304,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":13607,"thread_start_us":1261069,"threads_started":3,"wal-append.queue_time_us":349171}
I20260709 12:12:55.651341 13099 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:55.651513 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:55.652055 13290 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:55.653972 12873 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:55.697904 12873 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:55.698143 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.698930 12873 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.647706 13193 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:55.649855 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:55.656582 13144 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:55.657490 13291 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.657987 13285 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.657974 11930 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:55.658666 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.645130 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.614988 13279 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.644852 13007 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.662150 13115 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.663547 11175 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:12:55.663969 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:55.664526 13162 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:55.666002 11929 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:55.637562 13282 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:55.666776 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:55.667066 13277 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.667593 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.635321 11928 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:12:55.617735 13269 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.612746 13226 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.670444 11327 rpcz_store.cc:276] 0709 12:12:54.427355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.960600 (+533245us) service_pool.cc:224] Handling call
0709 12:12:54.974910 (+ 14310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:55.266976 (+292066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:55.267007 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:55.290347 (+ 23340us) log.cc:844] Serialized 52 byte log entry
0709 12:12:55.468284 (+177937us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:55.468354 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.468447 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.636682 (+168235us) raft_consensus.cc:1685] finished
0709 12:12:55.636708 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.636880 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27935,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5092,"thread_start_us":383522,"threads_started":2,"wal-append.queue_time_us":206689}
I20260709 12:12:55.674084 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.687844 11181 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:55.693241 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.695668 11079 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:55.681412 13176 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.927248 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.699461 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48066.4us
W20260709 12:12:55.943917 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.944180 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 157578us
I20260709 12:12:55.948295 11177 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
W20260709 12:12:55.755115 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.952389 11249 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:55.646135 13292 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:55.953433 11931 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:12:55.766345 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.951339 11182 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:55.793841 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.954229 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.929342 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.951658 11179 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:12:55.584883 11076 rpcz_store.cc:269] 0709 12:12:43.796754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.820660 (+ 23906us) service_pool.cc:224] Handling call
0709 12:12:55.584153 (+11763493us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:51.186140 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:51.273915 (+ 87775us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:51.273975 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:12:51.285816 (+ 11841us) tablet.cc:662] Decoding operations
0709 12:12:51.539785 (+253969us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:51.546280 (+  6495us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:51.550676 (+  4396us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:12:51.574923 (+ 24247us) tablet.cc:701] Row locks acquired
0709 12:12:51.574953 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:12:51.575172 (+   219us) write_op.cc:270] Start()
0709 12:12:51.575337 (+   165us) write_op.cc:276] Timestamp: P: 1783599171575102 usec, L: 0
0709 12:12:51.575366 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 12:12:51.576345 (+   979us) log.cc:844] Serialized 2069 byte log entry
0709 12:12:51.959148 (+382803us) op_driver.cc:464] REPLICATION: finished
0709 12:12:52.077691 (+118543us) write_op.cc:301] APPLY: starting
0709 12:12:52.077802 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:52.077964 (+   162us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:52.137083 (+ 59119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:52.242672 (+105589us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:52.243020 (+   348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:12:52.243118 (+    98us) write_op.cc:312] APPLY: finished
0709 12:12:52.243484 (+   366us) log.cc:844] Serialized 47 byte log entry
0709 12:12:52.243731 (+   247us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:52.243906 (+   175us) write_op.cc:454] Released schema lock
0709 12:12:52.243977 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:52.244494 (+   517us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:52.244540 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1397586,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":103110,"apply.run_cpu_time_us":4374,"apply.run_wall_time_us":176475,"num_ops":5,"prepare.queue_time_us":220218,"prepare.run_cpu_time_us":41889,"prepare.run_wall_time_us":723345,"raft.queue_time_us":79493,"raft.run_cpu_time_us":40030,"raft.run_wall_time_us":74095,"replication_time_us":383625,"thread_start_us":249953,"threads_started":4,"wal-append.queue_time_us":262405,"wal-append.run_cpu_time_us":15787,"wal-append.run_wall_time_us":1413827}]]}
I20260709 12:12:55.956043 11252 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:55.957019 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:55.699054 11927 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 2: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:55.680735 13271 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.699803 13019 raft_consensus.cc:2764] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:12:55.968600 11176 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:55.969273 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:55.700861 13099 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:12:55.970467 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.994715 11252 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:12:55.982316 11331 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:55.995990 11176 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:55.997107 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:55.997922 11176 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:55.998588 13153 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.998786 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.999161 13153 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.000051 11175 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:12:56.001032 12873 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:56.003677 11182 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:55.705452 13288 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:55.951423 13162 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:55.953159 11178 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:55.985080 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:12:56.006994 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46112.1us
I20260709 12:12:55.701406 13290 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:55.986158 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:55.987110 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.017491 13162 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.984923 13167 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.009243 11180 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:55.963955 11079 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:55.956015 13130 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:55.965282 13272 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:56.024578 13272 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.024992 13272 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:56.025180 13225 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:56.025676 13300 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:12:56.026271 13163 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.018770 13289 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:56.030242 13289 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:56.030771 13289 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:55.962203 13166 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.001390 13153 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:56.012491 13046 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:56.015069 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.033018 11175 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:56.034077 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.960413 13226 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.016995 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:56.035452 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.006908 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.036603 11248 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:55.956773 13285 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.022576 11181 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:12:56.033772 13167 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.035507 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.038620 11177 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:56.039131 11175 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:56.039579 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:55.970356 13274 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:56.010281 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:56.040506 13274 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.040881 13046 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:56.045027 13166 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:55.962116 13209 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:12:56.050508 13209 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:56.039930 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.053032 11175 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:12:56.004649 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.023752 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:56.064834 11249 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:55.953821 13176 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.001904 13115 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.050809 11079 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 3, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:56.067677 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:56.050952 13209 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:56.066828 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.069036 11182 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:55.987788 13099 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:56.072623 11079 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.074374 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:56.074414 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:55.943032 13159 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.018857 13162 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:56.079041 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 94596.4us
I20260709 12:12:56.079564 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.087598 13225 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:56.088878 13209 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.090407 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.094563 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.095628 11252 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:56.052511 13290 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:55.962001 13291 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.093312 11181 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:12:56.092993 11079 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.098382 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.024169 13292 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.077924 13277 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:56.106470 11079 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 3, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:56.094270 13284 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:56.111071 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.013s	sys 0.020s
I20260709 12:12:56.101162 13153 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:56.110014 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:56.112563 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.013s	sys 0.020s
I20260709 12:12:56.113696 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:56.094756 13225 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.118088 13225 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:56.118379 13130 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:56.119731 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:56.120450 11331 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:56.121865 11174 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:56.124759 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:56.125672 11252 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:56.126502 11255 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 1.
I20260709 12:12:56.127676 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:56.128515 11327 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:56.104224 13163 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.132391 11182 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:56.132329 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.091495 13115 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.136193 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.139134 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.136821 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.115587 13209 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:56.115489 13193 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.115298 13267 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.121253 13159 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:56.142438 11253 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:56.124672 13019 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:56.144201 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.144701 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:56.146842 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.150007 13167 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.169288 11176 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:12:56.111094 13144 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:12:56.168574 11324 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:56.152556 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.132161 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:56.200922 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49498.8us
W20260709 12:12:56.189113 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105257us
I20260709 12:12:56.201769 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:56.203119 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:12:56.148370 13173 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.004s
I20260709 12:12:56.204650 13167 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:56.216410 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:56.179948 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.204353 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:56.205536 13279 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:56.146790 13153 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:56.126422 13277 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.121172 13272 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:56.188581 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 97203us
I20260709 12:12:56.224239 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:56.128789 13130 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:12:56.227371 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.245973 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.246636 11328 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:56.246847 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.247691 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.249168 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.226447 11175 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 12:12:56.231683 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 95574.8us
I20260709 12:12:56.148538 13166 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.234771 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.235247 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.235090 13153 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.236197 13106 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:56.224370 13167 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.207949 13099 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.233042 13274 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:56.239442 13292 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.153716 13007 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:56.247691 11329 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:56.171161 13193 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:56.252794 13302 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:56.293500 11329 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:12:56.294030 11329 pending_rounds.cc:85] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 1
I20260709 12:12:56.293979 13302 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.294426 11329 pending_rounds.cc:107] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:12:56.294754 11329 raft_consensus.cc:2889] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:56.253841 13290 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.295481 13290 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:56.277887 13167 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.302445 13274 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:56.200030 13285 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:56.303201 13277 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.303800 13285 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:56.255580 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:56.307375 11328 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:12:56.179603 13226 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.145123 13163 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.312031 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.312954 13163 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.144083 13299 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:56.313906 13299 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:56.314495 13299 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:56.224104 13279 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.317972 13279 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.318151 13099 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.317246 13167 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.225104 13019 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:56.322620 13019 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.321208 13167 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:56.330090 13176 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:56.278515 11249 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:12:56.162106 13267 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.281210 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:56.334252 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53170.2us
I20260709 12:12:56.288411 11180 raft_consensus.cc:2410] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:12:56.335851 13267 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.335796 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:56.336634 13226 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:12:56.225804 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55126.9us
I20260709 12:12:56.256922 13269 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.335795 13288 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:12:56.340723 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 85221us
I20260709 12:12:56.341122 13288 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.341548 13288 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:56.342404 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:56.343016 13272 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.344483 13173 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:56.344518 13304 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:56.345664 13304 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.346005 13304 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.349166 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:56.349776 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.343834 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:56.345192 13173 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:56.310251 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.321158 13115 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.351281 13173 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.352844 13284 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:56.353330 13284 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.353772 13284 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.354130 13287 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:56.354751 13287 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.355177 13287 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:56.357872 13271 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:56.358814 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:56.362989 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.292229 13007 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.364311 13007 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:56.359989 13279 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.360852 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:56.360792 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:56.361595 13046 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:56.362505 11179 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:56.379623 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:56.395684 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.349485 13299 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.372902 11327 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
W20260709 12:12:56.399158 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.424988 11329 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
W20260709 12:12:56.430169 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 152269us
I20260709 12:12:56.294503 13302 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:56.364329 13305 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:12:56.441337 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80624.8us
W20260709 12:12:56.439211 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 218282us
I20260709 12:12:56.388806 13156 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:12:56.451431 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 71899.5us
I20260709 12:12:56.452557 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:56.363011 13226 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:12:56.474800 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.473694 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.481004 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.481875 11327 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
W20260709 12:12:56.484382 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 251095us
I20260709 12:12:56.446691 13291 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:56.445468 13156 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:56.461376 12923 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:56.461606 13046 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.492638 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.492981 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.493527 11329 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:56.494270 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.495075 11327 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:56.450011 13269 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:12:56.472466 11178 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:56.444700 13289 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.506846 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.520162 11324 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:56.472536 13106 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:12:56.507184 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.506489 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.497481 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:56.523604 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.529713 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.530030 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.532788 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:56.533622 11932 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:56.520890 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:56.536507 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.537220 11249 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:56.537765 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.490396 13305 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:12:56.538714 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:56.538897 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.539616 11248 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:56.493287 13156 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:56.495990 12923 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:56.551625 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.537266 13304 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.552759 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.496982 13291 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:12:56.922976 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.922842 11250 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:56.925387 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.926018 12923 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:56.926451 13327 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:56.928186 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.604900 13106 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.938639 12923 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.939723 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.940762 11252 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:56.639348 11179 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:12:56.655985 13271 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:56.632725 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.942291 12923 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.943035 11179 pending_rounds.cc:85] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 1
I20260709 12:12:56.943420 11179 pending_rounds.cc:107] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:12:56.943691 11179 raft_consensus.cc:2889] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:56.945581 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:56.950665 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.630374 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:56.912120 13130 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.856575 13141 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.912712 13159 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.512691 13284 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.828915 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1744) took 1189 ms. Trace:
I20260709 12:12:56.914084 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:56.609145 13302 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.604627 13321 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:56.657164 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:56.977957 13130 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.978641 13159 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.979698 13302 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:56.638588 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.980547 13130 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.639180 13299 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.981120 13159 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:56.981241 11175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:56.915350 11174 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
W20260709 12:12:56.918035 13288 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.005s
I20260709 12:12:56.918207 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.982928 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.983573 11176 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:12:56.918836 13099 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.000s
I20260709 12:12:56.919260 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.984326 11180 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
W20260709 12:12:56.673249 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 305922us
W20260709 12:12:56.876459 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.687808 13282 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.726s	user 0.011s	sys 0.000s
I20260709 12:12:56.540194 13225 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:56.921622 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.921275 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:56.634650 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.991915 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.992312 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.579770 13267 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:56.519629 13269 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:56.947701 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:56.664740 11178 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:12:56.551563 11331 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:56.578593 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.911898 13316 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.951431 11179 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:12:56.962580 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 510080us
I20260709 12:12:56.663177 13279 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:56.966199 11173 rpcz_store.cc:276] 0709 12:12:55.639708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.956214 (+316506us) service_pool.cc:224] Handling call
0709 12:12:55.956515 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:56.133320 (+176805us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:12:56.483256 (+349936us) log.cc:844] Serialized 14 byte log entry
0709 12:12:56.743171 (+259915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:56.743251 (+    80us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:12:56.743314 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.828531 (+ 85217us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:56.828601 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:56.828817 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":404429,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":137,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":1849,"spinlock_wait_cycles":406038784,"thread_start_us":511682,"threads_started":2}
I20260709 12:12:56.999522 11255 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:57.001362 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:57.001896 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.002652 11173 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:56.507959 13193 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:56.661753 13162 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.568s	user 0.002s	sys 0.008s
I20260709 12:12:56.507037 13312 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:56.982339 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.918555 11931 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:12:56.550086 13115 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:56.658787 11323 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:56.989180 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.822453 13173 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:56.982221 11174 pending_rounds.cc:85] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 2
I20260709 12:12:56.594736 13287 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.662283 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:56.995853 11330 raft_consensus.cc:2410] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:56.565611 11177 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 2.
I20260709 12:12:56.957621 13291 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.043721 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:56.997354 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.994800 11934 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:56.985908 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:56.769393 13292 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:57.058869 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:57.059244 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 73414us
I20260709 12:12:56.662842 13289 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:57.036777 13173 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:57.063289 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.063834 13173 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.064673 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.965793 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.993649 13288 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.010643 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:56.981887 13323 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:12:57.068490 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.028612 12981 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.013001 13316 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:56.982652 13141 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:57.016244 13290 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.720s	user 0.008s	sys 0.001s
I20260709 12:12:57.012348 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.016440 11251 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:12:57.017552 11179 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:57.099040 11173 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:57.018286 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.018929 13226 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:56.994410 13282 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:56.994104 13267 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.022002 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:57.101739 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.008944 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.016490 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.102587 11257 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:12:57.102820 11255 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
W20260709 12:12:57.103058 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94185.2us
I20260709 12:12:57.025656 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.027302 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.029284 11175 raft_consensus.cc:606] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.103822 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.103996 11175 raft_consensus.cc:614] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:56.998965 11178 pending_rounds.cc:85] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 2
I20260709 12:12:57.106648 11178 pending_rounds.cc:107] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:12:57.106973 11178 raft_consensus.cc:2889] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:57.107053 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.107775 13282 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:57.021447 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.042s	sys 0.072s
I20260709 12:12:56.821162 13007 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:57.108496 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.042s	sys 0.072s
I20260709 12:12:57.101439 13226 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:57.109009 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 665113us
I20260709 12:12:57.109346 13226 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.110186 13007 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.110738 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:57.030264 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.111462 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.111816 11933 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:57.112267 11257 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:57.038579 11249 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 6. Ops: []
I20260709 12:12:57.113178 12923 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:57.037058 11174 pending_rounds.cc:107] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:12:57.113356 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.113683 12923 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.113880 11174 raft_consensus.cc:2889] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:57.114187 12923 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.114773 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.115976 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.116734 11249 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:57.116850 13316 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.117007 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:56.556910 13046 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.117801 11935 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:57.065496 13173 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.119333 13046 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:57.119712 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.119951 13046 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:57.120461 11248 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
W20260709 12:12:57.041677 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.108943 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.121987 11248 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:12:57.122897 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.062494 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.060024 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:12:57.124082 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64155.1us
I20260709 12:12:57.124683 13159 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:57.125779 13282 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:57.012495 13284 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:57.067536 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:57.127280 11900 connection.cc:256] RPC call timeout handler was delayed by -1.0774373149999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
I20260709 12:12:57.128497 13282 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:57.130211 13225 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.029596 13115 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:57.060168 13292 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:57.131207 13115 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:56.993834 13322 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:12:57.134868 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.135402 13284 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:57.136565 11327 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:57.072577 13162 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.066588 13320 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.138028 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.138460 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.138676 11331 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:57.139091 11327 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:57.138461 13320 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:57.069685 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 283) took 1120 ms. Trace:
I20260709 12:12:57.140934 13320 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.060099 11932 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEARNER]: Advancing to term 5
I20260709 12:12:57.060925 13291 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.098480 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 247) took 1087 ms. Trace:
W20260709 12:12:57.154124 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 50397.1us
I20260709 12:12:57.081703 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.155125 11256 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:57.155187 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.087638 12981 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.160454 13141 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.161151 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.093799 13269 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.161868 11328 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:57.162564 13269 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.162914 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.163491 11328 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.165191 13282 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:12:57.100291 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82115us
I20260709 12:12:57.104620 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.166435 11324 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [2.2-5.4]
I20260709 12:12:57.105929 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.106163 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.108181 11326 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:57.108425 13267 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:56.767445 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.125869 13325 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:57.129508 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.197618 13325 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.198062 11178 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:57.198302 13046 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:12:57.198869 11296 connection.cc:256] RPC call timeout handler was delayed by -1.872721147999755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
I20260709 12:12:57.199196 13046 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:57.199509 11296 connection.cc:256] RPC call timeout handler was delayed by -1.872721147999755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 12:12:57.200021 11296 connection.cc:256] RPC call timeout handler was delayed by -1.8718211479997535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
I20260709 12:12:57.199791 13046 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:57.200487 11296 connection.cc:256] RPC call timeout handler was delayed by -1.8412211479999314s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
W20260709 12:12:57.200942 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6864211479999085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
I20260709 12:12:57.202431 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 288) took 1139 ms. Trace:
I20260709 12:12:57.202863 11326 rpcz_store.cc:276] 0709 12:12:56.062824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.063002 (+   178us) service_pool.cc:224] Handling call
0709 12:12:57.202190 (+1139188us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000c71d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:57.202359 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2384291584}
I20260709 12:12:57.204217 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.204900 11326 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:57.130082 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.131817 13292 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.205976 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.206118 11174 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.206560 13292 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:57.206851 13322 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:57.207278 13322 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.207710 13322 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.066104 13271 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:57.065397 11250 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:57.042141 13312 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:57.150856 11180 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:57.152009 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.159185 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.160092 13007 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:57.154104 11177 rpcz_store.cc:276] 0709 12:12:56.011505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.039334 (+ 27829us) service_pool.cc:224] Handling call
0709 12:12:57.077776 (+1038442us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54000d7d54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:57.098417 (+ 20641us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207932160}
W20260709 12:12:57.167582 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.208897 13328 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:57.218722 11249 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:12:57.220073 13328 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:57.220844 13328 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:57.224047 13141 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.224712 13141 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.226370 13304 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:57.227027 13141 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:57.098829 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:57.228570 13322 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.171110 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.174880 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:57.175971 11127 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:12:57.231007 11127 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 4a634acd9ff543cca551c8a28e72d6c4 already in progress
W20260709 12:12:57.231323 11127 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 4a634acd9ff543cca551c8a28e72d6c4 already in progress
I20260709 12:12:57.231245 13166 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:57.231587 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.232040 13166 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.232324 11324 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:57.232576 13166 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.177167 11254 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:57.184106 11331 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:57.234082 11327 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:12:57.235937 13046 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:57.237843 13322 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.193475 13288 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:57.186964 13226 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.192366 11934 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:57.177074 13162 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:12:57.201395 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6099211479997848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
I20260709 12:12:57.198135 13325 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.192693 13193 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:57.141528 11323 rpcz_store.cc:276] 0709 12:12:55.946799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.949037 (+  2238us) service_pool.cc:224] Handling call
0709 12:12:56.021129 (+ 72092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:57.035787 (+1014658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.036555 (+   768us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:12:57.066519 (+ 29964us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361548544}
I20260709 12:12:57.270175 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.271117 13287 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.191799 11932 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:12:57.271946 11932 pending_rounds.cc:85] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 2
I20260709 12:12:57.272224 11932 pending_rounds.cc:107] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:12:57.272450 11932 raft_consensus.cc:2889] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:57.273532 13115 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.243072 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.276032 13115 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.215380 11329 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:57.215829 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 56753.7us
W20260709 12:12:57.230996 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59986us
I20260709 12:12:57.277714 13272 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.278074 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.278352 13272 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:57.279312 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.279364 13272 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.280226 13304 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:57.209731 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.281471 11251 raft_consensus.cc:2410] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:57.173406 13209 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:12:57.282224 13193 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:57.282644 13193 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.277251 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.216041 13007 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.261480 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:12:57.280642 13304 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.209033 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:57.264864 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6009211479997703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 12:12:57.211225 13271 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:57.211665 13279 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.285547 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.299371 11331 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.286641 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.299911 11255 raft_consensus.cc:2368] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5.
I20260709 12:12:57.300221 11326 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.288684 11329 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:12:57.296751 11296 connection.cc:256] RPC call timeout handler was delayed by -1.5793211479999627s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260709 12:12:57.283032 13193 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:57.296824 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 87876.1us
I20260709 12:12:57.282459 13209 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.211318 12981 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.296361 13115 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:57.299685 13304 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:57.326737 13115 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:57.327852 13115 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.296485 13305 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:57.329716 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4821211479998055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 12:12:57.330214 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4281211479999456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
W20260709 12:12:57.330547 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3948211479998918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 12:12:57.330852 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3660211479998452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
I20260709 12:12:57.331012 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:57.332016 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.331974 13316 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:57.332782 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:12:57.333076 13314 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.332806 13316 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.333874 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.333971 13316 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:57.334815 13299 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:57.335234 13299 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:57.335575 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 169457us
I20260709 12:12:57.335613 13299 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.336751 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.337247 13288 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:57.338156 13046 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:57.338285 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.338604 13046 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.339131 13046 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.339519 13287 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:57.296798 13007 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.341838 13279 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.341926 13304 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.338685 13288 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:12:57.344274 13279 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.345587 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.347074 13106 raft_consensus.cc:2764] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:12:57.347606 13282 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.348197 13282 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.348876 13299 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.351035 13282 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:57.335824 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.354051 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.355069 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.355466 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.356076 11329 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:57.356118 11327 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:57.357914 13130 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:57.358068 13299 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.358461 13130 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.359869 13130 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:57.359880 13279 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:57.363183 13288 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:57.363376 13282 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:57.364394 13290 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:57.364773 13290 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:57.365783 13290 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.367393 13290 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:57.372175 13130 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.374250 13153 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:57.381201 11329 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:57.359305 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.387604 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.374382 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.279922 12923 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.331336 13209 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.392112 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.408962 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.409332 11251 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.395537 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.410286 11254 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.410959 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.395910 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.396281 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.419147 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.397017 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.420290 11256 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.396354 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.421084 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:57.413878 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 108460us
I20260709 12:12:57.422171 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.422380 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.397747 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.422792 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.423236 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:57.400810 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.424844 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.427201 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.427521 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:57.427959 11933 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:57.428439 11931 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:57.221844 13328 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.429924 13308 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:57.430389 13328 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.431934 13308 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:57.405575 13302 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.405599 13279 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:57.434100 13279 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.434583 13279 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.437318 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.419066 13153 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:57.441591 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.442052 13327 raft_consensus.cc:993] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:57.427989 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:57.443569 11934 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:57.445196 13226 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.423305 11253 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:57.417055 11331 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
W20260709 12:12:57.457039 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.423017 11178 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:57.419711 11249 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:57.433498 13302 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.484593 13302 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.214494 13312 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.446099 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:57.408682 13299 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:12:57.516105 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58073.6us
I20260709 12:12:57.423017 11173 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:57.383271 13285 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.455454 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.453598 13209 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:57.462049 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:57.463025 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.522449 11177 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:57.463025 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.525197 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.526078 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.526928 11327 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.527768 11326 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:12:57.462800 11254 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:57.383983 13304 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.474434 13193 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.481962 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.536417 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.536851 11253 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:57.428476 11927 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:57.516434 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 291) took 1169 ms. Trace:
I20260709 12:12:57.481501 13305 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:57.499511 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:57.406124 11328 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:12:57.486284 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61507.9us
W20260709 12:12:57.534574 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.555725 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.556751 11330 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:57.535356 11251 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:12:57.537101 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:57.494760 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.538707 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:57.498714 13321 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.004s
I20260709 12:12:57.403597 13291 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.465148 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.498787 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.491803 13099 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:57.550946 13322 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:57.541344 11325 rpcz_store.cc:276] 0709 12:12:56.347513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.348850 (+  1337us) service_pool.cc:224] Handling call
0709 12:12:57.511789 (+1162939us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54002855d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:57.516368 (+  4579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155977472}
I20260709 12:12:57.595229 11933 raft_consensus.cc:2368] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:12:57.521838 11926 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:57.574872 11254 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:12:57.581880 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.499236 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.495273 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.420993 13046 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.513007 13299 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.496263 13327 raft_consensus.cc:1081] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:12:57.569983 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.554889 11931 raft_consensus.cc:2376] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3. Re-sending same reply.
I20260709 12:12:57.515648 13173 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:12:57.589169 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 68581.8us
I20260709 12:12:57.485574 13166 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.592851 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:57.593187 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48982.9us
I20260709 12:12:57.586580 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.599069 11327 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:12:57.602026 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.630019 11174 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:57.543388 13226 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:57.608337 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.490303 13162 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.588806 13321 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:57.647538 11255 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:57.567054 13209 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:57.623703 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:57.651048 11180 raft_consensus.cc:2368] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5.
I20260709 12:12:57.591135 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.621352 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.651705 11326 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:57.625988 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57707.3us
I20260709 12:12:57.614394 13269 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:57.626049 11328 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:57.536237 13285 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.631004 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:57.628332 11935 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:57.634253 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:57.637521 11173 raft_consensus.cc:2368] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4.
I20260709 12:12:57.608878 11256 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:57.603933 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:57.646283 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.626017 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.672804 11177 raft_consensus.cc:2376] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5. Re-sending same reply.
I20260709 12:12:57.674026 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.674746 11177 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.626627 13299 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.653963 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.677548 11174 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:57.646425 13226 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.624291 13153 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:12:57.658268 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.654016 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:57.682106 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.682610 11327 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:57.683028 11173 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:12:57.658294 13342 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:12:57.660136 13099 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:57.660975 13282 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:57.623008 13193 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:57.709375 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 63234.5us
I20260709 12:12:57.552421 13271 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.676450 11257 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:57.639168 13304 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.659493 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:57.652515 13156 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.002s
I20260709 12:12:57.692268 11177 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:12:57.693392 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64464.9us
I20260709 12:12:57.715179 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:57.681429 13305 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:57.705092 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.720014 11179 raft_consensus.cc:2376] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3. Re-sending same reply.
I20260709 12:12:57.721175 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.665858 13159 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:57.604086 13288 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:12:57.711036 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.694635 13173 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.718104 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:57.678186 13292 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.642740 13267 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.723516 11179 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:57.731129 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.717002 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.724319 11178 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:57.729701 13345 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.713909 13327 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:57.734390 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:57.732697 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 111484us
I20260709 12:12:57.799363 11174 raft_consensus.cc:2376] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5. Re-sending same reply.
I20260709 12:12:57.678903 13269 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:57.775076 11251 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:57.778358 11326 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:57.774013 11329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39342
I20260709 12:12:57.712318 13130 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.779735 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:57.730607 13156 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.704689 13321 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:57.713979 13141 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.796756 11327 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:57.772994 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:57.732434 13341 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:12:57.736048 13193 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.835929 11329 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:57.951617 11329 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:57.713434 13007 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.956820 11250 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:57.836141 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:57.836421 11173 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
W20260709 12:12:57.965859 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.732336 13287 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:57.838791 11182 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:57.965997 11249 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:12:57.837227 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:57.860807 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 154184us
I20260709 12:12:57.800841 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:57.968659 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.781665 11323 raft_consensus.cc:2410] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:12:57.973320 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 262370us
I20260709 12:12:57.974402 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:12:57.936685 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 277247us
W20260709 12:12:57.975221 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 138111us
I20260709 12:12:57.800184 13302 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:57.984530 11252 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:57.732682 13351 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:57.837159 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:12:57.969506 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.957216 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.993692 11174 raft_consensus.cc:2368] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:12:57.773442 12923 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.992478 11253 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:57.942384 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:57.743996 13337 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.409s
I20260709 12:12:57.774299 13292 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:57.962458 11329 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:57.981041 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:58.000702 11173 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:57.845418 13345 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:57.828411 13289 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:57.998019 11179 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3.
I20260709 12:12:58.001976 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:57.842729 13153 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:57.736516 13328 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.245s
W20260709 12:12:57.977178 13314 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.004s
I20260709 12:12:57.952250 11930 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:12:57.744359 13316 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.015630 11329 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:57.961090 13321 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:12:58.018344 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49779.3us
I20260709 12:12:58.008731 13291 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.976886 13141 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:57.994851 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:57.993207 13173 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:58.029488 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:57.986776 13304 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:57.794613 13209 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.027295 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:58.013471 13289 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:58.042479 11249 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:12:58.059973 11174 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:58.027555 13312 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:57.995641 13130 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:12:58.064064 13343 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.436s	user 0.001s	sys 0.000s
I20260709 12:12:58.063824 13304 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.066879 13156 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:58.055564 11248 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:12:58.068257 13269 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.069268 13226 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:58.069442 13282 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:58.070283 13267 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.031569 13314 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:58.072163 13007 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:58.074440 13349 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:12:58.074349 13099 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:58.074867 13350 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:58.075975 13285 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:57.986476 13287 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:57.974169 13193 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:58.073330 13279 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.000s
W20260709 12:12:58.078130 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 103839us
I20260709 12:12:58.152820 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:58.090476 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.111971 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:58.042210 12923 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:58.122954 11174 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:58.043133 13046 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.166253 12923 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:58.167382 12923 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:58.169975 13173 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.170647 11328 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:12:58.171180 13193 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.172226 11177 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:12:58.175619 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1626) took 1148 ms. Trace:
I20260709 12:12:58.176254 11176 rpcz_store.cc:276] 0709 12:12:57.027603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.035340 (+  7737us) service_pool.cc:224] Handling call
0709 12:12:58.175295 (+1139955us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54000cd854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:58.175511 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785810688}
W20260709 12:12:58.180169 13352 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:12:58.181814 13279 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:58.184689 13292 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.141727 13363 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:12:58.202273 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.220646 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:58.485244 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 455786us
I20260709 12:12:58.438792 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 251) took 1311 ms. Trace:
I20260709 12:12:58.241005 13289 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.077603 13166 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.110978 13312 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:58.128688 13272 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.012s	sys 0.052s
I20260709 12:12:58.505592 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1782) took 1004 ms. Trace:
I20260709 12:12:58.507651 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:58.508306 11329 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
W20260709 12:12:58.137609 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 68708.2us
I20260709 12:12:58.128811 13305 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:58.156947 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:58.515594 11174 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:58.488257 13351 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:58.193547 13369 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:58.489195 11175 rpcz_store.cc:276] 0709 12:12:57.127792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.130387 (+  2595us) service_pool.cc:224] Handling call
0709 12:12:57.130722 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:57.193188 (+ 62466us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:12:57.588959 (+395771us) log.cc:844] Serialized 14 byte log entry
0709 12:12:57.680970 (+ 92011us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:57.705276 (+ 24306us) log.cc:844] Serialized 26 byte log entry
0709 12:12:57.705433 (+   157us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:57.705469 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.705540 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.438451 (+732911us) raft_consensus.cc:1685] finished
0709 12:12:58.438484 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.438701 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214238,"prepare.queue_time_us":282745,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":176117,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":35525,"thread_start_us":440399,"threads_started":2,"wal-append.queue_time_us":318214}
I20260709 12:12:58.487742 13279 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.153479 13345 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:58.456753 13287 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:58.185599 13141 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.239128 13316 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:12:58.428802 13325 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.127s	user 0.011s	sys 0.000s
I20260709 12:12:58.409658 13292 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.286784 13363 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:58.241659 13321 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:58.134883 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 334115us
W20260709 12:12:58.495605 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 342862us
I20260709 12:12:58.495513 11179 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
W20260709 12:12:58.392333 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.361449 13366 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.506050 11180 rpcz_store.cc:276] 0709 12:12:57.500799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.498484 (+997685us) service_pool.cc:224] Handling call
0709 12:12:58.498815 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.500399 (+  1584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.500428 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:58.500476 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:58.505364 (+  4888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.505432 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.505537 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.518648 11176 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:58.518831 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:58.497224 13099 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.536427 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:12:58.500522 13272 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.012s	sys 0.052s
I20260709 12:12:58.571115 11324 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:12:58.589809 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:58.564441 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.500658 13312 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:58.592873 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.572701 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:58.594236 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1783) took 1093 ms. Trace:
I20260709 12:12:58.590520 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:58.516441 13288 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.631479 11174 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:12:58.514779 13305 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:58.635183 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.631964 11176 rpcz_store.cc:276] 0709 12:12:57.501080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.498413 (+997333us) service_pool.cc:224] Handling call
0709 12:12:58.518479 (+ 20066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.593971 (+ 75492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.594180 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.629487 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:58.637830 13282 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:58.549403 13376 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.569788 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:58.631894 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:58.517354 13289 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:58.654503 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64050.3us
I20260709 12:12:58.650693 13305 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:58.628371 13287 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:58.631965 13279 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:58.532969 13284 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:58.637734 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 102058us
I20260709 12:12:58.664392 11173 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:58.671321 13331 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:12:58.630072 13046 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:58.638936 11174 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:12:58.568209 13156 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 12:12:58.647747 13106 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:58.628635 13226 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:12:58.675902 13166 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.663759 13225 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.630333 13325 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.630584 13338 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:12:58.628417 13099 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:58.631026 13366 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.685942 13284 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.704273 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:58.705133 13327 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:58.676100 13376 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.706736 13363 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.708796 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1774) took 1215 ms. Trace:
I20260709 12:12:58.723013 11178 rpcz_store.cc:276] 0709 12:12:57.493318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.494918 (+  1600us) service_pool.cc:224] Handling call
0709 12:12:58.708575 (+1213657us) spinlock_profiling.cc:243] Waited 92.6 ms on lock 0x7b540016f8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:58.708727 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212934400}
I20260709 12:12:58.695353 13279 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:58.709824 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:58.710472 11147 leader_election.cc:400] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5.
I20260709 12:12:58.710995 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1328) took 1316 ms. Trace:
I20260709 12:12:58.718601 13316 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.714071 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1627) took 1686 ms. Trace:
I20260709 12:12:58.736588 11257 rpcz_store.cc:276] 0709 12:12:57.394474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.410730 (+ 16256us) service_pool.cc:224] Handling call
0709 12:12:58.710786 (+1300056us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54002a62d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:58.710928 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":608984448}
W20260709 12:12:58.715569 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.718984 13327 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:58.627434 13292 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.681964 11248 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:58.719432 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:58.684332 13046 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:12:58.709743 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1617) took 1792 ms (client timeout 1669 ms). Trace:
W20260709 12:12:58.746316 11182 rpcz_store.cc:269] 0709 12:12:56.917543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.918890 (+  1347us) service_pool.cc:224] Handling call
0709 12:12:58.709489 (+1790599us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54001f45d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:58.709667 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964034944}
I20260709 12:12:58.712857 13099 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:58.737766 11176 raft_consensus.cc:606] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.749490 11176 raft_consensus.cc:614] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:58.736559 11147 leader_election.cc:403] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:58.738948 11181 rpcz_store.cc:276] 0709 12:12:57.027885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.067238 (+ 39353us) service_pool.cc:224] Handling call
0709 12:12:58.713883 (+1646645us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400364a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:58.714010 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3486947712}
I20260709 12:12:58.713996 11179 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:12:58.741806 11932 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:12:58.693667 13106 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.722383 13282 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:12:58.745544 11929 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:12:58.754346 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60028.8us
W20260709 12:12:58.778980 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59627.1us
I20260709 12:12:58.722150 13351 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:58.748392 13299 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.052s	user 0.014s	sys 0.000s
I20260709 12:12:58.781068 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.781373 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:58.731495 13279 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:58.761137 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92988.8us
I20260709 12:12:58.785194 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:58.749837 13046 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:58.797111 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 204524us
I20260709 12:12:58.771998 13369 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:58.799190 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.780361 13162 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:12:58.742033 13316 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.819977 13316 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:58.820806 13316 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:58.806922 11222 leader_election.cc:400] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:12:58.823732 11222 leader_election.cc:403] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:58.825290 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.826205 13226 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:12:58.826474 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.826916 13226 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:58.827476 13226 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:58.828065 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.828351 13226 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.829006 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.808132 13376 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:58.783073 13377 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:58.801769 13282 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.811224 13338 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.830960 13338 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:58.833160 13292 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:58.839131 11181 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:12:58.840413 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:12:58.812458 13378 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:12:58.813616 13322 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:58.843451 11181 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:12:58.839202 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.845466 13376 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.848070 11178 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:12:58.852686 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:58.852759 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 162272us
I20260709 12:12:58.821444 13316 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:58.855806 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:58.813694 13325 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:58.866575 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:58.870388 11178 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:58.871134 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:58.854422 13225 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:58.845178 13322 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.785447 13299 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:58.833241 13338 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.884124 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:58.884971 11328 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:58.861581 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:58.877246 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:58.801045 13099 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:12:58.891887 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.888937 11254 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:12:58.902382 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:58.907979 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.880594 13376 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:58.909909 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:58.909966 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:12:58.910800 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 129558us
I20260709 12:12:58.913453 11329 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:12:58.883688 13322 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:58.785705 13373 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:58.887899 13386 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:12:58.919812 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 134688us
I20260709 12:12:58.844349 13327 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:58.890292 13099 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:58.901901 13338 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.923779 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.883947 13209 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:12:58.924762 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:58.928488 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:58.930050 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:58.930670 11176 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:58.913193 13376 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.915760 13331 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:58.918329 13321 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.919404 13373 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.915252 13284 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.932049 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:58.934940 11178 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:58.938611 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.939101 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:58.941711 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:58.943387 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:58.945104 11329 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:12:58.945254 11257 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:12:58.945271 13377 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:58.946161 11257 raft_consensus.cc:606] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.949486 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:58.950599 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.935514 13376 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:58.952596 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.960866 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:58.965992 11325 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:58.963794 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:58.967024 11178 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:58.941655 13226 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.970938 11326 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:12:58.973618 13141 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:58.974177 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:58.953960 13325 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:58.974975 13141 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:58.975658 13325 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:58.976111 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:58.976707 13325 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.960122 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1788) took 1310 ms. Trace:
I20260709 12:12:58.974888 13287 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:58.930994 13389 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:12:58.941936 13381 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:12:58.938117 13373 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:58.977818 11175 rpcz_store.cc:276] 0709 12:12:57.649593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.562096 (+912503us) service_pool.cc:224] Handling call
0709 12:12:58.629491 (+ 67395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.629778 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.629808 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:58.643765 (+ 13957us) log.cc:844] Serialized 26 byte log entry
0709 12:12:58.860981 (+217216us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:58.861041 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.861143 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.959849 (+ 98706us) raft_consensus.cc:1685] finished
0709 12:12:58.959874 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.960049 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":175708,"threads_started":1,"wal-append.queue_time_us":281943}
I20260709 12:12:58.938489 13345 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:58.983961 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1793) took 1328 ms. Trace:
I20260709 12:12:58.952836 13377 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.992272 13345 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:58.941434 13366 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:58.992733 13162 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5.
I20260709 12:12:58.993726 13279 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:58.982831 11222 leader_election.cc:400] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4.
I20260709 12:12:58.940605 13282 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:58.952373 13327 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.014689 11329 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.018306 13305 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:58.952811 13106 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.020888 13305 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.990706 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:58.951066 13395 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.021929 11254 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:58.988451 11325 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:59.022802 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1801) took 1239 ms. Trace:
I20260709 12:12:59.023115 11325 rpcz_store.cc:276] 0709 12:12:57.783930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.784126 (+   196us) service_pool.cc:224] Handling call
0709 12:12:58.960700 (+1176574us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:59.022746 (+ 62046us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7500544}
I20260709 12:12:59.025246 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:58.941970 13387 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:12:59.026006 13387 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.026088 13226 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:59.026531 13387 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:12:59.026757 13226 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:59.027757 13226 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:58.992247 11180 rpcz_store.cc:276] 0709 12:12:57.655869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.628503 (+972634us) service_pool.cc:224] Handling call
0709 12:12:58.628975 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.634896 (+  5921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.634945 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:58.660082 (+ 25137us) log.cc:844] Serialized 26 byte log entry
0709 12:12:58.895800 (+235718us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:58.895840 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.895909 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.983740 (+ 87831us) raft_consensus.cc:1685] finished
0709 12:12:58.983759 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.983898 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1951,"raft.queue_time_us":130,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":10774,"thread_start_us":196600,"threads_started":1,"wal-append.queue_time_us":257370}
I20260709 12:12:58.998970 11257 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:12:58.984161 13401 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:58.933436 13338 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:58.983369 13389 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.981500 13321 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.983071 13299 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:58.981863 13099 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.008836 13325 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:58.978343 13159 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:12:59.054970 11173 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:59.009963 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 379) took 1313 ms. Trace:
I20260709 12:12:59.010391 13312 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.032775 11934 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 3: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:59.034425 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:12:59.037853 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 96243.9us
I20260709 12:12:59.057904 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.059360 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:59.044605 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 61897.2us
I20260709 12:12:58.977584 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:59.038133 11182 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:59.056334 11935 rpcz_store.cc:276] 0709 12:12:57.696356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.696593 (+   237us) service_pool.cc:224] Handling call
0709 12:12:59.009747 (+1313154us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b540069e454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:59.009895 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192526592}
I20260709 12:12:59.062000 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:58.931463 13316 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.067809 11325 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:12:58.982040 13288 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:12:59.044052 13162 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.037392 13401 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.048590 13366 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:12:59.044054 13389 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:12:59.077345 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:59.078642 11181 raft_consensus.cc:2368] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:12:59.068902 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:59.082821 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:59.085983 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.086874 11254 raft_consensus.cc:2410] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:12:59.087657 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.088160 11254 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.090034 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.091308 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.091789 11249 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.093957 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.094784 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.095315 11251 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.098120 13106 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:59.098666 13106 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:59.099543 13106 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.103161 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:59.104306 11176 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:12:59.104619 11178 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:59.107162 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:59.108481 11176 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:12:59.124909 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:59.129215 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:59.134145 11176 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.125710 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:59.136052 11178 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:59.110880 11932 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:59.137426 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:59.136394 11251 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:59.142287 11178 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:12:59.142989 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:59.144399 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.060638 13159 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.150435 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.152637 13271 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.155293 13338 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.065721 13289 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:59.180337 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.180936 11327 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.158756 13322 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.160364 13162 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.161322 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.073581 13099 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.072221 13282 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:59.161921 13401 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.163035 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:59.163650 13312 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.071519 11926 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:59.067363 13394 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:58.993536 13345 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:12:59.165195 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 87946.2us
I20260709 12:12:59.057005 13287 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.165913 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.219573 11329 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:12:59.146787 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:59.220337 11173 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:59.177959 11254 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:59.221484 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.177800 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:12:59.178833 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2285404140000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:12:59.223343 11251 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:59.223788 11251 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:59.181399 11176 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:59.184474 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:59.187616 13373 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.188844 11323 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:12:59.074689 13377 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 12:12:59.053432 12981 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:12:59.164669 11322 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:12:59.233831 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.234072 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1799) took 1508 ms. Trace:
I20260709 12:12:59.204737 11327 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:59.234476 13162 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.235100 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.235199 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.236374 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.206283 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.207201 11935 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:12:59.208771 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.182850 13153 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.225462 11178 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.196571 13288 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.242909 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:59.218722 11248 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:59.237798 13402 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.245584 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.224798 11251 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:59.242636 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:59.260949 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 222230us
I20260709 12:12:59.189718 13366 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.234506 11322 rpcz_store.cc:276] 0709 12:12:57.725885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.732860 (+  6975us) service_pool.cc:224] Handling call
0709 12:12:59.164249 (+1431389us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7eee 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:12:59.164426 (+   177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:59.233805 (+ 69379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.234006 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37457152}
I20260709 12:12:59.266742 11322 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:12:59.266935 13366 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.268188 11254 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:59.269573 11328 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:59.270051 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:59.271131 13387 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.273281 13387 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.273821 13279 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.274381 13279 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.262223 11325 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:59.261981 11249 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:59.262265 11251 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.278892 11222 leader_election.cc:400] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:12:59.242211 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.279724 11222 leader_election.cc:403] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:12:59.245591 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.288908 11323 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:59.231537 13289 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:12:59.294256 13322 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:59.296345 13159 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.298067 13225 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.301818 13373 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.302436 13159 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.246721 13153 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:59.304914 13405 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.002s
I20260709 12:12:59.305531 13153 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:59.275406 13402 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.307034 11327 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:59.307224 13406 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:12:59.307930 13402 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:59.275668 11926 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:12:59.229727 13287 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.310406 13287 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.312611 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:59.314287 11327 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:59.320164 11326 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.320402 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:59.327020 11251 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:12:59.328336 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.329288 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.332891 11178 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
W20260709 12:12:59.336802 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.344960 11326 raft_consensus.cc:2410] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:12:59.348214 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.319294 13279 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.366856 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:12:59.366935 13376 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.367916 11931 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:12:59.354506 13395 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.326071 13387 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:12:59.357046 13331 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.382308 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:59.382969 11935 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.385900 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:12:59.389039 11253 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:59.394065 11250 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
W20260709 12:12:59.394084 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.396440 13290 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:59.397017 13290 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:59.398131 13290 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.399379 13290 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:59.409061 13046 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.000s
W20260709 12:12:59.410133 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1311) took 2457 ms (client timeout 1876 ms). Trace:
W20260709 12:12:59.410668 11252 rpcz_store.cc:269] 0709 12:12:56.952753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.952899 (+   146us) service_pool.cc:224] Handling call
0709 12:12:59.409857 (+2456958us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b540033ccd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:59.410011 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2186593664}
I20260709 12:12:59.411850 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.412443 11252 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.415787 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.409608 13046 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:12:59.450356 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:12:59.483876 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 205135us
W20260709 12:12:59.586125 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 362491us
I20260709 12:12:59.630976 13403 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.660036 13403 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:59.661986 13380 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.925s	user 0.003s	sys 0.000s
I20260709 12:12:59.662956 11933 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:12:59.665225 11177 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:12:59.665879 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1786) took 2073 ms. Trace:
I20260709 12:12:59.666162 11177 rpcz_store.cc:276] 0709 12:12:57.592723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.664647 (+2071924us) service_pool.cc:224] Handling call
0709 12:12:59.664998 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.665596 (+   598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.665826 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.632262 13389 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.633904 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.667052 11325 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:12:59.669437 11175 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:12:59.671279 11929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 444) took 2689 ms (client timeout 2165 ms). Trace:
W20260709 12:12:59.671734 11929 rpcz_store.cc:269] 0709 12:12:56.981949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.997959 (+ 16010us) service_pool.cc:224] Handling call
0709 12:12:57.082450 (+ 84491us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:12:59.671026 (+2588576us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400690ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:59.671189 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3561484800}
I20260709 12:12:59.636020 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.673964 11330 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:59.676183 13389 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.676939 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.677554 11253 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.637326 13376 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.637877 13162 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:59.678534 13162 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.678970 13162 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.638319 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.682170 13413 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:12:59.682298 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44060.7us
I20260709 12:12:59.683182 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.684008 13413 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:59.684442 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.685258 13413 raft_consensus.cc:2764] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:12:59.686091 13355 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:59.686512 13355 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.687337 13355 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.688417 13355 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.690227 11253 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:59.690656 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.691232 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.691366 11251 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.691980 11253 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:12:59.692833 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.639575 11935 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:59.640332 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:12:59.641335 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:59.677564 11325 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:12:59.693765 11174 raft_consensus.cc:606] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Received request to transfer leadership to TS 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:59.694190 11174 raft_consensus.cc:614] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:59.694384 13166 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:59.694780 13166 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.695119 13166 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:59.694192 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52959.7us
I20260709 12:12:59.698920 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:59.699693 13345 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:59.699808 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:59.700196 13345 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.695130 11254 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:12:59.701499 11253 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:12:59.701248 13345 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.703624 11927 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:12:59.704455 13279 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:12:59.704998 13279 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.706446 13279 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:12:59.708978 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:59.709683 11148 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.832s (SENT) [suppressed 19 similar messages]
I20260709 12:12:59.714910 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.734802 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.646957 13115 raft_consensus.cc:1064] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:12:59.342247 13159 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.236s
I20260709 12:12:59.653532 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.751914 11251 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:12:59.752333 11248 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:12:59.752916 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:12:59.655134 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1794) took 1999 ms. Trace:
W20260709 12:12:59.655833 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1331) took 2257 ms (client timeout 2179 ms). Trace:
W20260709 12:12:59.658973 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.753587 11248 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
W20260709 12:12:59.754557 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.755180 11254 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:12:59.755614 11248 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:12:59.755751 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.756352 11325 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.756484 11254 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:12:59.757880 11254 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:59.759136 11254 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:59.645452 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:59.726943 13166 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.727645 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.760881 11250 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:59.762039 13166 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.735374 13279 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:59.762816 11330 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:12:59.736485 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.728082 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.743705 13350 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:59.764765 13350 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:12:59.765655 13350 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.766598 13350 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.766744 13166 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:59.767354 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.710285 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:12:59.768177 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.701880 11253 pending_rounds.cc:85] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 2
I20260709 12:12:59.768654 11253 pending_rounds.cc:107] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:12:59.768987 11253 raft_consensus.cc:2889] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:59.753798 11179 rpcz_store.cc:276] 0709 12:12:57.656172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.629242 (+973070us) service_pool.cc:224] Handling call
0709 12:12:58.713744 (+ 84502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.751915 (+ 38171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.654929 (+903014us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b54001f45d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:59.655072 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131659648}
I20260709 12:12:59.742390 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:12:59.751384 11173 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:12:59.770773 11174 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:59.768054 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57901.7us
I20260709 12:12:59.771739 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:12:59.772012 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 338188us
I20260709 12:12:59.772954 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.773186 13290 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:59.754237 11255 rpcz_store.cc:269] 0709 12:12:57.398459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.436988 (+ 38529us) service_pool.cc:224] Handling call
0709 12:12:59.655640 (+2218652us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54000870d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:12:59.655764 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481142016}
I20260709 12:12:59.773757 13290 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.774616 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:12:59.774634 11255 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:59.776412 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.776726 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.777352 11322 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:12:59.777595 13321 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:59.777729 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.777936 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.778172 13321 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.778496 13173 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:59.779439 11330 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:12:59.753942 13115 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.779863 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:59.774920 13159 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.778952 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.780970 11329 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:12:59.775864 13290 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.737739 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.778563 13321 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.782088 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:12:59.782407 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 96804us
I20260709 12:12:59.782946 11256 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:12:59.783310 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:12:59.762764 13279 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.783977 13173 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:59.784637 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.784869 13413 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:59.785267 13413 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:12:59.785619 13413 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.786350 13320 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:12:59.786773 13320 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.787117 13320 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.788537 11325 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:59.789129 13289 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.789827 13374 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:59.784644 13279 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.789943 13209 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:12:59.747424 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.793030 13272 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.793648 13272 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.794220 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.794545 13393 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.795261 13393 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:59.795696 13393 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.795987 13272 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:12:59.796377 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.796536 13393 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.762128 11250 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:12:59.797972 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:59.797842 13325 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:59.798585 13366 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:12:59.798257 13162 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.799913 13325 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:59.800699 13162 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.802585 13358 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:12:59.803475 11079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 236) took 15349 ms (client timeout 15000 ms). Trace:
I20260709 12:12:59.804378 13305 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.805200 13305 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.807173 11329 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:12:59.807592 11329 pending_rounds.cc:85] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:12:59.807516 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.807818 13305 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.807987 11329 pending_rounds.cc:107] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:12:59.808312 11329 raft_consensus.cc:2889] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:59.808387 13159 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:12:59.809954 11330 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:12:59.810354 11330 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:12:59.810354 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:12:59.811215 11330 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.812381 11330 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:12:59.812917 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:12:59.806146 13345 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:59.813756 13325 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:59.814329 13345 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.815442 13272 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:59.817302 13272 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:59.819022 13272 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:59.820369 13413 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.822270 13173 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:59.820753 13272 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.824295 11325 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:12:59.824945 13272 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.814446 13325 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:12:59.794032 11248 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:12:59.828408 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:59.815644 13345 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:59.803874 11079 rpcz_store.cc:269] 0709 12:12:44.454137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:44.454759 (+   622us) service_pool.cc:224] Handling call
0709 12:12:59.803387 (+15348628us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:56.514541 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:56.515327 (+   786us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:56.515380 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:12:56.515402 (+    22us) tablet.cc:662] Decoding operations
0709 12:12:56.516583 (+  1181us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:56.516670 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:56.516703 (+    33us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:12:56.579094 (+ 62391us) tablet.cc:701] Row locks acquired
0709 12:12:56.609746 (+ 30652us) write_op.cc:260] PREPARE: finished
0709 12:12:56.610026 (+   280us) write_op.cc:270] Start()
0709 12:12:56.610238 (+   212us) write_op.cc:276] Timestamp: P: 1783599176609938 usec, L: 0
0709 12:12:56.610271 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:12:56.611115 (+   844us) log.cc:844] Serialized 4072 byte log entry
0709 12:12:57.490774 (+879659us) op_driver.cc:464] REPLICATION: finished
0709 12:12:57.799871 (+309097us) write_op.cc:301] APPLY: starting
0709 12:12:57.845277 (+ 45406us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:58.030945 (+185668us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:58.075602 (+ 44657us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:58.407026 (+331424us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:58.407623 (+   597us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:12:58.407712 (+    89us) write_op.cc:312] APPLY: finished
0709 12:12:58.408320 (+   608us) log.cc:844] Serialized 77 byte log entry
0709 12:12:58.535028 (+126708us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:58.535187 (+   159us) write_op.cc:454] Released schema lock
0709 12:12:58.535251 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:58.535680 (+   429us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:58.535708 (+    28us) write_op.cc:454] Released schema lock
0709 12:12:59.661380 (+1125672us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
Metrics: {"mutex_wait_us":24062,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":213934,"apply.run_cpu_time_us":17253,"apply.run_wall_time_us":798768,"num_ops":10,"prepare.queue_time_us":234850,"prepare.run_cpu_time_us":28637,"prepare.run_wall_time_us":488624,"raft.queue_time_us":493519,"raft.run_cpu_time_us":26811,"raft.run_wall_time_us":235409,"replication_time_us":880295,"spinlock_wait_cycles":4985984,"thread_start_us":934826,"threads_started":5,"wal-append.queue_time_us":607538,"wal-append.run_cpu_time_us":36505,"wal-append.run_wall_time_us":1161188}]]}
I20260709 12:12:59.828171 13325 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.825173 13159 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:59.643191 13401 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.826550 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:59.874159 11257 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:12:59.874311 13159 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:59.875031 13159 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.875221 13401 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.847934 11330 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:12:59.876106 11330 pending_rounds.cc:85] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:12:59.876555 11330 pending_rounds.cc:107] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:12:59.849946 13321 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.851184 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:12:59.797977 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.768873 13373 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:12:59.874215 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47767.3us
I20260709 12:12:59.877578 11175 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:59.877743 13373 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.878582 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:12:59.878767 13373 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.879528 13162 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:59.860404 11256 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:59.880968 13321 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.856962 13413 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:12:59.881300 11173 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:59.881305 13355 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:59.880457 13162 consensus_queue.cc:1243] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:12:59.644156 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:12:59.816179 11174 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:12:59.880765 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.883759 11249 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:12:59.876192 13159 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:12:59.884773 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 239417us
I20260709 12:12:59.885367 13413 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:59.885929 11222 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 2 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 2. Current term is 3.
I20260709 12:12:59.885061 13393 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:12:59.879833 13316 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:12:59.882426 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 84535.1us
I20260709 12:12:59.760433 11254 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:12:59.877022 11330 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:12:59.889981 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.890120 13290 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:12:59.890513 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.891633 13290 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:59.887909 13393 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:12:59.887657 13316 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.895555 13316 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:59.895694 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.896920 13289 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:59.898260 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:12:59.898759 13320 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:12:59.894279 13290 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:59.901916 13374 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.886788 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:59.903923 13316 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.905944 13393 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:12:59.853618 13403 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:12:59.915066 13361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.346s	user 0.000s	sys 0.014s
I20260709 12:12:59.861446 13046 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:12:59.926146 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:12:59.642535 11252 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:12:59.927680 11327 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:12:59.928573 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:12:59.955054 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:00.049124 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.049300 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:00.053329 11252 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:12:59.887599 11222 leader_election.cc:400] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5.
I20260709 12:13:00.009764 11325 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:13:00.048565 11324 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:12:59.989533 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:00.054978 11222 leader_election.cc:403] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:12:59.865262 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 98728.8us
I20260709 12:13:00.059899 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:00.054832 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 168123us
I20260709 12:12:59.753517 11251 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:59.954751 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74392.5us
I20260709 12:13:00.066020 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:12:59.928754 11076 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 3, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:00.070538 11076 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 2 to 3, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 3 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:13:00.054154 13361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.485s	user 0.000s	sys 0.018s
I20260709 12:12:59.936584 11323 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:00.084247 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:00.074499 11076 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:00.079457 13141 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:00.094138 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 206890us
I20260709 12:12:59.995692 13166 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:00.080948 13209 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:00.082159 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:00.077349 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:00.083338 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:00.086244 11254 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:00.084686 13403 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5.
I20260709 12:13:00.098814 11076 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 2, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:13:00.080157 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 90719.7us
I20260709 12:12:59.988055 13325 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.073704 13272 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:00.134464 13272 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.134941 13272 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:00.108594 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.109593 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:00.111284 11251 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:00.082276 13412 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.138290 11253 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:00.094345 13312 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.125733 11076 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 2 to 3, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:00.105721 13290 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:00.130668 13361 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:00.131210 13422 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:00.149524 13316 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:00.150447 13316 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:00.131453 13401 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.105119 13209 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.133086 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:00.133740 13366 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:00.095115 13393 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:00.082058 13046 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:00.125451 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:00.143688 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46737.2us
I20260709 12:13:00.145131 11076 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:00.200557 11076 catalog_manager.cc:5697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:00.146193 13335 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:00.164497 11325 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:00.165792 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 107114us
I20260709 12:13:00.204281 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:00.205482 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:00.067744 13173 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:13:00.174013 13106 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:00.178089 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1620) took 1039 ms. Trace:
I20260709 12:13:00.168373 13209 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:00.202068 11076 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 2 to 4, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 4 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:13:00.204339 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 155119us
I20260709 12:13:00.199450 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:00.209682 11253 rpcz_store.cc:276] 0709 12:12:59.139354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.770114 (+630760us) service_pool.cc:224] Handling call
0709 12:13:00.138087 (+367973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.138661 (+   574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.177775 (+ 39114us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b5400069350. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:00.177994 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845382912}
I20260709 12:13:00.141366 13141 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:13:00.169275 11222 leader_election.cc:400] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:13:00.139858 13166 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:13:00.214083 11076 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:00.131963 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48711us
I20260709 12:13:00.146050 13412 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:00.141916 13325 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:00.237816 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:00.238045 11222 leader_election.cc:403] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:13:00.246594 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 77541.1us
W20260709 12:13:00.238466 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 169179us
I20260709 12:13:00.147696 13312 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:00.240170 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:00.249379 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:00.203233 13046 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:00.208235 13366 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:00.252161 11325 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:00.263098 11330 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:00.243582 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:00.269511 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:00.270011 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:00.272239 11182 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:00.273317 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:00.273546 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:00.273964 11182 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:00.275069 11180 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:00.276064 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:00.276882 11182 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:00.273967 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:00.278028 11933 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:00.279311 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:00.209481 13106 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:00.279313 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:00.279572 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:00.258401 13312 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:00.256763 13046 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:00.251037 13271 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:00.254530 13369 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:00.288517 11178 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:13:00.264248 11222 connection.cc:256] RPC call timeout handler was delayed by -1.203096953999875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
I20260709 12:13:00.284027 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:00.285017 11329 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:00.299551 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:00.299786 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:00.300247 11329 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:00.252837 13387 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
W20260709 12:13:00.301578 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:00.300240 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.285872 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:00.240423 13279 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:13:00.318020 13106 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:00.319392 13106 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:00.246662 13355 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:00.280414 11182 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:00.295037 13166 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:00.322108 13166 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:00.323160 13428 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:13:00.323088 13166 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:00.309021 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:00.327731 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 230) took 1082 ms. Trace:
W20260709 12:13:00.328656 13430 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.178s	user 0.000s	sys 0.002s
I20260709 12:13:00.260140 13289 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:13:00.315718 13412 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:00.328152 11328 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:00.330596 11176 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:00.240710 13443 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:00.287031 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:00.278218 12981 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:13:00.294039 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:00.275044 12923 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:13:00.290087 13115 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:13:00.315451 13419 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:13:00.296726 13437 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:00.331470 11175 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:00.332108 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45209.4us
I20260709 12:13:00.332922 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:00.372516 11177 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:00.375313 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:00.375944 11251 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.335083 11182 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:13:00.377697 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.310295 11325 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:00.379976 11256 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:00.278398 13350 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:00.313396 13431 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:13:00.328193 11255 rpcz_store.cc:276] 0709 12:12:59.245163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.775897 (+530734us) service_pool.cc:224] Handling call
0709 12:12:59.776187 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.776550 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.776577 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.777009 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.047933 (+270924us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:00.047979 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.048057 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.327477 (+279420us) raft_consensus.cc:1685] finished
0709 12:13:00.327505 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.327666 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20958,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":136451,"threads_started":1,"wal-append.queue_time_us":439167}
W20260709 12:13:00.339450 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55500us
I20260709 12:13:00.299472 13271 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:00.353497 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49454.1us
W20260709 12:13:00.317710 13426 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 12:13:00.316296 13418 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:13:00.368070 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.369812 13403 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.299647 13369 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:00.383561 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:00.385381 11250 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:00.295791 13401 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:13:00.390220 11256 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:00.413569 11173 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:13:00.392709 11177 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:00.416575 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:00.339633 13162 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.336921 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:00.393194 11251 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:00.398124 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:00.417717 11935 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:00.417749 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:00.418339 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:00.418808 11178 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:00.371186 13437 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.407840 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:00.408802 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:00.411989 11257 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:00.390311 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:00.393124 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.419788 11177 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:00.366637 13438 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:00.370888 13373 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 12:13:00.830291 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:00.824558 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.448341 11179 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:00.542279 11178 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:00.555740 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:00.446393 11176 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:00.516211 11933 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:00.879664 11933 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:13:00.585565 11296 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.9.198.254:36219, user_credentials={real_user=slave}} blocked reactor thread for 56178.8us
I20260709 12:13:00.614351 11257 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:00.431914 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:00.533825 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 273545us
I20260709 12:13:00.414872 13413 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.446372 11181 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:00.897096 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:00.883302 11175 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.512s	user 0.015s	sys 0.010s
I20260709 12:13:00.399835 13271 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:00.494411 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:00.539839 11328 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:00.898272 11175 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:00.882819 11257 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:13:00.394008 13115 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:00.555682 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 219377us
W20260709 12:13:00.898124 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.898733 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:00.497862 13425 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
W20260709 12:13:00.503933 13426 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:13:00.886484 12923 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:00.516934 13418 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:13:00.398406 13358 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.506s
I20260709 12:13:00.672001 11926 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:00.416339 13325 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:00.461696 13345 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:00.447099 13355 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:00.479610 13374 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:00.939076 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:00.965240 11931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.011s	sys 0.034s
W20260709 12:13:00.964998 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66383.8us
W20260709 12:13:00.525774 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 202527us
I20260709 12:13:00.460903 13162 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:00.939198 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 548927us
I20260709 12:13:00.446179 13279 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:00.969166 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.447824 13350 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:00.555115 13434 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:13:00.497596 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:00.412624 13289 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.556s
I20260709 12:13:00.686230 13290 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:00.431478 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:00.401993 13366 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:00.430488 13437 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:00.887511 13361 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.462747 13335 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:00.422830 13403 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:00.905009 13443 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.585870 13369 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:00.963716 13436 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:01.246297 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 349256us
W20260709 12:13:01.047820 11931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.011s	sys 0.034s
W20260709 12:13:01.350065 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.298321 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.354835 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.350710 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:01.357519 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:00.989693 12923 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.358898 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.359208 11251 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:01.359740 12923 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:00.973124 13225 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.363454 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:01.347493 13106 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.028s	user 0.011s	sys 0.001s
I20260709 12:13:01.362622 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:01.186955 13393 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.990s	user 0.009s	sys 0.000s
W20260709 12:13:01.246683 11928 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.002s	sys 0.019s
I20260709 12:13:01.137837 13451 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.683905 13371 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.319289 13272 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.184s	user 0.012s	sys 0.000s
W20260709 12:13:01.169649 13422 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.021s
I20260709 12:13:01.417910 11174 raft_consensus.cc:2410] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:13:01.164894 13374 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:00.917725 13413 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.360997 13368 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.304251 13355 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.361681 13335 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:01.431316 13335 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:01.432235 13209 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.187s	user 0.011s	sys 0.000s
I20260709 12:13:01.432788 13209 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.433182 13335 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:00.967238 13115 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.433933 13473 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:01.434585 13473 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.435377 13413 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:01.435828 13115 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.436146 13413 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:01.436568 13115 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:01.436879 13473 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.377646 13369 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:01.419047 11928 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.003s	sys 0.021s
I20260709 12:13:01.426585 13371 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.441799 13374 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.442571 13371 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:01.365957 13290 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.444016 13374 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.444572 13374 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.444815 13290 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.445031 13374 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.370625 11927 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:01.453245 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 505) took 1042 ms. Trace:
I20260709 12:13:01.453696 11927 rpcz_store.cc:276] 0709 12:13:00.410333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.459816 (+ 49483us) service_pool.cc:224] Handling call
0709 12:13:01.453155 (+993339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:01.370445 11248 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:01.455039 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1366) took 1070 ms. Trace:
I20260709 12:13:01.455448 11248 rpcz_store.cc:276] 0709 12:13:00.384332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.389588 (+  5256us) service_pool.cc:224] Handling call
0709 12:13:01.454971 (+1065383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944}
I20260709 12:13:01.445366 13290 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.456875 13290 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.424714 13451 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.459362 13451 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:01.245113 13271 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:01.429142 13422 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.000s	sys 0.024s
I20260709 12:13:01.462055 11249 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:13:01.428969 13272 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.464195 13272 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.422683 11322 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:01.470969 13320 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.471581 13368 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.472317 13290 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.473269 13290 raft_consensus.cc:2764] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:13:01.473804 13368 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:01.474079 13162 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.032377 13345 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.350904 13443 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:01.376744 13289 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.452103 13350 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.423892 13393 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:01.373775 13225 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.502586 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1804) took 1233 ms. Trace:
I20260709 12:13:01.423348 13446 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:13:01.449015 13373 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.481521 13355 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:01.360539 13361 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.491572 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:01.492385 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1200) took 1195 ms. Trace:
I20260709 12:13:01.523144 11256 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:01.492539 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:01.492743 13369 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.493196 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 503) took 1092 ms. Trace:
I20260709 12:13:01.497176 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1626) took 1093 ms. Trace:
I20260709 12:13:01.525887 11251 rpcz_store.cc:276] 0709 12:13:00.404237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.431543 (+ 27306us) service_pool.cc:224] Handling call
0709 12:13:01.496989 (+1065446us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b540010b154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:01.497108 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085899008}
I20260709 12:13:01.372453 13279 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.500545 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 324) took 1583 ms. Trace:
I20260709 12:13:01.501292 13345 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:01.503021 13454 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:01.509991 13335 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.509802 13321 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.503206 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.511580 13449 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.009s	user 0.004s	sys 0.006s
I20260709 12:13:01.512957 11174 rpcz_store.cc:276] 0709 12:13:00.269339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.269786 (+   447us) service_pool.cc:224] Handling call
0709 12:13:01.502387 (+1232601us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400364a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:01.502525 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2637315456}
I20260709 12:13:01.527912 11326 rpcz_store.cc:276] 0709 12:12:59.917215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.919697 (+  2482us) service_pool.cc:224] Handling call
0709 12:12:59.920050 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.920290 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.920319 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.920785 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.096602 (+175817us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:00.096637 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.096698 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.500333 (+1403635us) raft_consensus.cc:1685] finished
0709 12:13:01.500352 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.500484 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":175464,"threads_started":1,"wal-append.queue_time_us":347802}
I20260709 12:13:01.524153 11931 rpcz_store.cc:276] 0709 12:13:00.401185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.401929 (+   744us) service_pool.cc:224] Handling call
0709 12:13:00.402318 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.402833 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.402876 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.403457 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.363086 (+959629us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:01.363122 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.363194 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.492921 (+129727us) raft_consensus.cc:1685] finished
0709 12:13:01.492948 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.493123 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":280943,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":74,"thread_start_us":959302,"threads_started":1,"wal-append.queue_time_us":1084669}
I20260709 12:13:01.532243 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1625) took 1130 ms. Trace:
I20260709 12:13:01.534471 13321 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.558218 11256 rpcz_store.cc:276] 0709 12:13:00.401764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.428265 (+ 26501us) service_pool.cc:224] Handling call
0709 12:13:01.491409 (+1063144us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b5800196898. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:01.532157 (+ 40748us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62714752}
I20260709 12:13:01.522984 11324 rpcz_store.cc:276] 0709 12:13:00.296778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.297263 (+   485us) service_pool.cc:224] Handling call
0709 12:13:00.348641 (+ 51378us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:00.348926 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.366239 (+ 17313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.366284 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.395331 (+ 29047us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.899442 (+504111us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:00.899480 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.899546 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.491975 (+592429us) raft_consensus.cc:1685] finished
0709 12:13:01.492003 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.492305 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16894,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":15098,"spinlock_wait_cycles":48494336,"thread_start_us":439716,"threads_started":1,"wal-append.queue_time_us":1069489}
I20260709 12:13:01.536249 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.560725 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.436586 13413 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:01.449934 13447 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:01.567783 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.554719 11930 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.589885 13046 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:01.594274 13443 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:01.598034 13271 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.598286 13413 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.598857 13271 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:01.599514 13271 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.599669 13413 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.600231 13271 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:01.600536 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.601123 13271 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:01.603186 13279 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:01.603956 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.605924 13046 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:01.606263 13361 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.606894 13447 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:01.607978 13447 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.611474 11929 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 5. Ops: []
I20260709 12:13:01.613688 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 459) took 1237 ms. Trace:
I20260709 12:13:01.614391 13355 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:01.535149 11146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.015s	sys 0.061s
I20260709 12:13:01.539776 13454 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:01.615182 11146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.015s	sys 0.061s
I20260709 12:13:01.615321 13454 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:01.512216 13403 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.611967 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1368) took 1226 ms. Trace:
I20260709 12:13:01.618949 11322 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:01.617930 11255 rpcz_store.cc:276] 0709 12:13:00.385159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.394815 (+  9656us) service_pool.cc:224] Handling call
0709 12:13:00.563611 (+168796us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b5800196898. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:00.875598 (+311987us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:00.896635 (+ 21037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.896677 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:00.904810 (+  8133us) log.cc:844] Serialized 52 byte log entry
0709 12:13:01.362016 (+457206us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:01.362043 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.362090 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.611614 (+249524us) raft_consensus.cc:1685] finished
0709 12:13:01.611635 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.611900 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":19048,"spinlock_wait_cycles":20837248,"thread_start_us":319620,"threads_started":1,"wal-append.queue_time_us":619486}
I20260709 12:13:01.619398 13373 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.612799 13209 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:01.601519 13271 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:01.619997 13373 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.623769 11182 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:01.624365 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:01.614105 11929 rpcz_store.cc:276] 0709 12:13:00.376514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.393944 (+ 17430us) service_pool.cc:224] Handling call
0709 12:13:01.611028 (+1217084us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:01.611262 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.613385 (+  2123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.613604 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14704896}
I20260709 12:13:01.633517 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:01.543998 13335 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.627169 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:01.634330 11176 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:01.523161 13289 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.631305 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:01.637046 11179 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:13:01.631078 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:01.638648 11182 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:01.626644 13279 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.628247 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:01.626536 13355 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.656598 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:01.655925 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1360) took 1280 ms. Trace:
I20260709 12:13:01.659338 11176 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:01.660578 11331 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:01.661032 11331 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:01.663466 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:01.664458 11329 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:01.657537 11931 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
W20260709 12:13:01.621588 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1197) took 1338 ms (client timeout 1694 ms). Trace:
I20260709 12:13:01.679700 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:01.702541 11253 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:01.646373 13438 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.023s	sys 0.073s
I20260709 12:13:01.705103 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:01.706017 11324 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:01.550282 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.620990 13271 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.661755 11331 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.710747 11331 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:01.712673 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 362064us
I20260709 12:13:01.700100 11932 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:01.715976 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 504) took 1306 ms. Trace:
I20260709 12:13:01.716445 11932 rpcz_store.cc:276] 0709 12:13:00.409965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.410784 (+   819us) service_pool.cc:224] Handling call
0709 12:13:01.687831 (+1277047us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:01.699927 (+ 12096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.715608 (+ 15681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.715899 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85293440}
W20260709 12:13:01.699681 11175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1806) took 1428 ms (client timeout 1837 ms). Trace:
W20260709 12:13:01.717885 11175 rpcz_store.cc:269] 0709 12:13:00.271132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.273338 (+  2206us) service_pool.cc:224] Handling call
0709 12:13:01.687041 (+1413703us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b5400364f54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:01.699591 (+ 12550us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131834240}
I20260709 12:13:01.719929 11175 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:13:01.708781 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 345373us
I20260709 12:13:01.723764 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:01.705019 13445 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.150s	user 0.003s	sys 0.003s
I20260709 12:13:01.735174 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:01.710271 13438 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.023s	sys 0.074s
I20260709 12:13:01.709820 13446 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.749857 13438 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:01.750039 13446 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.665836 13355 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.750986 13438 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:01.752518 13446 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.752826 13355 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:01.730808 13444 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.207s	user 0.006s	sys 0.000s
I20260709 12:13:01.666309 13448 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:01.753989 13448 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.754925 13141 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:01.755435 13141 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:01.756417 11177 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:01.757670 13361 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.758188 13361 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.760334 13361 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.745180 13403 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.761588 13403 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.762321 13335 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.763574 13403 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.763803 13401 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.764154 13403 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.764287 13401 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:01.766350 13401 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.767191 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 326) took 1778 ms. Trace:
I20260709 12:13:01.767580 11327 rpcz_store.cc:276] 0709 12:12:59.988370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.050454 (+ 62084us) service_pool.cc:224] Handling call
0709 12:13:00.050837 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.155727 (+104890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.155779 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.167380 (+ 11601us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.336882 (+169502us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:00.336922 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.336995 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.682823 (+1345828us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:01.766900 (+ 84077us) raft_consensus.cc:1685] finished
0709 12:13:01.766925 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.767132 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18685,"raft.queue_time_us":152331,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":34295,"spinlock_wait_cycles":102260480,"thread_start_us":219821,"threads_started":2,"wal-append.queue_time_us":195331}
I20260709 12:13:01.768810 11930 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:01.725692 13373 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.733428 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:01.799715 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 66362.6us
I20260709 12:13:01.748805 13468 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:13:01.665889 13472 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:01.746447 11182 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:01.790534 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:01.651064 13279 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.790905 11327 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:01.820452 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:01.792368 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44581.7us
I20260709 12:13:01.823200 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.746326 11175 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:01.803673 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 444915us
W20260709 12:13:01.815724 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 90606.4us
I20260709 12:13:01.833896 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:01.832520 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.798807 13470 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.791529 13115 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:01.844483 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:01.702062 11322 rpcz_store.cc:269] 0709 12:13:00.283005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.289351 (+  6346us) service_pool.cc:224] Handling call
0709 12:13:01.621379 (+1332028us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b54000c6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:01.621515 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2292441344}
I20260709 12:13:01.838734 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:01.799098 13374 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:01.816720 13290 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:01.809949 13355 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:01.847289 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:01.837278 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:01.833282 13446 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:01.839469 13166 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.284s	user 0.015s	sys 0.032s
I20260709 12:13:01.832340 13472 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:01.810351 12923 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:13:01.836328 13209 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.901145 11178 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:01.902096 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54888.4us
W20260709 12:13:01.903252 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 64954us
W20260709 12:13:01.938410 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 314245us
I20260709 12:13:01.870434 13401 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.857858 13335 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:01.832930 13271 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:01.915110 13358 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:13:01.918445 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.921344 13475 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.093s	user 0.000s	sys 0.003s
I20260709 12:13:01.830227 13373 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:13:01.926981 11255 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:01.927858 13479 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:01.929337 13478 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:01.931172 13476 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:13:01.903944 11932 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:13:01.997354 11932 pending_rounds.cc:85] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 6
I20260709 12:13:01.877478 13290 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:01.901779 13225 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:01.943071 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.013s	sys 0.028s
W20260709 12:13:02.122651 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.013s	sys 0.028s
I20260709 12:13:01.863963 13466 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.943395 11331 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:13:01.948020 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:01.949813 11251 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:02.123664 11331 pending_rounds.cc:85] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 4
I20260709 12:13:02.124015 11331 pending_rounds.cc:107] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260709 12:13:01.905184 13166 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.016s	sys 0.032s
I20260709 12:13:01.816605 13437 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:02.146039 11180 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.030s	sys 0.038s
I20260709 12:13:01.958177 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:01.980772 13443 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.983673 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:01.985337 11930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 462) took 1607 ms (client timeout 1606 ms). Trace:
I20260709 12:13:01.989194 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1361) took 1610 ms. Trace:
W20260709 12:13:02.157490 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 209521us
I20260709 12:13:02.157737 11249 rpcz_store.cc:276] 0709 12:13:00.378618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.379728 (+  1110us) service_pool.cc:224] Handling call
0709 12:13:00.402523 (+ 22795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.462470 (+1059947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.988898 (+526428us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400379e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:01.989125 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2436116352}
I20260709 12:13:01.988953 13115 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:01.990551 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:01.994743 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:01.997696 11932 pending_rounds.cc:107] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:13:01.942184 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:01.883286 13403 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:01.942484 11253 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:01.910279 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 330) took 1757 ms. Trace:
W20260709 12:13:02.051381 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.015s	sys 0.047s
I20260709 12:13:02.032709 13290 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:01.910612 13355 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.867853 13470 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:01.954254 13209 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:01.992462 13401 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.054466 13289 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:01.997402 11929 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:01.992568 13446 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:02.118911 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 117337us
I20260709 12:13:02.296576 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:01.910848 13272 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:01.910542 13374 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.128s
I20260709 12:13:01.866169 13361 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:13:01.877826 13471 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.202s
I20260709 12:13:01.910704 13387 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:01.836577 13465 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.128s
I20260709 12:13:02.000149 13464 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:02.124315 11331 raft_consensus.cc:2889] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:02.064888 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:02.065544 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.147401 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:02.146624 11180 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.030s	sys 0.038s
I20260709 12:13:02.153578 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.303474 13272 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.172135 11932 raft_consensus.cc:2889] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:02.172289 11178 raft_consensus.cc:606] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:02.172737 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1358) took 1798 ms (client timeout 1694 ms). Trace:
W20260709 12:13:02.359407 11253 rpcz_store.cc:269] 0709 12:13:00.374423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.389218 (+ 14795us) service_pool.cc:224] Handling call
0709 12:13:01.679452 (+1290234us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b5800196898. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:02.172646 (+493194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13814144}
I20260709 12:13:02.173702 11175 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:02.174101 11250 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:02.232712 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:02.361304 11935 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:02.119801 13412 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:02.175714 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.015s	sys 0.048s
W20260709 12:13:02.142069 13345 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.007s
I20260709 12:13:02.267298 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:02.363206 11931 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:13:02.234789 13454 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.003s
I20260709 12:13:02.287041 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1648) took 1854 ms. Trace:
I20260709 12:13:02.369433 11173 rpcz_store.cc:276] 0709 12:13:00.434627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.460034 (+ 25407us) service_pool.cc:224] Handling call
0709 12:13:00.480539 (+ 20505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.516328 (+ 35789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.516369 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.898516 (+382147us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.425899 (+527383us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:01.425959 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.426051 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.286751 (+860700us) raft_consensus.cc:1685] finished
0709 12:13:02.286784 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.286976 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":323234,"raft.queue_time_us":8561,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":20431,"thread_start_us":383395,"threads_started":1,"wal-append.queue_time_us":663939}
I20260709 12:13:02.159319 13393 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.129s
I20260709 12:13:02.160619 13225 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 12:13:02.171010 13413 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:02.306077 11324 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.599s	user 0.007s	sys 0.004s
I20260709 12:13:02.389253 11324 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:02.307993 11256 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:02.231957 13481 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.168s
I20260709 12:13:02.154312 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.374868 13345 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.391100 13481 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:02.391348 11251 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:02.391610 13481 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:02.375389 13412 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.393258 13345 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:02.316123 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 378401us
I20260709 12:13:02.307619 13477 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.360255 11257 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:02.239532 13403 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:02.401198 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 429272us
I20260709 12:13:02.369427 13166 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:02.321669 13488 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:02.155555 13466 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:02.317669 13448 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:02.379297 13454 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.384514 13162 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 12:13:02.157531 11930 rpcz_store.cc:269] 0709 12:13:00.377487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.394074 (+ 16587us) service_pool.cc:224] Handling call
0709 12:13:01.492379 (+1098305us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:01.985012 (+492633us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b540069fad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:01.985208 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48121856}
I20260709 12:13:02.393011 13481 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:02.327767 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 180481us
I20260709 12:13:02.330241 13387 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:02.364158 13272 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:02.159109 13484 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.269s
I20260709 12:13:02.368853 13279 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.142903 13472 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:02.443006 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 284641us
I20260709 12:13:02.394712 13468 raft_consensus.cc:2764] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:13:02.394991 13483 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:02.509935 13505 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.509981 13483 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:02.510604 13505 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.511413 13483 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.512948 13505 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.429164 13166 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:02.422045 12923 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.358388 13464 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:02.539552 13464 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.522495 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.540300 11253 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:02.523233 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.175563 11323 rpcz_store.cc:276] 0709 12:13:00.152623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.191677 (+ 39054us) service_pool.cc:224] Handling call
0709 12:13:00.192104 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.240209 (+ 48105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.245840 (+  5631us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.254991 (+  9151us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.369854 (+114863us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:00.369909 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.369989 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.686902 (+1316913us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:01.863684 (+176782us) raft_consensus.cc:1685] finished
0709 12:13:01.863717 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.907532 (+ 43815us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26240,"raft.queue_time_us":47216,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5232,"spinlock_wait_cycles":3243776,"thread_start_us":114504,"threads_started":1,"wal-append.queue_time_us":299123}
I20260709 12:13:02.403855 13477 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.542955 13272 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:02.544332 13470 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.523890 13466 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:02.525107 13485 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:13:02.527860 13387 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.585045 13046 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:02.585688 13046 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:02.528579 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1202) took 1172 ms. Trace:
I20260709 12:13:02.587432 11328 rpcz_store.cc:276] 0709 12:13:01.355999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.356139 (+   140us) service_pool.cc:224] Handling call
0709 12:13:01.356496 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.653043 (+296547us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:01.653081 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.653556 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:13:02.159176 (+505620us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:02.159218 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.159300 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.528336 (+369036us) raft_consensus.cc:1685] finished
0709 12:13:02.528356 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.528515 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4349,"raft.queue_time_us":116230,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":726380,"threads_started":2,"wal-append.queue_time_us":709936}
I20260709 12:13:02.521660 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.591502 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.427873 13412 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.398572 13413 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.570254 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:02.612880 11255 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:02.545917 13272 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.574221 13321 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:02.636229 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.023s	sys 0.059s
W20260709 12:13:02.580384 13501 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:13:02.580260 12923 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.461215 13446 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.573333 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:02.589812 11250 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:13:02.546589 13115 raft_consensus.cc:993] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:13:02.592361 11257 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
W20260709 12:13:02.614482 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.620076 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 158473us
I20260709 12:13:02.545928 13209 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:02.427666 13488 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:02.638249 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.024s	sys 0.059s
I20260709 12:13:02.571856 13477 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:02.650213 11253 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:02.588024 13437 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:02.654102 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:02.546053 13290 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:02.668126 13115 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:13:02.668524 13504 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:13:02.668766 13115 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:02.658864 13502 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:02.672161 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 335) took 1012 ms. Trace:
I20260709 12:13:02.672278 13502 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.672925 13502 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:02.678903 13512 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:02.679388 13512 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:02.680207 13512 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.681854 13166 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:02.683019 12923 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.586872 13046 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:02.672590 11325 rpcz_store.cc:276] 0709 12:13:01.659795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.659931 (+   136us) service_pool.cc:224] Handling call
0709 12:13:01.660391 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.660885 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.660932 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.661561 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.976524 (+314963us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b1800358cc0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:02.313531 (+337007us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:02.313572 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.313684 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.670014 (+356330us) raft_consensus.cc:1685] finished
0709 12:13:02.670037 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.672084 (+  2047us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5355,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":65571,"spinlock_wait_cycles":25909632,"thread_start_us":336752,"threads_started":1,"wal-append.queue_time_us":760390}
I20260709 12:13:02.685670 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 339) took 1019 ms. Trace:
I20260709 12:13:02.655469 13482 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:02.681407 13512 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:02.656942 13279 raft_consensus.cc:1081] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:13:02.655454 13401 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:02.747263 11181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.011s
W20260709 12:13:02.747872 11181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.000s	sys 0.011s
W20260709 12:13:02.711725 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 311706us
W20260709 12:13:02.712495 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 291859us
I20260709 12:13:02.690313 11329 rpcz_store.cc:276] 0709 12:13:01.666463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.666610 (+   147us) service_pool.cc:224] Handling call
0709 12:13:01.667064 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.667630 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.667676 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.668300 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:13:02.120193 (+451893us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:02.491750 (+371557us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.685345 (+193595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.685418 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.685459 (+    41us) raft_consensus.cc:1685] finished
0709 12:13:02.685470 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.685595 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":193232,"prepare.queue_time_us":264430,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":127,"spinlock_wait_cycles":38446848,"thread_start_us":645304,"threads_started":2,"wal-append.queue_time_us":638678}
I20260709 12:13:02.723371 13115 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.684247 13393 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.761687 13393 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.764662 13393 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.750820 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 285) took 1125 ms. Trace:
I20260709 12:13:02.741077 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 340) took 1072 ms. Trace:
I20260709 12:13:02.742295 11255 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:02.641798 13173 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:13:02.765600 11181 rpcz_store.cc:276] 0709 12:13:01.625235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.625398 (+   163us) service_pool.cc:224] Handling call
0709 12:13:01.625776 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.626058 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:02.141906 (+515848us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.750170 (+608264us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.750348 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:02.750423 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:02.750490 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.750583 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.750745 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":702105,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"thread_start_us":1070062,"threads_started":2}
I20260709 12:13:02.765769 11330 rpcz_store.cc:276] 0709 12:13:01.668783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.669049 (+   266us) service_pool.cc:224] Handling call
0709 12:13:01.669360 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.700244 (+ 30884us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:02.121026 (+420782us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.740695 (+619669us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:02.740752 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:02.740802 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.740880 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.741019 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83056,"prepare.queue_time_us":683912,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"thread_start_us":885890,"threads_started":2}
I20260709 12:13:02.765332 13393 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:02.744972 13271 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.748s	user 0.006s	sys 0.006s
I20260709 12:13:02.783416 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.785482 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:02.787344 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.657377 13446 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:02.639717 13479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.018s	sys 0.005s
I20260709 12:13:02.740554 13482 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:02.750226 13506 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:02.774911 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:02.791251 13106 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:02.827347 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:02.807932 11930 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:02.795890 13046 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:02.810083 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:02.810537 13413 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:13:02.811121 13464 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:13:02.811585 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.812817 13477 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:02.812925 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.745283 13476 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:13:02.817442 13479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.018s	sys 0.005s
I20260709 12:13:02.793857 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 461) took 2375 ms. Trace:
I20260709 12:13:02.826136 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.817857 13454 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:02.829906 11182 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:02.838078 11932 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:13:02.940500 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.793450 13393 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:02.851608 11175 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:02.955575 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.855005 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.932898 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:02.935971 13290 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:02.818302 13469 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.937407 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:02.937786 11176 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:02.937701 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.013024 11329 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:02.826359 13403 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:02.828271 13506 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:02.942770 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 342) took 1238 ms. Trace:
I20260709 12:13:02.944187 11327 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:02.949582 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:02.850167 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 75293.2us
I20260709 12:13:02.797947 13438 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:13:03.011039 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:02.935001 13477 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.013059 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:03.012188 13454 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.355051 13454 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.355984 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.356699 11329 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:03.016494 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.357713 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.358592 11330 raft_consensus.cc:2410] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:03.359308 11328 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:03.015965 13271 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:03.363751 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.364069 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:03.360926 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.365497 11327 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:03.129859 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:02.937063 11248 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:13:02.934428 13466 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:03.362812 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.019s	sys 0.020s
I20260709 12:13:03.373059 11176 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:03.373400 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.019s	sys 0.020s
W20260709 12:13:03.373742 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 825941us
I20260709 12:13:03.376539 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.376734 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.377297 11257 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:03.365254 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:03.378572 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.379412 11256 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:03.377204 11253 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:13:03.379263 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:03.377197 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.380988 11250 raft_consensus.cc:2410] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:03.381510 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:03.382979 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.354161 11326 rpcz_store.cc:276] 0709 12:13:01.703848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.708338 (+  4490us) service_pool.cc:224] Handling call
0709 12:13:01.708710 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.708959 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:02.284288 (+575329us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.942322 (+658034us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:02.942395 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:02.942460 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.942546 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.942697 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":264283,"prepare.queue_time_us":675435,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":335694,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":39685,"thread_start_us":873564,"threads_started":3,"wal-append.queue_time_us":258058}
I20260709 12:13:03.384138 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.359653 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:03.357355 13454 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:03.146059 13514 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
I20260709 12:13:03.402806 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.299978 13446 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:03.218597 13523 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.220930 13393 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:03.385502 13271 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.109760 13290 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:03.399950 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.398669 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:03.030205 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.401894 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:03.404410 11250 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:03.408172 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.011291 13279 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.407732 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:03.121464 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:03.425666 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:03.429072 11256 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:03.425441 11253 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:03.412873 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.385854 13526 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.392474 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:03.426903 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:03.393857 13046 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.540s	user 0.004s	sys 0.008s
I20260709 12:13:03.421063 11328 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:03.425153 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.411440 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:03.386487 13466 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.394012 13368 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:03.438500 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:03.396080 13477 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:03.411666 13453 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:03.031219 13469 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.441581 11253 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:03.442909 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:03.443799 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:03.673800 11174 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:03.428781 11322 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:03.440506 13466 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:03.675845 11182 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:03.444566 13368 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:03.677894 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.684731 11326 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:03.474423 13046 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.671131 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:03.678943 13454 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:03.675438 13403 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.682833 13535 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:03.425313 13290 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:03.677583 13438 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.091840 13482 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:13:03.696766 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.008s	sys 0.030s
W20260709 12:13:03.918789 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.009s	sys 0.031s
I20260709 12:13:03.679600 13522 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.680897 13477 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:03.396569 13506 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.421989 13523 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:03.436486 13279 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:03.708949 11325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.014s	sys 0.027s
W20260709 12:13:03.946342 11325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.014s	sys 0.027s
W20260709 12:13:03.721705 11175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.005s	sys 0.038s
I20260709 12:13:03.692696 13335 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.694936 13401 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:03.442857 13526 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.723150 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:03.416301 13530 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:03.427564 13393 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.793527 11256 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:03.811056 11328 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 1.
I20260709 12:13:03.749977 13534 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:03.811049 13472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.050s	sys 0.082s
I20260709 12:13:03.743968 13518 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:13:03.772125 13483 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.020s	sys 0.061s
I20260709 12:13:02.937399 11928 rpcz_store.cc:276] 0709 12:13:00.377164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.381779 (+  4615us) service_pool.cc:224] Handling call
0709 12:13:00.430421 (+ 48642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.438328 (+1007907us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:01.438379 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.439098 (+   719us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.836208 (+397110us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:01.858696 (+ 22488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.866039 (+  7343us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.722575 (+856536us) raft_consensus.cc:1685] finished
0709 12:13:02.722607 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.752962 (+ 30355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":442096,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":30746,"thread_start_us":1243796,"threads_started":2,"wal-append.queue_time_us":882974}
W20260709 12:13:03.993278 13528 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:13:03.833933 13454 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.921792 13438 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.833386 13046 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:03.911731 13535 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:03.924036 13446 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:13:03.911105 13403 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:03.939347 13368 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:03.924435 13512 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.179s	user 0.013s	sys 0.000s
W20260709 12:13:03.950234 13519 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:13:03.948249 13453 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:03.987112 13487 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.693s	user 0.003s	sys 0.001s
I20260709 12:13:03.961143 13543 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.926947 13477 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:03.999356 13472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.145s	user 0.051s	sys 0.083s
I20260709 12:13:03.983096 13530 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:03.985651 13469 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:04.007129 13483 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.020s	sys 0.062s
I20260709 12:13:03.989261 13523 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.003708 13518 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.925786 13522 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.016264 13454 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:04.042204 13512 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.049506 13543 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.062922 13469 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.066448 13512 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.066830 13469 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.067752 13535 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.068702 13512 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:04.060657 13502 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.387s	user 0.015s	sys 0.000s
I20260709 12:13:04.060036 13453 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.069919 13502 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.065469 13518 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:04.073017 13502 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:04.058260 13524 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
I20260709 12:13:04.074726 13469 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:04.005103 13534 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:04.070703 13522 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:04.059243 13520 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 12:13:04.079668 13534 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:04.062280 13537 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.153s	user 0.000s	sys 0.002s
I20260709 12:13:04.061883 13477 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:04.052105 13531 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
I20260709 12:13:04.086059 13543 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.065186 13526 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:04.083035 13530 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.088976 13530 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:04.091964 13518 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.104223 13466 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:04.105381 13538 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 12:13:04.109560 13488 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:03.947131 11175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.006s	sys 0.038s
W20260709 12:13:04.253880 13478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.183s	user 0.015s	sys 0.015s
W20260709 12:13:04.287640 13478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.239s	user 0.016s	sys 0.016s
I20260709 12:13:04.290231 13521 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.295591 13466 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:04.144760 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1835) took 1148 ms. Trace:
W20260709 12:13:03.772091 11180 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.012s	sys 0.070s
W20260709 12:13:03.886241 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.148095 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1826) took 1333 ms. Trace:
W20260709 12:13:03.946801 11223 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.009s	sys 0.009s
W20260709 12:13:04.154508 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 452928us
I20260709 12:13:04.158000 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 242) took 1641 ms. Trace:
W20260709 12:13:04.167152 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.300450 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.169902 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1815) took 2516 ms. Trace:
I20260709 12:13:04.173300 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 243) took 1652 ms. Trace:
I20260709 12:13:04.174691 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1837) took 1232 ms. Trace:
I20260709 12:13:04.174903 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:03.909524 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.057s	sys 0.065s
I20260709 12:13:04.178521 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1204) took 1808 ms. Trace:
I20260709 12:13:03.993685 11323 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:04.187414 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 390) took 2533 ms. Trace:
I20260709 12:13:04.187875 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 469) took 1883 ms. Trace:
I20260709 12:13:04.190006 11330 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:04.201207 13046 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:04.203214 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 1.90543e+06us
W20260709 12:13:04.316165 11223 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.010s	sys 0.009s
I20260709 12:13:04.208259 13279 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:04.316948 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 915397us
I20260709 12:13:03.947606 13271 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:04.299105 11177 rpcz_store.cc:276] 0709 12:13:02.814217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.814368 (+   151us) service_pool.cc:224] Handling call
0709 12:13:02.814670 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.815019 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.815047 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:02.815460 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.113831 (+298371us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:03.404160 (+290329us) log.cc:844] Serialized 14 byte log entry
0709 12:13:03.404341 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.404405 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.147547 (+743142us) raft_consensus.cc:1685] finished
0709 12:13:04.147589 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.147922 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":289354,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":278493,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":22493,"thread_start_us":271136,"threads_started":1,"wal-append.queue_time_us":936170}
I20260709 12:13:04.211318 13290 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.209007 13141 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.320396 13141 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:04.300024 11251 rpcz_store.cc:276] 0709 12:13:02.516899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.519062 (+  2163us) service_pool.cc:224] Handling call
0709 12:13:02.519395 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.529041 (+  9646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.529073 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:02.529612 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 12:13:02.741839 (+212227us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:02.741895 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.741996 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.157364 (+1415368us) raft_consensus.cc:1685] finished
0709 12:13:04.157405 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.157757 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51710,"raft.queue_time_us":152,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":211770,"threads_started":1,"wal-append.queue_time_us":1140793}
I20260709 12:13:03.960752 13335 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.314535 11926 rpcz_store.cc:276] 0709 12:13:02.304524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.308341 (+  3817us) service_pool.cc:224] Handling call
0709 12:13:02.386935 (+ 78594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:02.541888 (+154953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.541925 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:02.541971 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:02.542009 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.187320 (+1645311us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540041b754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:04.187415 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.187695 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335745792}
I20260709 12:13:04.328789 13335 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.314944 11929 rpcz_store.cc:276] 0709 12:13:01.654076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.667177 (+ 13101us) service_pool.cc:224] Handling call
0709 12:13:01.746425 (+ 79248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:02.295286 (+548861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.186579 (+1891293us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54003b5454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:04.187052 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380002688}
I20260709 12:13:04.301913 11248 rpcz_store.cc:276] 0709 12:13:02.520888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.522958 (+  2070us) service_pool.cc:224] Handling call
0709 12:13:04.172750 (+1649792us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400068e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:04.173051 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719976704}
I20260709 12:13:03.981429 13401 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.221846 13403 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:01.658450 11254 rpcz_store.cc:276] 0709 12:13:00.375092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.378331 (+  3239us) service_pool.cc:224] Handling call
0709 12:13:00.378620 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.378825 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:00.839873 (+461048us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.655203 (+815330us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.655439 (+   236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:01.655505 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:01.655571 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.655674 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.655845 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":301354,"prepare.queue_time_us":49280,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":12369,"raft.queue_time_us":377009,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":32935,"thread_start_us":941509,"threads_started":3,"wal-append.queue_time_us":763876}
I20260709 12:13:03.985037 13393 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.225342 13438 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:04.116822 13532 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.004s
W20260709 12:13:04.114780 13542 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:13:04.259696 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 303153us
W20260709 12:13:04.436223 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70531.2us
I20260709 12:13:04.259490 13567 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.269848 13560 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:04.270598 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4094575960000384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
I20260709 12:13:04.278048 13559 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:04.280980 13518 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:04.298904 11180 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.806s	user 0.012s	sys 0.071s
I20260709 12:13:04.307447 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:04.302177 11324 rpcz_store.cc:276] 0709 12:13:02.941592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.941748 (+   156us) service_pool.cc:224] Handling call
0709 12:13:02.990422 (+ 48674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.016370 (+ 25948us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:13:03.016419 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.361521 (+345102us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.918713 (+557192us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:03.918775 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.918879 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.174007 (+255128us) raft_consensus.cc:1685] finished
0709 12:13:04.174046 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.174365 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22491,"raft.queue_time_us":21627,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":15066,"thread_start_us":493485,"threads_started":1,"wal-append.queue_time_us":619573}
I20260709 12:13:04.307871 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.291785 13521 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.310384 11331 rpcz_store.cc:276] 0709 12:13:02.370177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.370546 (+   369us) service_pool.cc:224] Handling call
0709 12:13:02.371077 (+   531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.371462 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.371502 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:02.394013 (+ 22511us) log.cc:844] Serialized 564 byte log entry
0709 12:13:02.520538 (+126525us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:02.520575 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.520649 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.177842 (+1657193us) raft_consensus.cc:1685] finished
0709 12:13:04.177891 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.178286 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":126216,"threads_started":1,"wal-append.queue_time_us":1292666}
I20260709 12:13:04.297544 11181 rpcz_store.cc:276] 0709 12:13:02.996265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.001015 (+  4750us) service_pool.cc:224] Handling call
0709 12:13:03.091682 (+ 90667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.092055 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.092093 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.373648 (+281555us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.833837 (+460189us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:03.833902 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.834005 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.143614 (+309609us) raft_consensus.cc:1685] finished
0709 12:13:04.143721 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.144296 (+   575us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126330,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":439340,"threads_started":1,"wal-append.queue_time_us":622649}
I20260709 12:13:04.342900 13374 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:04.341584 13401 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:04.344915 13393 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.439316 13279 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:04.343365 13355 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:04.436734 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 119660us
I20260709 12:13:04.465760 13521 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:04.440891 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1834) took 1329 ms. Trace:
I20260709 12:13:04.466660 13559 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.466917 11325 rpcz_store.cc:276] 0709 12:13:02.938963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.939094 (+   131us) service_pool.cc:224] Handling call
0709 12:13:02.939465 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:02.939678 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:02.939951 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 12:13:04.243333 (+1303382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.243455 (+   122us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:04.243579 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.246617 (+  3038us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.268060 (+ 21443us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8263,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":115638,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":126,"thread_start_us":1302679,"threads_started":2,"wal-append.queue_time_us":1066478}
I20260709 12:13:04.470536 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.471922 11323 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:13:04.472683 13373 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:04.473202 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:04.450657 11296 connection.cc:256] RPC call timeout handler was delayed by -1.295157596000081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
I20260709 12:13:04.451598 13560 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.452451 13481 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:04.454072 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1209) took 1014 ms. Trace:
I20260709 12:13:04.454111 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1834) took 1515 ms. Trace:
I20260709 12:13:04.449250 13374 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:04.456480 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 149276us
I20260709 12:13:04.300908 11179 rpcz_store.cc:276] 0709 12:13:01.653206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.653480 (+   274us) service_pool.cc:224] Handling call
0709 12:13:01.653967 (+   487us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:01.872414 (+218447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.872451 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:01.914740 (+ 42289us) log.cc:844] Serialized 52 byte log entry
0709 12:13:02.327132 (+412392us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:02.492103 (+164971us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.492590 (+   487us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.796704 (+304114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.168817 (+1372113us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400363654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:04.168940 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.169247 (+   307us) raft_consensus.cc:1685] finished
0709 12:13:04.169276 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.169662 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28888,"prepare.queue_time_us":65875,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":46199,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":12914,"spinlock_wait_cycles":479653632,"thread_start_us":781848,"threads_started":3,"wal-append.queue_time_us":516338,"wal-append.run_cpu_time_us":3203,"wal-append.run_wall_time_us":1213201}
W20260709 12:13:04.445537 11327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.000s	sys 0.013s
I20260709 12:13:04.457055 13403 raft_consensus.cc:2764] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:13:04.474478 13571 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:13:04.475167 13567 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.343636 13554 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:04.476651 13471 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:13:04.481268 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 287) took 2853 ms. Trace:
I20260709 12:13:04.481737 11177 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:04.481932 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:04.482952 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:04.483372 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:04.802779 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.809448 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.807636 11328 rpcz_store.cc:276] 0709 12:13:03.439909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.307326 (+867417us) service_pool.cc:224] Handling call
0709 12:13:04.453985 (+146659us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.812716 11180 rpcz_store.cc:276] 0709 12:13:01.627529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.627661 (+   132us) service_pool.cc:224] Handling call
0709 12:13:01.627994 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.628300 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:02.358017 (+729717us) log.cc:844] Serialized 14 byte log entry
0709 12:13:04.476920 (+2118903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.480746 (+  3826us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:04.480869 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.480970 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.481183 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6360,"prepare.queue_time_us":765137,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":53,"raft.queue_time_us":34227,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":2776243,"threads_started":3,"wal-append.queue_time_us":302190,"wal-append.run_cpu_time_us":10645,"wal-append.run_wall_time_us":1775155}
I20260709 12:13:04.814319 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.705452 13366 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:04.815188 13366 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:04.815575 11328 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:04.816857 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.816609 13366 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.706236 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:04.820554 13366 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:04.811918 13554 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.819034 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1848) took 1125 ms. Trace:
I20260709 12:13:04.568066 11932 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEARNER]: Advancing to term 6
I20260709 12:13:04.447674 13563 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:04.822701 11327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.416s	user 0.000s	sys 0.027s
I20260709 12:13:04.477909 13559 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:04.502202 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.058s	sys 0.066s
I20260709 12:13:04.703351 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1844) took 1330 ms. Trace:
I20260709 12:13:04.486850 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 349) took 1542 ms. Trace:
I20260709 12:13:04.484006 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:04.780019 13552 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:04.826036 11296 connection.cc:256] RPC call timeout handler was delayed by -1.295157596000081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
I20260709 12:13:04.758188 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:04.746729 11223 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.941s (SENT) [suppressed 5 similar messages]
W20260709 12:13:04.832211 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.485342 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:04.705545 13454 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
I20260709 12:13:04.503716 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:04.842248 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105234us
I20260709 12:13:04.834323 11328 rpcz_store.cc:276] 0709 12:13:03.693386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.816429 (+1123043us) service_pool.cc:224] Handling call
0709 12:13:04.818967 (+  2538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.739619 11929 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:13:04.495733 11322 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.721s	user 0.017s	sys 0.004s
I20260709 12:13:04.701113 13371 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:04.847421 13465 raft_consensus.cc:476] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Not starting pre-election -- already a leader
I20260709 12:13:04.851449 11174 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:13:04.843587 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1052575960000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
I20260709 12:13:04.808257 11175 rpcz_store.cc:276] 0709 12:13:02.938921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.956684 (+ 17763us) service_pool.cc:224] Handling call
0709 12:13:02.957152 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.010898 (+ 53746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.010928 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.011424 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:13:04.114264 (+1102840us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:04.286422 (+172158us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:04.286495 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.286665 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.453663 (+166998us) raft_consensus.cc:1685] finished
0709 12:13:04.453685 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.453948 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53537,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":106,"spinlock_wait_cycles":23156608,"thread_start_us":1169455,"threads_started":1,"wal-append.queue_time_us":879939}
I20260709 12:13:04.839762 11326 rpcz_store.cc:276] 0709 12:13:03.372802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.383901 (+ 11099us) service_pool.cc:224] Handling call
0709 12:13:04.702840 (+1318939us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54001c43d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:04.703081 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689726848}
W20260709 12:13:04.859658 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0476575960001355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
I20260709 12:13:04.868031 11176 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:04.840340 11329 rpcz_store.cc:276] 0709 12:13:02.944057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.370888 (+426831us) service_pool.cc:224] Handling call
0709 12:13:03.371301 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:03.945324 (+574023us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:04.176747 (+231423us) log.cc:844] Serialized 14 byte log entry
0709 12:13:04.428347 (+251600us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.442108 (+ 13761us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:04.457732 (+ 15624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.486420 (+ 28688us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54000c5dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:04.486470 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.486685 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21459,"prepare.queue_time_us":138950,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":112988,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":10987,"spinlock_wait_cycles":1293642624,"thread_start_us":358049,"threads_started":2,"wal-append.queue_time_us":261963}
I20260709 12:13:04.870088 11935 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:04.870935 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 413488us
I20260709 12:13:04.866482 11173 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:04.865048 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.863545 13465 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:04.873126 13465 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.875463 13465 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:04.876082 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.844578 13454 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.885095 13480 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:04.886180 13480 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.886482 13371 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.888805 13371 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:04.888873 13480 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:04.879585 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 1.50026e+06us
I20260709 12:13:04.884107 11324 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:04.881947 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.867287 13373 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:04.864805 11323 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:13:04.850934 13552 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.895727 11328 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:04.898082 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:04.900286 13115 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:04.900688 13506 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:04.900951 13115 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:04.901582 13115 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:04.890420 13567 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:04.893816 11322 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:04.908247 11182 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:04.909162 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 348) took 1965 ms. Trace:
I20260709 12:13:04.910842 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:04.921756 13480 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.922765 13289 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:04.923203 13289 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:04.923538 13401 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:04.924284 13401 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.924461 13289 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.922497 13480 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.928861 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:04.937691 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.927258 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 355) took 1815 ms (client timeout 2340 ms). Trace:
I20260709 12:13:04.933585 11329 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:04.939697 11330 rpcz_store.cc:269] 0709 12:13:03.111703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.359391 (+247688us) service_pool.cc:224] Handling call
0709 12:13:04.926986 (+1567595us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b5400323cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:04.927183 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822880640}
I20260709 12:13:04.927691 11326 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:04.925611 13289 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:04.909616 13374 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.948566 13335 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.910524 11327 rpcz_store.cc:276] 0709 12:13:02.943769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.388180 (+444411us) service_pool.cc:224] Handling call
0709 12:13:03.388555 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:03.388835 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:03.406555 (+ 17720us) log.cc:844] Serialized 14 byte log entry
0709 12:13:04.864948 (+1458393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.881440 (+ 16492us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:04.894548 (+ 13108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.899652 (+  5104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.909103 (+  9451us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1914,"prepare.queue_time_us":17486,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":2925,"raft.queue_time_us":9725,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":9579,"thread_start_us":1432937,"threads_started":1,"wal-append.queue_time_us":871052}
I20260709 12:13:04.892030 13563 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:04.954732 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1849) took 1200 ms. Trace:
I20260709 12:13:04.926617 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:04.957188 11326 rpcz_store.cc:276] 0709 12:13:03.743533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.858397 (+1114864us) service_pool.cc:224] Handling call
0709 12:13:04.943465 (+ 85068us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.959494 11328 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:04.942178 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:04.954380 13403 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.962755 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.965484 11248 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:04.965698 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:04.965680 11254 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:13:04.967484 13480 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:04.970407 11330 raft_consensus.cc:2410] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:13:04.960922 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:04.959713 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.973310 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.940639 13373 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:04.966202 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1838) took 2014 ms (client timeout 1633 ms). Trace:
I20260709 12:13:04.979259 11182 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:04.980165 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:04.983078 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:04.982170 13567 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:04.960497 13335 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:04.987877 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73221.6us
I20260709 12:13:04.994302 13413 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.994899 13413 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:04.991531 13560 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.997282 11253 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:13:04.997761 11253 pending_rounds.cc:85] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 2
I20260709 12:13:04.998044 11253 pending_rounds.cc:107] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:13:04.998383 11253 raft_consensus.cc:2889] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.001519 13335 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:04.994102 13305 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:13:05.006385 13443 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:05.006592 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.006958 13443 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:05.007513 13443 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:05.007815 13482 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:05.007882 13166 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.978710 13480 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.007709 11331 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:05.004382 13574 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:13:05.015981 11327 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:13:04.979110 11322 rpcz_store.cc:269] 0709 12:13:02.942071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.011671 (+ 69600us) service_pool.cc:224] Handling call
0709 12:13:04.955434 (+1943763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.018348 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:05.008970 13401 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.024652 13446 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.025441 13553 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:05.025979 13166 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.027060 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.028822 13554 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.029731 13374 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.037882 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.038956 11325 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:05.039268 11323 raft_consensus.cc:2410] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:05.039806 13480 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:05.040421 13413 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:05.043394 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:05.062564 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51346.6us
I20260709 12:13:05.065129 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 351) took 2116 ms. Trace:
I20260709 12:13:05.045253 11079 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.046680 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.065516 11325 rpcz_store.cc:276] 0709 12:13:02.945154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.021350 (+2076196us) service_pool.cc:224] Handling call
0709 12:13:05.038752 (+ 17402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.061346 (+ 22594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.061583 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.050261 13335 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:05.051362 13373 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.051887 11076 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 257) took 8103 ms. Trace:
I20260709 12:13:05.053619 13552 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:04.988566 13403 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.068823 11331 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:05.070459 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.048072 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:05.067368 11324 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:05.073908 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1216) took 1138 ms. Trace:
I20260709 12:13:05.072435 11257 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:05.074492 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.074851 11248 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:05.075060 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.076004 11256 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:05.078095 13403 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.073210 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1850) took 1308 ms. Trace:
I20260709 12:13:05.079109 11323 rpcz_store.cc:276] 0709 12:13:03.763538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.915453 (+1151915us) service_pool.cc:224] Handling call
0709 12:13:05.071557 (+156104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.080772 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.054072 13454 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.082528 11323 raft_consensus.cc:2410] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:05.082350 13446 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.083343 13563 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.083534 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.084219 11323 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:05.084779 13446 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.085590 11254 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:05.086382 13510 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:05.086915 13510 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:05.087024 13480 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.087491 13510 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:05.088397 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 350) took 2144 ms. Trace:
I20260709 12:13:05.088866 11330 rpcz_store.cc:276] 0709 12:13:02.944836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.985012 (+2040176us) service_pool.cc:224] Handling call
0709 12:13:05.023463 (+ 38451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.087930 (+ 64467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.087971 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.088051 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:05.088103 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.088166 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.088301 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.090566 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.090831 11251 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:05.093091 11322 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:05.060632 13553 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:05.097146 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:05.091150 11330 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
I20260709 12:13:05.098192 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.074258 11324 rpcz_store.cc:276] 0709 12:13:03.935179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.483684 (+548505us) service_pool.cc:224] Handling call
0709 12:13:05.073852 (+590168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.099720 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.099647 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:05.101258 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:05.101660 11322 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:05.103413 13166 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.104776 11182 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:05.107843 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:05.108695 11182 raft_consensus.cc:2410] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:05.108611 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:05.109447 11927 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:05.109959 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:05.110798 11251 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:05.110597 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:05.114486 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:05.115715 11331 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:05.097576 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:05.097602 11932 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
W20260709 12:13:05.116588 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 43786.5us
I20260709 12:13:05.124655 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.122866 11932 pending_rounds.cc:85] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 2
I20260709 12:13:05.126010 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.127816 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.127852 11932 pending_rounds.cc:107] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:05.129851 11932 raft_consensus.cc:2889] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.130054 11328 raft_consensus.cc:2410] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:05.132253 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.133276 11251 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:13:05.134443 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.135088 11331 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:05.136003 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.137236 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.138034 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:05.139680 13576 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:13:05.139763 13553 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.141278 13553 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:05.142174 13335 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.171514 13335 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.174062 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.144737 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.151327 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.154990 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.178155 11256 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:05.156404 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:05.158764 13540 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.179314 11253 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:05.179656 13540 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.180804 11251 raft_consensus.cc:2410] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:05.182385 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:05.159878 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:05.184455 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 289529us
I20260709 12:13:05.161495 11327 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:05.165066 11331 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:05.188524 11329 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:05.189072 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1218) took 1205 ms. Trace:
I20260709 12:13:05.189832 11249 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:05.189459 11147 connection.cc:256] RPC call timeout handler was delayed by -2.5841133399999308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260709 12:13:05.190807 11329 rpcz_store.cc:276] 0709 12:13:03.984002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.877392 (+893390us) service_pool.cc:224] Handling call
0709 12:13:05.189011 (+311619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.193347 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.192659 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 352) took 2200 ms. Trace:
I20260709 12:13:05.195463 11324 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:05.195104 11327 rpcz_store.cc:276] 0709 12:13:02.989407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.160820 (+2171413us) service_pool.cc:224] Handling call
0709 12:13:05.161173 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.185305 (+ 24132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.189980 (+  4675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.199203 11330 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:05.193367 11079 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.198803 11325 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
W20260709 12:13:05.217936 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 291453us
I20260709 12:13:05.220484 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.222908 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.223182 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.223661 11329 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:05.223935 11330 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:05.202345 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.193027 13335 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:05.192971 11322 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 2.
W20260709 12:13:05.227972 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44111.1us
I20260709 12:13:05.219465 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.232918 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:05.234599 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:05.242300 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.202654 13577 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:05.227269 13481 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:05.248648 13554 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:05.245677 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:05.249780 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48997.2us
I20260709 12:13:05.249758 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.247054 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 353) took 2222 ms. Trace:
I20260709 12:13:05.257735 11327 rpcz_store.cc:276] 0709 12:13:03.024385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.197605 (+2173220us) service_pool.cc:224] Handling call
0709 12:13:05.226251 (+ 28646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.236904 (+ 10653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.236946 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.246772 (+  9826us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:05.246812 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.246870 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.246970 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.259697 11329 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:05.246572 13481 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.261313 11331 raft_consensus.cc:2410] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:05.263509 13289 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.263782 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.264005 13289 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.264669 11327 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:13:05.249401 13540 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:05.275517 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.278474 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.266346 13481 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:05.279747 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.251895 13443 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.281211 11326 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:05.071148 11076 rpcz_store.cc:276] 0709 12:12:56.948324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.948597 (+   273us) service_pool.cc:224] Handling call
0709 12:13:05.051585 (+8102988us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:01.347973 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:01.461944 (+113971us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:01.461981 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:13:01.462000 (+    19us) tablet.cc:662] Decoding operations
0709 12:13:01.462964 (+   964us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:01.463022 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:01.463047 (+    25us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:13:01.463805 (+   758us) tablet.cc:701] Row locks acquired
0709 12:13:01.463824 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:13:01.463976 (+   152us) write_op.cc:270] Start()
0709 12:13:01.464088 (+   112us) write_op.cc:276] Timestamp: P: 1783599181463921 usec, L: 0
0709 12:13:01.464108 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:13:01.464774 (+   666us) log.cc:844] Serialized 3752 byte log entry
0709 12:13:02.546447 (+1081673us) op_driver.cc:464] REPLICATION: finished
0709 12:13:03.154101 (+607654us) write_op.cc:301] APPLY: starting
0709 12:13:03.154218 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:03.154520 (+   302us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:03.226727 (+ 72207us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:03.355346 (+128619us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:03.355558 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:13:03.355611 (+    53us) write_op.cc:312] APPLY: finished
0709 12:13:03.355895 (+   284us) log.cc:844] Serialized 71 byte log entry
0709 12:13:03.356785 (+   890us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:03.356920 (+   135us) write_op.cc:454] Released schema lock
0709 12:13:03.356977 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:03.398108 (+ 41131us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e932a 00007f3ec666ef14 00007f3ec666eec3 00007f3ec666ee38 00007f3ec666edf3 00007f3ec666e0dc 00007f3ec6697e16 00007f3ec667fcbb 00007f3ec666a39d 00007f3ec666dd75
0709 12:13:03.409896 (+ 11788us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:03.412273 (+  2377us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":595493,"apply.run_cpu_time_us":4485,"apply.run_wall_time_us":224677,"num_ops":9,"prepare.queue_time_us":891731,"prepare.run_cpu_time_us":7087,"prepare.run_wall_time_us":357157,"raft.queue_time_us":608261,"raft.run_cpu_time_us":24372,"raft.run_wall_time_us":854522,"replication_time_us":1082155,"spinlock_wait_cycles":5594880,"thread_start_us":1220395,"threads_started":4,"wal-append.queue_time_us":401805,"wal-append.run_cpu_time_us":21847,"wal-append.run_wall_time_us":2569526}]]}
W20260709 12:13:05.289951 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 90127.7us
I20260709 12:13:05.295666 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.283962 13166 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:05.299857 13555 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:13:05.300565 13540 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:05.313551 13166 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:05.315223 13166 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:05.292801 13289 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:05.317349 13481 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.321489 13481 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.325977 13540 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:05.327551 13540 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.289207 13371 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:05.291647 13335 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:05.289537 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:05.339581 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.352874 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.339728 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44196.1us
I20260709 12:13:05.340884 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:05.341197 13563 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:05.353139 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:05.356184 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 1846) took 1914 ms. Trace:
I20260709 12:13:05.356534 11326 rpcz_store.cc:276] 0709 12:13:03.441576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.964804 (+1523228us) service_pool.cc:224] Handling call
0709 12:13:05.356027 (+391223us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54001c3ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:05.356130 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698381184}
I20260709 12:13:05.357779 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.359170 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:05.359997 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60540.6us
I20260709 12:13:05.360112 11324 raft_consensus.cc:2376] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6. Re-sending same reply.
I20260709 12:13:05.366822 11173 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.339007 13371 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.348232 13335 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.374285 11182 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:05.380328 11328 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:05.374022 13481 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:05.354722 13563 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:05.389865 11331 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 12:13:05.359006 11322 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:05.394521 11327 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:13:05.390420 11331 pending_rounds.cc:85] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 1
I20260709 12:13:05.401304 11331 pending_rounds.cc:107] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:13:05.401726 11331 raft_consensus.cc:2889] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.375176 13371 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:05.379432 13335 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:05.384116 13481 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:05.404841 11329 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
W20260709 12:13:05.400122 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.400712 11327 pending_rounds.cc:85] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:13:05.410101 11327 pending_rounds.cc:107] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:05.410467 11327 raft_consensus.cc:2889] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.407831 11324 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:05.411571 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1212) took 1572 ms. Trace:
I20260709 12:13:05.412125 11324 rpcz_store.cc:276] 0709 12:13:03.839124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.367374 (+1528250us) service_pool.cc:224] Handling call
0709 12:13:05.396428 (+ 29054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.411172 (+ 14744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.411492 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.409217 11329 pending_rounds.cc:85] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:13:05.418982 11329 pending_rounds.cc:107] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:05.399184 11079 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 6, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:05.421470 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.423700 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:05.389554 13563 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:05.418313 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1211) took 1743 ms. Trace:
I20260709 12:13:05.425475 11327 rpcz_store.cc:276] 0709 12:13:03.792700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.365155 (+1572455us) service_pool.cc:224] Handling call
0709 12:13:05.394169 (+ 29014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.417946 (+ 23777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.418235 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.434938 11329 raft_consensus.cc:2889] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:05.436934 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44575.4us
I20260709 12:13:05.439446 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1845) took 2027 ms. Trace:
W20260709 12:13:05.442106 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48074.8us
I20260709 12:13:05.444398 11173 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:05.439957 11330 rpcz_store.cc:276] 0709 12:13:03.412081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.357148 (+1945067us) service_pool.cc:224] Handling call
0709 12:13:05.379035 (+ 21887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.424597 (+ 45562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.424667 (+    70us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.439159 (+ 14492us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:05.439200 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.439264 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.439385 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.446310 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1205) took 2058 ms. Trace:
I20260709 12:13:05.452713 11331 rpcz_store.cc:276] 0709 12:13:03.387752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.365087 (+1977335us) service_pool.cc:224] Handling call
0709 12:13:05.378681 (+ 13594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.445960 (+ 67279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.446232 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.455320 13443 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.414876 13481 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:05.461730 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.468325 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:05.469164 11324 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:05.473382 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:05.467656 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1208) took 2044 ms. Trace:
W20260709 12:13:05.474926 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 277824us
I20260709 12:13:05.475401 11329 rpcz_store.cc:276] 0709 12:13:03.424233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.358177 (+1933944us) service_pool.cc:224] Handling call
0709 12:13:05.388811 (+ 30634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.467282 (+ 78471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.467568 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.475111 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1847) took 1796 ms. Trace:
I20260709 12:13:05.477371 11325 rpcz_store.cc:276] 0709 12:13:03.678505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.365246 (+1686741us) service_pool.cc:224] Handling call
0709 12:13:05.397424 (+ 32178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.444371 (+ 46947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.444406 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.457978 (+ 13572us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:05.474866 (+ 16888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.474933 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.475052 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.477378 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:05.413769 13510 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.472674 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:05.459618 13115 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.000s
I20260709 12:13:05.485898 11328 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:05.488319 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1215) took 1563 ms. Trace:
I20260709 12:13:05.488505 11325 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:05.488777 11328 rpcz_store.cc:276] 0709 12:13:03.925196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.459562 (+1534366us) service_pool.cc:224] Handling call
0709 12:13:05.485532 (+ 25970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.487986 (+  2454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.488245 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:05.493870 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 114407us
I20260709 12:13:05.490197 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:05.494793 11328 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:05.503566 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.501837 11900 connection.cc:256] RPC call timeout handler was delayed by -2.1528447670000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 12:13:05.508029 11900 connection.cc:256] RPC call timeout handler was delayed by -1.7292743840000639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.248s for a response
I20260709 12:13:05.489987 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:05.508335 11900 connection.cc:256] RPC call timeout handler was delayed by -1.67944476699995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:13:05.508697 11900 connection.cc:256] RPC call timeout handler was delayed by -1.5248266769999645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 12:13:05.509465 11330 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:05.510151 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1214) took 1586 ms. Trace:
I20260709 12:13:05.510468 11330 rpcz_store.cc:276] 0709 12:13:03.924083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.445577 (+1521494us) service_pool.cc:224] Handling call
0709 12:13:05.488092 (+ 42515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.509904 (+ 21812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.510095 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.513119 13553 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.515772 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:05.516345 13553 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.517060 13373 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:05.517707 13373 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.518639 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:05.519315 13553 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.519934 13173 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:05.520452 13173 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.520818 13173 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:05.526110 11326 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:13:05.526556 11326 pending_rounds.cc:85] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 1
I20260709 12:13:05.526788 11326 pending_rounds.cc:107] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:13:05.527057 11326 raft_consensus.cc:2889] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:05.528259 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.527529 11174 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:05.486933 11079 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 6, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:05.542346 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.519908 13553 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:05.478700 11331 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:13:05.546917 11331 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:05.547465 11331 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:05.501210 13443 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:13:05.495172 13115 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.528379 13373 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:05.560890 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46755.9us
I20260709 12:13:05.563127 11182 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:05.566272 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:05.570854 11325 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:05.573877 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:05.588871 11079 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 3 to 4, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 4 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.590873 11079 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 3 to 6, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.592527 11079 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 3 to 4, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 4 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:13:05.601037 11079 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:05.617868 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 91163.4us
I20260709 12:13:05.570554 13373 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:13:05.654778 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1213) took 1738 ms. Trace:
I20260709 12:13:05.655344 11326 rpcz_store.cc:276] 0709 12:13:03.913402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.372182 (+1458780us) service_pool.cc:224] Handling call
0709 12:13:05.441262 (+ 69080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.554529 (+113267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.651915 (+ 97386us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b54000e6854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:05.652093 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165430656}
W20260709 12:13:05.697198 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.719697 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 76140.2us
I20260709 12:13:05.721520 11328 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:05.717123 11331 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.727713 11326 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:05.729507 11324 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:05.731127 11325 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:05.730279 11079 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 6, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:05.737550 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 249485us
I20260709 12:13:05.740940 11331 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:05.749450 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 259538us
I20260709 12:13:05.735570 11322 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:05.765360 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39356 (request call id 1222) took 1573 ms. Trace:
I20260709 12:13:05.767639 11331 rpcz_store.cc:276] 0709 12:13:04.168528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.472388 (+1303860us) service_pool.cc:224] Handling call
0709 12:13:05.741652 (+269264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.751964 13443 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.769388 13443 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.771003 11325 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:13:05.771530 11325 pending_rounds.cc:85] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:13:05.771849 11325 pending_rounds.cc:107] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:13:05.773437 11325 raft_consensus.cc:2889] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.748210 13510 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.731980 13173 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.771633 11079 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.785740 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:05.789520 11326 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:05.749789 13578 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:05.793807 11324 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:05.764319 13454 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:05.766819 13540 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:05.803319 13586 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:13:05.805424 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:05.792241 13578 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.802294 11079 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 3, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 3 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.817031 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1223) took 1648 ms. Trace:
I20260709 12:13:05.817615 11329 rpcz_store.cc:276] 0709 12:13:04.169012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.476075 (+1307063us) service_pool.cc:224] Handling call
0709 12:13:05.484822 (+  8747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.489383 (+  4561us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:05.722556 (+233173us) log.cc:844] Serialized 14 byte log entry
0709 12:13:05.816664 (+ 94108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.816717 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:05.816766 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.816836 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.816968 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":262983,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"thread_start_us":266205,"threads_started":2}
W20260709 12:13:05.825518 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 361766us
I20260709 12:13:05.799091 13577 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:05.854380 11147 connection.cc:256] RPC call timeout handler was delayed by -1.13716377500009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
I20260709 12:13:05.804672 13115 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:05.876116 13588 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:13:05.801201 13590 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:05.879901 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.852663 11079 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 3 to 5, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 5 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.877832 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1220) took 1711 ms. Trace:
I20260709 12:13:05.877112 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:05.826934 13578 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:05.896389 11327 rpcz_store.cc:276] 0709 12:13:04.167167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.471640 (+1304473us) service_pool.cc:224] Handling call
0709 12:13:05.482638 (+ 10998us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:05.492816 (+ 10178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.492864 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:05.513747 (+ 20883us) log.cc:844] Serialized 52 byte log entry
0709 12:13:05.649358 (+135611us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:05.734251 (+ 84893us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:05.734312 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.734414 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:05.877430 (+143016us) raft_consensus.cc:1685] finished
0709 12:13:05.877460 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.877756 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2808,"raft.queue_time_us":6359,"raft.run_cpu_time_us":3025,"raft.run_wall_time_us":152942,"spinlock_wait_cycles":4623232,"thread_start_us":177243,"threads_started":1,"wal-append.queue_time_us":253101}
W20260709 12:13:05.900789 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 105081us
I20260709 12:13:05.853551 13577 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.834676 13443 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.801262 13563 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.905158 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:05.906499 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.830472 13366 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:05.915866 11079 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 3, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 3 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.928190 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:05.931455 11182 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:05.932847 13166 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:05.933741 13587 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:13:05.934692 13166 raft_consensus.cc:2764] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:13:05.937140 13555 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:05.913415 13366 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:05.938015 13366 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.938230 11176 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:05.916296 13115 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:13:05.941828 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:05.941746 13590 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.835250 13540 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.944375 13590 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:05.944712 13596 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:05.945205 13596 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.855759 13371 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:05.909204 13577 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:05.950192 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50174.5us
I20260709 12:13:05.874696 13591 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 12:13:05.945550 13596 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:05.953328 13601 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.953951 13601 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.956270 13601 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.942049 13481 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:05.944746 13540 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.951175 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:05.959353 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:05.959527 13481 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:05.960108 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:05.934856 13593 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:13:05.960613 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:05.979610 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 109009us
W20260709 12:13:05.982239 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60419.5us
W20260709 12:13:05.989375 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.990002 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.858237 13335 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:05.977658 13446 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:05.991189 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:05.976820 13374 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:06.003942 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 201987us
W20260709 12:13:06.005318 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 128255us
I20260709 12:13:06.007153 13609 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:05.986625 13454 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:05.991981 13563 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:06.011950 13446 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.010901 13374 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:06.014772 13454 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.037355 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:06.016567 13480 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:06.026687 13289 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:06.040526 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80024.3us
I20260709 12:13:06.028051 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1840) took 2953 ms. Trace:
I20260709 12:13:06.034026 13446 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:06.079263 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.057770 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:06.073060 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.035918 13374 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.038544 13454 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.054365 13289 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:06.101949 13620 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:06.074007 13371 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:06.058920 13617 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:06.094475 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:06.064682 13173 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:06.052562 13591 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
W20260709 12:13:06.106915 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 114615us
I20260709 12:13:06.078179 13563 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:13:06.096979 13289 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.112358 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:06.117513 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:06.121145 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.121869 11173 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:06.124704 11173 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:06.126143 13335 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:06.069873 13604 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:13:06.127476 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.128150 11173 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:06.130214 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.130836 11173 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:06.132546 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.134828 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.137452 11173 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:06.056509 11323 rpcz_store.cc:276] 0709 12:13:03.063836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.280162 (+2216326us) service_pool.cc:224] Handling call
0709 12:13:05.366206 (+ 86044us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:05.496497 (+130291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.511360 (+ 14863us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:05.581812 (+ 70452us) log.cc:844] Serialized 52 byte log entry
0709 12:13:05.787581 (+205769us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:05.962278 (+174697us) log.cc:844] Serialized 14 byte log entry
0709 12:13:05.988190 (+ 25912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.993881 (+  5691us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:06.003377 (+  9496us) raft_consensus.cc:1685] finished
0709 12:13:06.005087 (+  1710us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:06.016912 (+ 11825us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10283,"prepare.queue_time_us":117932,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2427,"raft.queue_time_us":25108,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":22673,"thread_start_us":254126,"threads_started":2,"wal-append.queue_time_us":274468}
I20260709 12:13:06.140951 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.142355 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:06.143776 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:06.144382 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.132316 13596 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:06.154467 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 142222us
I20260709 12:13:06.154857 11182 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:06.113122 13481 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:06.111253 13607 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:13:06.154268 13554 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:06.159956 13554 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:06.160305 13554 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:06.112629 13115 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:13:06.160693 13554 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.111470 13553 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:13:06.162611 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.164036 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.165046 11178 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:13:06.137060 13366 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:06.157616 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:06.172000 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.115077 13617 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:06.175076 11180 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:06.178056 11182 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:06.179168 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.180347 11179 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:06.161059 13446 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.166003 13604 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:06.197011 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:06.211450 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.170377 13481 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:06.165354 13578 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:06.223881 11222 connection.cc:256] RPC call timeout handler was delayed by -1.8139859890000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
W20260709 12:13:06.225291 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:06.231413 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 119212us
I20260709 12:13:06.229744 13604 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:06.232293 13590 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:06.233151 13590 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:06.233341 13604 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:06.234032 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:06.234256 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:06.235500 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:06.235699 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:06.236745 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:06.237264 13403 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:06.238106 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:06.238045 13403 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:06.238803 13596 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:06.239311 13619 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.239498 13596 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:06.239991 13596 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:06.240249 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:06.239943 13619 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:06.240549 13403 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:06.241104 11328 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:06.241393 13403 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:06.197383 13563 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:06.240593 13596 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.242300 13615 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:06.242669 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:06.230448 13609 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:06.244585 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:06.253129 11330 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:06.247521 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:06.251075 13615 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:06.237401 11327 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:06.190299 13607 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 12:13:06.267580 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 123889us
I20260709 12:13:06.196506 13601 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:13:06.281064 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:06.305285 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 361) took 1109 ms. Trace:
I20260709 12:13:06.318825 11252 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 12:13:06.319350 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:06.305944 11331 rpcz_store.cc:276] 0709 12:13:05.195776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.781767 (+585991us) service_pool.cc:224] Handling call
0709 12:13:05.782173 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.782617 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:06.004570 (+221953us) log.cc:844] Serialized 14 byte log entry
0709 12:13:06.304727 (+300157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:06.304814 (+    87us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:06.304884 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:06.304991 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:06.305193 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66031,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2136,"thread_start_us":456250,"threads_started":3}
I20260709 12:13:06.321570 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:06.321877 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.322263 11253 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:06.322487 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.323235 11182 raft_consensus.cc:2368] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5.
I20260709 12:13:06.252072 13563 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:06.321601 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 263899us
I20260709 12:13:06.325665 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:06.326501 11249 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:06.327751 11256 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
W20260709 12:13:06.328980 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74311.4us
I20260709 12:13:06.329711 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:06.332444 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:06.332669 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:06.331597 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:06.334308 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:06.340464 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:06.342577 13628 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.343272 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.343163 13628 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:06.346316 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:06.346416 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.342356 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.357064 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:06.329792 13563 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:06.362875 11179 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.363816 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:06.374341 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:06.376513 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:06.384198 13173 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:06.384375 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:06.367903 11323 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:06.401705 11255 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:06.401820 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:06.362563 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.357297 13628 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:06.428020 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71232.5us
I20260709 12:13:06.384418 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:06.394934 13335 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:06.390089 13609 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:06.437052 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:06.402375 13403 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:06.450079 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:06.421981 13480 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:06.450860 13366 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:06.464190 13609 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:06.464504 13403 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.505657 13403 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.348305 13615 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:06.500330 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 123915us
W20260709 12:13:06.511304 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 183507us
I20260709 12:13:06.437144 13621 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:06.435643 13620 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:13:06.514047 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.440471 13619 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:06.520202 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:06.444434 13335 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.392228 13629 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 12:13:06.463480 13480 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.526449 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:06.525413 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:06.385221 13481 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.597702 13619 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:13:06.526268 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.598595 13620 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:06.598901 11931 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:06.598665 13622 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.214s
I20260709 12:13:06.518144 13173 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:06.462653 13623 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:06.626477 13629 consensus_queue.cc:1243] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:13:06.626623 13623 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.627060 13623 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:06.525475 13602 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:13:06.579532 11147 connection.cc:256] RPC call timeout handler was delayed by -1.5021153120001145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3229s for a response
I20260709 12:13:06.523435 13621 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.509011 13609 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:06.599221 13620 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:06.363577 13617 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.569350 13480 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.625856 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:06.629869 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 104555us
I20260709 12:13:06.581133 13615 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:06.572431 13335 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:06.582738 13578 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:06.530419 13374 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.572690 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.572857 13635 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:06.648249 11926 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:06.651292 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:06.662112 11147 connection.cc:256] RPC call timeout handler was delayed by -1.3365153120000741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3413s for a response
I20260709 12:13:06.637389 13621 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:06.676788 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 228388us
I20260709 12:13:06.672245 13374 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:13:06.646605 13602 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:06.638358 13555 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.637095 13620 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.657155 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:06.669956 13624 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:13:06.684516 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 164430us
I20260709 12:13:06.686048 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:06.688145 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:06.670377 13578 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:06.648351 13618 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.265s
I20260709 12:13:06.658545 13622 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:13:06.676116 13635 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:06.681937 13609 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:13:06.709764 13632 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
W20260709 12:13:06.710469 13289 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.001s
I20260709 12:13:06.710906 13289 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.682367 13627 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:06.702778 11181 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:06.700965 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:06.713784 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:06.702865 11222 leader_election.cc:400] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5.
I20260709 12:13:06.704322 11177 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:06.706135 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.707244 13635 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.714926 11222 leader_election.cc:403] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:06.718442 11176 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:06.720700 11179 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:06.701921 13578 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:06.703311 13591 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.699373 13166 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:06.737886 11327 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:06.722064 11929 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:06.738708 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:06.744801 11177 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
W20260709 12:13:06.720026 13630 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
W20260709 12:13:06.747534 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.751787 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab915731e0347ae8353a7f300c44401"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:06.754346 11174 raft_consensus.cc:606] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:06.754792 11174 raft_consensus.cc:614] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:06.743737 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.795604 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:06.801234 13604 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:06.801851 13604 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.802297 13604 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:06.757867 13591 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.813001 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:06.814618 13650 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:06.814759 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:06.719907 13627 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.815485 13650 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:06.815639 11322 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
W20260709 12:13:06.823854 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 492927us
I20260709 12:13:06.732657 13648 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 12:13:06.846868 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.848728 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:06.852517 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:06.853367 11928 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:06.856277 13600 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:06.856873 13600 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:06.843839 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:06.775472 11926 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:06.722539 13173 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:06.863502 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 123756us
I20260709 12:13:06.816785 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:06.760267 13602 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:06.879133 13554 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.000s
I20260709 12:13:06.771173 13373 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
W20260709 12:13:06.886698 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 68478.3us
I20260709 12:13:06.907226 13371 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:06.796939 13166 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:06.817780 13640 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 12:13:06.956826 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.952277 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:06.882869 13590 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:06.952586 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 205623us
W20260709 12:13:06.996930 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.886698 11322 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:06.954496 13373 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:07.056818 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 243870us
I20260709 12:13:06.993984 13634 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:06.901059 13554 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:06.999339 13617 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:07.112591 13645 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
W20260709 12:13:07.099907 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 411791us
I20260709 12:13:07.134248 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:07.002611 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:07.120365 13634 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:13:07.123416 13660 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:07.118057 13373 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:07.099290 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 178900us
W20260709 12:13:07.135273 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 132795us
I20260709 12:13:07.152630 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:07.163618 13289 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:07.166880 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:07.169080 13655 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:07.169363 13637 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.169775 13664 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:07.170080 13655 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:07.170130 13637 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.171204 13673 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:13:07.174944 11326 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:07.175444 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 486986us
I20260709 12:13:07.151777 13373 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:07.181743 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:07.178009 13648 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:07.132359 13617 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:07.197072 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 199319us
I20260709 12:13:07.175014 13664 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:07.212364 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.173795 13591 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:07.183168 13637 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:07.269543 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 104490us
W20260709 12:13:07.203739 13335 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.001s
I20260709 12:13:07.309024 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:07.227466 13555 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.023s
W20260709 12:13:07.327986 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 200966us
I20260709 12:13:07.229598 13599 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:07.338261 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 106508us
I20260709 12:13:07.186321 13673 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.142s
I20260709 12:13:07.232494 13664 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:07.185811 13648 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:07.337062 13659 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.186066 13621 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.331910 13555 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:07.353358 13555 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:07.354869 13305 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:07.355369 13305 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:07.351372 13577 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.401s	user 0.011s	sys 0.002s
W20260709 12:13:07.357797 13652 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 12:13:07.357676 13577 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.360195 13577 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:07.351732 13596 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.102s	user 0.013s	sys 0.000s
I20260709 12:13:07.352078 13671 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:13:07.360092 13670 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.356528 13305 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.337759 13599 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:07.366864 13659 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.328190 13335 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:07.433709 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 230637us
I20260709 12:13:07.347877 13666 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:07.365343 11327 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:07.348135 13648 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:07.391669 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:07.434722 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:07.346968 13166 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:07.361100 13577 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.432940 13599 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:07.442802 13679 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:13:07.441499 13305 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:07.443354 13659 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:07.444538 13289 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:13:07.444995 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53468.7us
I20260709 12:13:07.363636 13446 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:07.390233 11322 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:07.365751 13596 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.535096 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:07.603886 11174 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 2: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:07.511202 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:07.585460 11252 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:07.548808 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1248) took 1315 ms. Trace:
W20260709 12:13:07.608084 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.599278 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:07.574950 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:07.619837 11256 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:07.449591 13563 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.067s	user 0.011s	sys 0.001s
W20260709 12:13:07.726879 13620 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.029s	user 0.006s	sys 0.014s
I20260709 12:13:07.720100 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:07.721846 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:07.727610 13620 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.727921 13578 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.631137 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:07.729460 11257 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:13:07.720273 13621 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:07.729717 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 420776us
W20260709 12:13:07.685623 13615 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.027s	user 0.017s	sys 0.007s
I20260709 12:13:07.731078 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:07.731844 13621 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.732474 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:07.732390 13621 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.711896 11255 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:13:07.681288 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.003s	sys 0.023s
W20260709 12:13:07.735697 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.003s	sys 0.027s
W20260709 12:13:07.541699 13667 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
W20260709 12:13:07.737391 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 404972us
I20260709 12:13:07.729094 13578 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:07.734143 11255 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:07.708112 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1848) took 1569 ms. Trace:
W20260709 12:13:07.543344 13635 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.813s	user 0.012s	sys 0.009s
I20260709 12:13:07.438436 13591 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.290s
I20260709 12:13:07.685887 11933 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:13:07.440004 13655 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:07.459486 13623 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.832s	user 0.013s	sys 0.013s
W20260709 12:13:07.450979 13454 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.357s	user 0.016s	sys 0.001s
W20260709 12:13:07.543651 13669 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:13:07.740185 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:07.746129 11176 rpcz_store.cc:276] 0709 12:13:06.138959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.139066 (+   107us) service_pool.cc:224] Handling call
0709 12:13:06.139349 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:06.718866 (+579517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.707729 (+988863us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400364a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:07.708014 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324812544}
W20260709 12:13:07.682760 13677 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:13:07.607065 11328 rpcz_store.cc:276] 0709 12:13:06.233511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.241737 (+  8226us) service_pool.cc:224] Handling call
0709 12:13:06.242060 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:06.443474 (+201414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:06.443513 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:06.524982 (+ 81469us) log.cc:844] Serialized 26 byte log entry
0709 12:13:06.952841 (+427859us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:06.952914 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:06.953014 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:07.548538 (+595524us) raft_consensus.cc:1685] finished
0709 12:13:07.548558 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:07.548741 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24304,"raft.queue_time_us":288438,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":11628,"thread_start_us":382683,"threads_started":2,"wal-append.queue_time_us":665953}
W20260709 12:13:07.699638 13480 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.025s	user 0.000s	sys 0.013s
I20260709 12:13:07.545809 13446 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:07.458951 13577 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.438302 13666 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:07.550221 13403 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.044s	user 0.014s	sys 0.000s
W20260709 12:13:07.542870 13650 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.727s	user 0.005s	sys 0.010s
I20260709 12:13:07.710348 13607 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:07.430536 13670 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.461508 13627 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:07.458292 13289 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:07.776784 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 485) took 1447 ms. Trace:
I20260709 12:13:07.548448 11926 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEARNER]: Advancing to term 5
I20260709 12:13:07.627691 13554 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:07.755038 11255 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.790171 13454 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.546087 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:07.721287 13335 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:07.454766 13617 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:07.787550 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.653569 13685 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:07.546525 13373 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.771503 13590 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:13:07.799182 13480 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:07.712502 13604 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.910s	user 0.013s	sys 0.006s
I20260709 12:13:07.495287 13676 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.339s
W20260709 12:13:07.802026 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.019s	sys 0.022s
I20260709 12:13:07.746558 13563 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.859184 11255 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:07.558112 13599 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:07.819073 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64582.4us
I20260709 12:13:07.752640 13664 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.769712 13635 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.801401 13655 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.822129 13687 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.822933 13622 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:07.801074 13620 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:07.815992 13173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.004s	sys 0.012s
I20260709 12:13:07.830322 11933 rpcz_store.cc:276] 0709 12:13:06.329101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.383988 (+ 54887us) service_pool.cc:224] Handling call
0709 12:13:07.685276 (+1301288us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:07.685507 (+   231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:07.771474 (+ 85967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.776690 (+  5216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4677888}
I20260709 12:13:07.780402 13623 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:07.753772 13602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.018s	sys 0.050s
W20260709 12:13:07.834180 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 497022us
I20260709 12:13:07.834563 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 315) took 1095 ms. Trace:
I20260709 12:13:07.815637 13666 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:07.891067 13670 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:07.892050 11296 connection.cc:256] RPC call timeout handler was delayed by -2.5320939709999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 12:13:07.892580 11296 connection.cc:256] RPC call timeout handler was delayed by -2.4825939709999147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 12:13:07.892915 11296 connection.cc:256] RPC call timeout handler was delayed by -2.4807939709999118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 12:13:07.893232 11296 connection.cc:256] RPC call timeout handler was delayed by -2.4384939710000708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 12:13:07.837558 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.894459 13617 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.895053 13617 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.896544 13577 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:07.897197 13577 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:07.837675 11934 raft_consensus.cc:2410] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:07.820653 13650 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5.
I20260709 12:13:07.842046 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1241) took 1618 ms. Trace:
I20260709 12:13:07.825086 13648 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.819116 13607 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:07.930145 11329 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:07.863456 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:07.818781 13403 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.873556 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:07.917645 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:07.849494 13596 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:07.888629 13446 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:07.889144 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.899531 11331 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:07.900715 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:07.893559 11296 connection.cc:256] RPC call timeout handler was delayed by -1.5888939710000614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2775s for a response
I20260709 12:13:07.901259 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:07.881847 11182 rpcz_store.cc:276] 0709 12:13:06.739134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.739298 (+   164us) service_pool.cc:224] Handling call
0709 12:13:06.739607 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:06.748106 (+  8499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:06.748139 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:06.748634 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:13:07.000142 (+251508us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:07.000196 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.000294 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:07.725357 (+725063us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b48000030d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:07.830079 (+104722us) raft_consensus.cc:1685] finished
0709 12:13:07.830099 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:07.834495 (+  4396us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9423,"raft.run_cpu_time_us":2885,"raft.run_wall_time_us":973438,"spinlock_wait_cycles":5861248,"thread_start_us":183163,"threads_started":1,"wal-append.queue_time_us":379094}
I20260709 12:13:07.924404 11324 rpcz_store.cc:276] 0709 12:13:06.223988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.231316 (+  7328us) service_pool.cc:224] Handling call
0709 12:13:06.231771 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:06.879884 (+648113us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b1800462580. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66e4bfb 00007f3ec66ef64b
0709 12:13:06.880018 (+   134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:06.880043 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:06.880550 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:13:07.003955 (+123405us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:07.003999 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.005664 (+  1665us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:07.383188 (+377524us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:07.841703 (+458515us) raft_consensus.cc:1685] finished
0709 12:13:07.841740 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:07.841942 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25903,"raft.queue_time_us":203973,"raft.run_cpu_time_us":2412,"raft.run_wall_time_us":943491,"spinlock_wait_cycles":15664896,"thread_start_us":298714,"threads_started":2,"wal-append.queue_time_us":415166}
W20260709 12:13:07.857195 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.020s	sys 0.023s
W20260709 12:13:07.883937 13173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.004s	sys 0.012s
I20260709 12:13:07.873512 13335 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:07.928867 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1251) took 1220 ms. Trace:
I20260709 12:13:07.958333 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:07.959910 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:07.935046 11180 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:07.936123 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:07.888646 11927 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:07.962036 13695 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:07.962622 13695 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:07.962952 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:07.960033 11325 rpcz_store.cc:276] 0709 12:13:06.708560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.719735 (+ 11175us) service_pool.cc:224] Handling call
0709 12:13:06.737452 (+ 17717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:06.737876 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:06.737904 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:06.738349 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:13:07.459498 (+721149us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b1800209440. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:07.777485 (+317987us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:07.777525 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.777600 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:07.907642 (+130042us) raft_consensus.cc:1685] finished
0709 12:13:07.907671 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:07.928796 (+ 21125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"spinlock_wait_cycles":46902656,"thread_start_us":317811,"threads_started":1,"wal-append.queue_time_us":1093448}
I20260709 12:13:07.878261 13622 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:07.964629 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 734409us
I20260709 12:13:07.964833 11255 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:13:07.965557 11255 pending_rounds.cc:85] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 4
W20260709 12:13:07.966131 11932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 404) took 1616 ms (client timeout 1571 ms). Trace:
I20260709 12:13:07.859221 13604 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:07.897997 13656 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.881229 13621 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:07.966185 11175 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:07.964520 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:07.990464 11174 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:07.903294 13554 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:07.897795 13627 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:07.978843 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 142108us
I20260709 12:13:07.995205 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:07.996222 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:07.997746 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:07.981277 11322 raft_consensus.cc:2410] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:07.901374 13563 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.002385 13693 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:07.966838 11932 rpcz_store.cc:269] 0709 12:13:06.344352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.349129 (+  4777us) service_pool.cc:224] Handling call
0709 12:13:07.958135 (+1609006us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:07.960439 (+  2304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7831936}
I20260709 12:13:07.897887 13577 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:08.006696 13695 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:07.993417 13656 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.009177 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 372) took 1254 ms. Trace:
I20260709 12:13:08.009598 11329 rpcz_store.cc:276] 0709 12:13:06.754402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.767325 (+ 12923us) service_pool.cc:224] Handling call
0709 12:13:06.793943 (+ 26618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:07.930751 (+1136808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.008941 (+ 78190us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003237d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:08.009114 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2612396800}
I20260709 12:13:08.010174 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:07.898360 13628 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.897504 13617 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.011471 13628 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.012063 13617 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:07.898192 13637 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:07.985031 11249 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:07.988481 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.007238 13577 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.025000 11173 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:08.014549 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.015333 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.002403 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.016202 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.028465 11179 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:08.023850 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.024382 11174 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:07.937192 13602 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:08.026034 11177 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:13:08.029681 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.978001 13335 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:07.967590 11255 pending_rounds.cc:107] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260709 12:13:07.989750 11929 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:08.030578 11180 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:13:08.054546 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 94713.7us
I20260709 12:13:08.083695 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.030956 11252 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.032858 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.053782 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:08.082070 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 84435.6us
I20260709 12:13:08.115082 11255 raft_consensus.cc:2917] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 12:13:08.116544 11255 raft_consensus.cc:2889] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:08.118219 11255 pending_rounds.cc:107] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:13:08.070109 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.119237 11182 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:08.105901 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 103577us
I20260709 12:13:08.123275 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.117281 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:08.142907 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.119637 11255 raft_consensus.cc:2889] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:08.143349 11178 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:07.958669 13289 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.105662 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:07.995126 13446 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:08.063388 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:07.926725 13687 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.156920 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1411) took 1626 ms. Trace:
I20260709 12:13:08.105154 13563 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:08.159478 11255 rpcz_store.cc:276] 0709 12:13:06.530154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.594793 (+ 64639us) service_pool.cc:224] Handling call
0709 12:13:06.625992 (+ 31199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:07.711765 (+1085773us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b54003e8120. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6663dc0 00007f3ec66ee1c0 00007f3ec66ef231 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c
0709 12:13:08.156663 (+444898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.156863 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7711616}
I20260709 12:13:08.165577 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.019336 13685 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.106056 13656 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:08.120108 13650 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:08.185921 13454 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:08.186201 13636 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:08.105728 13335 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:08.054812 13621 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:08.032596 13596 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:08.164407 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.030182 13627 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.193501 13627 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:08.169734 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.035701 13554 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.196237 13554 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:08.164512 13629 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:13:08.197261 11252 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:08.066792 13637 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:08.202396 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 85214.6us
I20260709 12:13:08.158623 13373 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:08.211843 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69021.9us
I20260709 12:13:08.212875 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:08.204253 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:08.163947 13601 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:08.218596 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.219726 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.224080 11255 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:08.233393 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.187990 13602 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:08.240496 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:08.191509 13596 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:08.221385 11248 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.243678 11932 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.244493 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:08.248934 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.236558 13563 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:08.168751 13446 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.250785 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:08.250945 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:08.251663 11933 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:08.252839 13446 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.254024 13648 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.254734 13713 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:08.255067 11926 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:13:08.255326 13713 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.119413 13607 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.257337 13607 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:08.257604 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 517462us
I20260709 12:13:08.258317 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:08.258569 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:08.259709 13643 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.153s
I20260709 12:13:08.260365 13648 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:08.261902 13697 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:08.262185 13715 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:08.263183 11182 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:08.262852 13685 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:08.262250 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:08.264605 13446 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:08.265024 13446 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.265383 13446 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:08.266115 13604 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.266027 13622 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.267571 13577 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.267170 13602 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.270318 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.270537 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.270993 11251 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:08.271163 13702 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:08.271453 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.272033 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.272194 13621 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:08.272396 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 488) took 1426 ms. Trace:
I20260709 12:13:08.272616 13621 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.272785 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.273332 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.273494 13621 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.274902 11257 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:08.299741 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51959.3us
I20260709 12:13:08.332171 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.338073 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.310014 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.338616 11181 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:08.339641 11182 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.273991 11253 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:08.342198 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 93825.2us
I20260709 12:13:08.315399 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.350589 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.351320 11178 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.272871 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.339488 13577 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 12:13:08.254025 13403 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:08.372821 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.426913 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.426826 11254 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.414958 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.355777 11932 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:08.428737 11175 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:08.429553 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:08.414363 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 168136us
I20260709 12:13:08.430497 13620 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:08.432051 13687 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:08.433346 13578 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.003s
I20260709 12:13:08.429869 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.435034 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:08.415014 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71515.6us
I20260709 12:13:08.445851 11253 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.227340 13619 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:08.446054 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.273945 13628 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:08.236375 13664 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:08.245764 13554 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:08.439751 11325 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:08.453868 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:08.440855 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.442579 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.276434 13596 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.443660 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:08.448176 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.449430 11328 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.317832 13618 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:08.351660 13697 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.427831 13680 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.251048 13563 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.333580 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.437891 11329 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:08.480952 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.482517 11257 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:08.483372 13373 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:08.483850 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.484485 11257 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:08.478842 13289 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:08.456327 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.446334 13403 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:08.443349 13599 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.460000 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.451897 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.446492 13619 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.461107 13604 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:08.494751 11323 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:08.495468 13619 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.496064 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.496966 11323 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:08.457875 13554 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.499795 13554 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:08.272820 11928 rpcz_store.cc:276] 0709 12:13:06.887439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.887706 (+   267us) service_pool.cc:224] Handling call
0709 12:13:06.888291 (+   585us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:06.888847 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:06.888897 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:07.966776 (+1077879us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b2c000a0068. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:13:08.006150 (+ 39374us) log.cc:844] Serialized 52 byte log entry
0709 12:13:08.155300 (+149150us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:08.158753 (+  3453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.168761 (+ 10008us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:08.272051 (+103290us) raft_consensus.cc:1685] finished
0709 12:13:08.272077 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:08.272333 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20690,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":28113664,"thread_start_us":98691,"threads_started":1,"wal-append.queue_time_us":193013}
I20260709 12:13:08.465428 13697 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:08.477988 13680 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.451606 13577 consensus_queue.cc:1243] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:13:08.466152 13707 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260709 12:13:08.452256 13601 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:08.466853 13578 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:08.467686 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.468663 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:08.488693 13666 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.003s
I20260709 12:13:08.470357 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.474558 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:08.486096 11329 raft_consensus.cc:2368] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6.
I20260709 12:13:08.492650 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.488031 11248 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:08.493721 13622 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:08.528775 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.529672 11329 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:08.504253 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.489305 11254 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:08.486536 13289 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:08.536597 11148 leader_election.cc:400] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6.
I20260709 12:13:08.537099 11148 leader_election.cc:403] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:13:08.511827 11182 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:08.458906 13618 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.521337 11325 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.469132 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.521286 11253 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:08.479189 13563 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:08.542268 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 67797.4us
W20260709 12:13:08.540545 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 70294.6us
I20260709 12:13:08.543013 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:08.520859 13578 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.551728 11256 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:08.535036 13554 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.539311 13289 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:08.485570 13446 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.564160 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.565685 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.566637 11174 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:13:08.555233 11328 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:08.569006 11254 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.555614 13591 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:08.568765 11180 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:08.550385 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.571337 13680 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:08.571914 13709 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.314s	user 0.004s	sys 0.000s
I20260709 12:13:08.572037 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.572875 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.573102 13623 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:08.573434 13621 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:08.574815 11934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 399) took 2300 ms (client timeout 1707 ms). Trace:
I20260709 12:13:08.578402 11251 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.578243 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.561902 13373 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:08.581446 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.592649 11174 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:08.601897 11248 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:08.602053 13578 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.603498 13578 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.603648 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:08.594440 11934 rpcz_store.cc:269] 0709 12:13:06.274440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.280791 (+  6351us) service_pool.cc:224] Handling call
0709 12:13:07.721660 (+1440869us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:08.574543 (+852883us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540059a6d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:08.574717 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254466048}
I20260709 12:13:08.606779 11176 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:08.534976 13666 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.578771 13591 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:08.614176 13591 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.615902 13666 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:08.617426 11927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 482) took 2337 ms (client timeout 1955 ms). Trace:
W20260709 12:13:08.617911 11927 rpcz_store.cc:269] 0709 12:13:06.279608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.281272 (+  1664us) service_pool.cc:224] Handling call
0709 12:13:07.719913 (+1438641us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:08.617231 (+897318us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b5400650254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:08.617353 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26537216}
I20260709 12:13:08.618556 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.619191 11252 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.619524 13697 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.630664 11252 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:13:08.631400 11178 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:08.633620 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:08.635288 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 101281us
I20260709 12:13:08.583092 11323 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:08.636161 11252 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.605839 13673 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.638375 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:08.639998 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.605926 13373 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.641131 11328 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:08.638527 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62125.2us
I20260709 12:13:08.639099 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.642850 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:08.645257 11254 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.635069 13697 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.642091 13628 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:13:08.650350 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.672863 11325 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:08.650897 13371 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:08.674101 13628 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:08.674521 13628 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:08.675470 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.676168 11324 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.675541 13628 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:08.678323 13628 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.644160 13373 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.651187 13591 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.682668 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.682255 13602 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:08.653728 13446 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:08.654003 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.644552 13714 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:08.654982 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:08.655139 11930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 408) took 2290 ms (client timeout 1738 ms). Trace:
I20260709 12:13:08.658883 13637 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:08.660091 13554 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:08.660295 13596 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:08.663475 13712 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
I20260709 12:13:08.664739 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.700359 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.665015 11248 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:08.653419 13619 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.666083 13732 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:08.666313 13655 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.666774 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:08.667812 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:08.668592 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.670503 11329 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:08.581768 13618 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.667191 13673 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6.
I20260709 12:13:08.651489 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.685585 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.705602 11250 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 6. Ops: []
I20260709 12:13:08.687726 13480 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:08.689530 11324 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:08.691439 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.683750 13602 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:08.709091 11176 raft_consensus.cc:1217] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Deduplicated request from leader. Original: 2.3->[3.4-5.6]   Dedup: 3.5->[5.6-5.6]
I20260709 12:13:08.692425 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.693575 11254 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.680998 13622 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:08.694876 13600 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.697180 13634 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:08.701545 11257 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:08.701524 11255 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:08.702294 13732 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.703665 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:08.698001 11930 rpcz_store.cc:269] 0709 12:13:06.365022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.502041 (+137019us) service_pool.cc:224] Handling call
0709 12:13:08.654937 (+2152896us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540034f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:08.655070 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2857155840}
I20260709 12:13:08.704294 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.706699 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.706749 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:08.708739 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.726630 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:08.709717 11176 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [3.4-5.6]
I20260709 12:13:08.709084 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:08.734417 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.734905 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.736039 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:08.711421 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 67286.9us
I20260709 12:13:08.711004 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.737080 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:08.737259 11254 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.738354 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.738915 11254 raft_consensus.cc:2376] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5. Re-sending same reply.
I20260709 12:13:08.739707 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.733454 13480 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:13:08.740584 11254 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.740950 13480 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:08.741320 13480 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.714651 13600 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.715379 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.749498 13604 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:08.750399 11255 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.751282 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.751720 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 317) took 1841 ms. Trace:
I20260709 12:13:08.751888 11255 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:08.752128 11177 rpcz_store.cc:276] 0709 12:13:06.910250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.910641 (+   391us) service_pool.cc:224] Handling call
0709 12:13:06.951921 (+ 41280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:08.147674 (+1195753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.751464 (+603790us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002e40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:08.751658 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2469453056}
I20260709 12:13:08.752954 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.715278 13602 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.749321 13680 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:08.754570 13602 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.755131 13673 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:08.708845 13590 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:13:08.716464 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:08.706236 13733 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:08.761580 11324 raft_consensus.cc:2410] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:13:08.721313 11325 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.708976 13637 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:08.729161 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:08.693199 13714 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.739514 13563 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:08.741884 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.698325 13335 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.766646 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:08.736438 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.737191 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.743673 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.741420 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.746615 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.768985 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:08.768107 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.769683 11252 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.747797 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.770547 11174 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.747663 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.771234 11329 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:08.749297 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.720072 13289 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:08.774196 13680 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:08.710682 13481 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:13:08.774994 13698 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:08.753571 11255 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:08.753362 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:08.709163 13635 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.762174 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:08.731454 13732 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.761479 13733 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.789435 11181 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:08.763720 13622 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:08.702950 11328 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:08.773723 11254 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.773784 11250 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.774622 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.771747 13697 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:08.756474 13607 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:08.715896 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.796648 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:08.759479 13173 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.763608 13578 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.791697 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.792279 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:08.767148 13673 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:08.791713 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:08.793066 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.774919 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:08.775121 13446 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:08.768440 13714 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:08.778080 13666 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:13:08.796674 11257 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
W20260709 12:13:08.778553 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 58639.8us
I20260709 12:13:08.795306 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.798537 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.791008 13622 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:08.800057 11325 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:13:08.800400 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.785431 13554 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:08.786341 13591 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.784358 13698 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.798674 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:08.925179 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.808385 11934 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:13:08.912493 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.789242 13604 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:13:08.814469 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.791421 13733 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:08.926862 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:08.802881 11932 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:13:08.928586 11329 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:08.933154 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:08.933368 11252 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:08.926622 11250 raft_consensus.cc:2376] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5. Re-sending same reply.
I20260709 12:13:08.875713 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:13:08.934558 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 159690us
W20260709 12:13:08.937105 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0964662989999852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
I20260709 12:13:08.937359 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:08.937563 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0910662989999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 12:13:08.938100 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:08.938158 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0910662989999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 12:13:08.940411 11331 raft_consensus.cc:2410] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:08.796221 13640 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:08.874955 13735 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:13:08.935381 11932 pending_rounds.cc:85] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 4
I20260709 12:13:08.947476 11932 pending_rounds.cc:107] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260709 12:13:08.948347 11932 raft_consensus.cc:2917] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 12:13:08.949738 11932 raft_consensus.cc:2889] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:08.945993 11325 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:08.951014 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:08.930919 13622 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.802256 13697 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.957260 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:08.969754 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.969589 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.934427 13446 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:08.971760 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:08.939746 13627 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:08.942205 13703 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:08.969403 13554 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:08.979403 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:08.980365 11252 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:13:08.981289 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.981904 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:08.982125 13650 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:09.010497 11173 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:08.969009 13640 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:08.873993 13578 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:08.982853 13591 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:08.983400 13698 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.025115 13578 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:08.986267 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:09.028980 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:08.974758 13628 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:09.029999 13628 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:09.030527 13628 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.032399 13671 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:09.032859 13671 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:09.033916 13540 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:09.034168 11252 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:09.035298 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:09.035794 13335 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:09.036032 13600 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:09.009828 13289 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:08.976800 11930 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:09.019511 13563 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:08.974680 13599 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:08.968842 13403 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:13:08.955632 13708 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:08.990111 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:09.038407 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:09.039018 11248 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:09.038882 13708 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:09.039245 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49263.4us
I20260709 12:13:09.040238 11248 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:09.041010 13708 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:09.019555 13627 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:09.042255 13627 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.044411 13627 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:09.044487 13600 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:09.020285 13675 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.041549 11248 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:13:09.045621 13675 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.047181 11248 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:08.992825 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 384) took 1026 ms. Trace:
I20260709 12:13:08.997144 13596 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:09.047791 11322 rpcz_store.cc:276] 0709 12:13:07.966157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.001023 (+ 34866us) service_pool.cc:224] Handling call
0709 12:13:08.001295 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:08.194954 (+193659us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:08.195334 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:13:08.314176 (+118842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:08.314227 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:08.314669 (+   442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.991977 (+677308us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b1800336520. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419
0709 12:13:08.992511 (+   534us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54001c48d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:08.992558 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:08.992753 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1068,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":805,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":33038,"spinlock_wait_cycles":470383616,"thread_start_us":118031,"threads_started":1,"wal-append.queue_time_us":265369}
I20260709 12:13:08.998479 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1256) took 1103 ms. Trace:
I20260709 12:13:08.987525 11174 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:09.005326 11323 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:09.006901 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:09.020212 11325 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:08.809803 13173 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.035882 11252 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.049510 11330 rpcz_store.cc:276] 0709 12:13:07.895124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.899808 (+  4684us) service_pool.cc:224] Handling call
0709 12:13:07.900190 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:08.234173 (+333983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:08.234219 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:08.234281 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:08.234334 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.998227 (+763893us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b540042b8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:08.998275 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:08.998416 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767510272}
I20260709 12:13:09.003280 13673 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.051167 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:09.047477 13563 raft_consensus.cc:1064] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:13:09.075300 11322 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:08.798930 13480 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.022409 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:09.061738 13628 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.062937 13738 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:09.064226 13166 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:09.069777 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.045732 13373 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.073884 13173 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:09.027276 13637 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.041535 13708 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:09.089371 11322 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:09.101735 11322 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:09.090899 11330 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:09.091989 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.054687 13634 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.130582 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.135351 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.135715 11331 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:09.136804 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 343754us
W20260709 12:13:09.145301 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.147112 11177 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:09.070096 13732 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.148653 11180 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:09.116297 13604 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.163209 11252 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:09.164238 11248 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:09.165062 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:09.164943 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:09.170991 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.177328 11178 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:09.175410 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 226128us
I20260709 12:13:09.076066 13671 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.146497 13590 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:13:09.138846 13634 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.156945 13697 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.250995 11248 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:09.169234 13446 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.185536 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:09.274138 11252 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:09.224026 11257 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:09.276615 11257 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:13:09.277133 11257 pending_rounds.cc:85] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 4
I20260709 12:13:09.277484 11257 pending_rounds.cc:107] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:13:09.195016 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:09.278363 11255 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:09.185045 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:09.279901 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94994.3us
I20260709 12:13:09.280870 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:09.212782 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.282176 11257 raft_consensus.cc:2889] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:09.224596 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.211565 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:09.287786 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.215497 13718 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.865s	user 0.003s	sys 0.003s
I20260709 12:13:09.300348 11256 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:13:09.303614 13510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.007s	sys 0.014s
I20260709 12:13:09.300199 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:09.170990 13604 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.308091 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.222355 13675 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:09.310860 11252 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:13:09.208038 13366 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:09.312928 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:09.314177 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:09.308058 13510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.007s	sys 0.014s
I20260709 12:13:09.316015 13366 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:09.316471 11935 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:09.318827 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.320237 13619 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:09.322566 11249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1408) took 2940 ms (client timeout 2552 ms). Trace:
I20260709 12:13:09.174589 13742 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.147s
I20260709 12:13:09.319096 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.330807 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.305419 13743 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:09.304949 13637 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:09.336830 11249 rpcz_store.cc:269] 0709 12:13:06.382063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.382206 (+   143us) service_pool.cc:224] Handling call
0709 12:13:09.322324 (+2940118us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54002a58d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:09.322461 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3667900800}
I20260709 12:13:09.338320 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:09.342859 11252 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:13:09.345443 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:09.326893 13602 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.572s	user 0.005s	sys 0.006s
I20260709 12:13:09.346925 11174 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:09.348353 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 218814us
I20260709 12:13:09.327144 13554 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:09.353986 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.355898 11933 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:09.307518 13627 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:13:09.359490 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1260) took 1463 ms. Trace:
I20260709 12:13:09.343598 13628 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:09.345333 13289 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:09.364569 11249 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:13:09.345757 13743 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.365499 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:09.378345 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.379323 11174 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:09.367588 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.369118 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.369311 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:09.383783 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.360013 11326 rpcz_store.cc:276] 0709 12:13:07.896538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.901807 (+  5269us) service_pool.cc:224] Handling call
0709 12:13:07.902240 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:08.083979 (+181739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:08.084012 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:08.106289 (+ 22277us) log.cc:844] Serialized 26 byte log entry
0709 12:13:08.292574 (+186285us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:08.292628 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.292727 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:09.214680 (+921953us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:09.348733 (+134053us) raft_consensus.cc:1685] finished
0709 12:13:09.348764 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.359412 (+ 10648us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109167,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":178670336,"thread_start_us":310604,"threads_started":2,"wal-append.queue_time_us":204273}
I20260709 12:13:09.384418 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.357728 13554 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:09.386646 11174 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:09.383484 11933 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:09.390017 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.397804 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:09.398553 11326 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
W20260709 12:13:09.398455 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 84543.6us
I20260709 12:13:09.399755 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:09.389392 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:09.399937 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:09.401345 11249 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:09.402247 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:09.382172 13563 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.403321 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:09.400198 11174 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:09.376979 13373 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:09.409914 13604 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:09.410122 13714 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.382714 13743 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:09.406406 13732 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:09.388170 13675 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.393363 13554 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.415925 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.442131 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:09.420086 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:09.443801 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:09.445029 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:09.417447 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:09.434309 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.393036 13366 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.437212 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:09.407145 11326 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 3: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:09.457746 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.459946 11328 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:09.446938 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.465123 11328 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:09.433809 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.413215 13289 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:09.447077 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:09.472549 13289 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:09.472940 13601 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:09.473798 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.474417 13742 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.475098 13742 consensus_queue.cc:1243] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:13:09.475207 13590 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.476043 13590 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.476454 13742 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:09.431418 13739 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:09.481535 13738 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:09.482070 13738 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:09.484073 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.484869 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.448135 11327 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:09.441586 13756 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:13:09.449862 13748 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:13:09.450533 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:09.455291 13747 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:13:09.455751 11179 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:09.438069 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.461971 11935 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:09.463343 13732 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:09.471558 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1268) took 1019 ms. Trace:
I20260709 12:13:09.473664 13373 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:09.486572 13590 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:09.497995 13732 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.498327 13590 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:09.498651 13732 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:09.498693 13373 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:09.499265 13373 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:09.499610 13590 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.500262 13373 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.500998 13590 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.472977 13289 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.504139 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:09.505319 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.478758 13742 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.485733 13739 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:09.542989 11179 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:13:09.447381 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:09.462659 13676 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:09.462133 13685 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:09.468092 13366 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:09.508055 13618 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.478138 13601 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:09.510179 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 72168.9us
I20260709 12:13:09.510236 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:09.503372 11325 rpcz_store.cc:276] 0709 12:13:08.452150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.053487 (+601337us) service_pool.cc:224] Handling call
0709 12:13:09.078996 (+ 25509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:09.087068 (+  8072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:09.087115 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:09.103348 (+ 16233us) log.cc:844] Serialized 26 byte log entry
0709 12:13:09.301115 (+197767us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:09.301171 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.301270 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:09.471340 (+170070us) raft_consensus.cc:1685] finished
0709 12:13:09.471363 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.471498 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13465,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":20048,"spinlock_wait_cycles":49408,"thread_start_us":144805,"threads_started":1,"wal-append.queue_time_us":207731}
W20260709 12:13:09.512674 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1657) took 1244 ms (client timeout 1513 ms). Trace:
W20260709 12:13:09.513430 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.502065 13607 consensus_queue.cc:1175] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "786851d4227e4dd89b75ccdea0b1f03b" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 3 index: 5 } }
I20260709 12:13:09.509347 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:09.518065 13655 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:09.482985 13738 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.519946 13714 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:09.457248 13446 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:09.456662 13619 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.591569 13738 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:09.522797 11326 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:13:09.592236 11326 pending_rounds.cc:85] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 4
I20260709 12:13:09.592526 11326 pending_rounds.cc:107] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:13:09.592795 11326 raft_consensus.cc:2889] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:09.523577 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:09.594033 11257 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.594405 11325 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:13:09.595822 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.596475 13698 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.597030 13698 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:09.597821 13166 consensus_queue.cc:1175] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "786851d4227e4dd89b75ccdea0b1f03b" responder_term: 6 status { last_received { term: 6 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 6 index: 8 } }
I20260709 12:13:09.599366 13698 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:09.486697 11253 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:09.490191 13713 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:09.600756 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:09.601416 11253 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:09.602347 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:09.603104 11253 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:09.603577 11257 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:09.604007 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:09.503611 11328 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:09.570416 11323 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:13:09.572988 11251 rpcz_store.cc:269] 0709 12:13:08.268523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.271780 (+  3257us) service_pool.cc:224] Handling call
0709 12:13:09.512456 (+1240676us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400069354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:09.512610 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697002368}
I20260709 12:13:09.577040 11933 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:09.572419 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:09.634647 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 62318.3us
I20260709 12:13:09.635385 11253 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:09.582212 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 134886us
I20260709 12:13:09.585397 13623 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:09.522318 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:09.501549 13373 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.605162 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:09.612011 13601 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.571403 13732 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.529940 11935 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
W20260709 12:13:09.622988 13733 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.682s	user 0.012s	sys 0.000s
I20260709 12:13:09.578413 13675 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:13:09.625352 13655 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:09.628456 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.628628 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.637064 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:09.715035 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:09.716179 11253 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:09.637233 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:09.717435 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:09.644526 11328 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:09.691920 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:09.653409 13623 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.644260 13685 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.700840 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:09.708549 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:09.635527 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:09.763441 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 128007us
W20260709 12:13:09.768051 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 130908us
I20260709 12:13:09.769121 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.734880 11933 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:09.736373 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:09.654269 13627 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:09.746886 13623 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:09.748427 13698 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:09.748538 11932 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:09.642987 13676 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:09.749503 13622 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.774s	user 0.009s	sys 0.002s
I20260709 12:13:09.750238 11254 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:13:09.751570 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:09.791282 11325 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:09.754267 11326 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:09.754957 13713 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:09.755872 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:09.797910 11328 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.770612 11253 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:09.770989 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:09.772039 11174 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:09.781806 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.788167 11323 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:09.796589 11933 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:13:09.654649 13743 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.781919 13673 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:09.745335 13554 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.776500 13714 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:09.817425 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:09.643968 13739 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.808075 13446 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:09.812744 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.597s (SENT) [suppressed 20 similar messages]
I20260709 12:13:09.820639 11253 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:13:09.813803 13635 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:09.775952 13627 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:09.815840 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:09.826462 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.797122 13590 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:09.803725 13698 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:09.779932 13637 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:13:09.831526 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:09.843830 11324 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:09.711609 13733 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.818245 13673 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.777433 13685 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:09.852197 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.839938 13680 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:09.830155 13627 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.816834 13714 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:09.806125 13622 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.803259 13676 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.808660 13713 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:09.871611 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.869954 11175 raft_consensus.cc:2368] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5.
I20260709 12:13:09.851238 13619 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:09.866030 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.878167 11253 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:09.857332 13673 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:09.869890 13734 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.108s	user 0.003s	sys 0.005s
W20260709 12:13:09.888803 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.889917 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1276) took 1212 ms. Trace:
I20260709 12:13:09.828302 13446 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.869362 13714 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.869385 11328 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:09.831470 13635 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.906615 13743 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:09.907421 13743 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:09.907806 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1882) took 1162 ms. Trace:
I20260709 12:13:09.907955 13743 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:09.908458 11182 rpcz_store.cc:276] 0709 12:13:08.745191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.906632 (+1161441us) service_pool.cc:224] Handling call
0709 12:13:09.907053 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.907316 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:09.907362 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:09.907425 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:09.907493 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.907572 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.907729 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:09.855455 13590 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:13:09.914954 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1870) took 1225 ms. Trace:
I20260709 12:13:09.868058 13618 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:13:09.915614 11178 rpcz_store.cc:276] 0709 12:13:08.685211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.692146 (+  6935us) service_pool.cc:224] Handling call
0709 12:13:09.909894 (+1217748us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54000d9b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:09.910073 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074446080}
I20260709 12:13:09.903329 11330 rpcz_store.cc:276] 0709 12:13:08.677726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.444295 (+766569us) service_pool.cc:224] Handling call
0709 12:13:09.465472 (+ 21177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.466217 (+   745us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:09.529563 (+ 63346us) log.cc:844] Serialized 14 byte log entry
0709 12:13:09.889466 (+359903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:09.889533 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:09.889593 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.889714 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.889851 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112269,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":52,"thread_start_us":351189,"threads_started":1}
I20260709 12:13:09.855314 13675 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:13:09.924942 11182 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:09.859246 13655 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:09.936106 11178 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:09.918244 11930 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:09.909176 13743 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:09.962013 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.947364 13690 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:09.966814 11176 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:09.966816 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.938356 13732 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:09.977665 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:09.965960 13690 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:09.978485 11330 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
W20260709 12:13:09.980458 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 167749us
W20260709 12:13:09.978506 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 207576us
I20260709 12:13:09.923830 13698 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:09.988790 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 147295us
I20260709 12:13:09.985168 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:09.994123 13742 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:09.994719 13742 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:09.995576 13742 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:09.996773 13742 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.998142 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1884) took 1252 ms. Trace:
I20260709 12:13:09.998528 11177 rpcz_store.cc:276] 0709 12:13:08.745716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.753089 (+  7373us) service_pool.cc:224] Handling call
0709 12:13:09.997916 (+1244827us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54000d7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:09.998058 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825446784}
I20260709 12:13:09.999720 13289 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.003438 13591 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:10.003890 13591 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:10.004784 13591 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.036688 13591 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:10.038684 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:10.039549 11256 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:10.007071 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 326) took 1264 ms. Trace:
I20260709 12:13:10.041828 11175 rpcz_store.cc:276] 0709 12:13:08.743066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.022159 (+279093us) service_pool.cc:224] Handling call
0709 12:13:10.006830 (+984671us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b5400347bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:10.006998 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1823829760}
W20260709 12:13:10.042435 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1869) took 1357 ms (client timeout 1597 ms). Trace:
I20260709 12:13:10.043224 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:10.044210 11175 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:10.008002 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1413) took 1219 ms. Trace:
I20260709 12:13:10.045087 11254 rpcz_store.cc:276] 0709 12:13:08.788326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.795086 (+  6760us) service_pool.cc:224] Handling call
0709 12:13:10.007837 (+1212751us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54001cf7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:10.007943 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893006592}
I20260709 12:13:10.046479 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:10.047924 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:10.050275 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.051633 11929 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:10.052213 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 496) took 1256 ms. Trace:
I20260709 12:13:10.052659 11929 rpcz_store.cc:276] 0709 12:13:08.795947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.798985 (+  3038us) service_pool.cc:224] Handling call
0709 12:13:10.050062 (+1251077us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:10.052133 (+  2071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5983616}
I20260709 12:13:10.054870 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:10.011708 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:10.055949 11173 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:10.056331 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:10.015455 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:10.057101 11253 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:10.058009 13289 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:10.058810 13289 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:10.059307 13289 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:10.060207 13289 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.016321 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:10.049521 11254 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:10.018788 13648 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:10.022068 13739 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:10.022553 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.022116 13622 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:10.024523 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:10.028887 13733 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:09.994234 13713 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:10.063874 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:10.064213 11173 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:10.062420 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1432) took 1033 ms. Trace:
I20260709 12:13:10.034626 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:10.067440 11255 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:10.055099 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 324) took 1312 ms. Trace:
I20260709 12:13:10.068816 11254 rpcz_store.cc:276] 0709 12:13:09.028792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.048956 (+1020164us) service_pool.cc:224] Handling call
0709 12:13:10.049297 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.062116 (+ 12819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.062339 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:10.069427 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.009445 13767 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:13:10.032120 13693 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:10.066465 11181 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:10.043074 11179 rpcz_store.cc:269] 0709 12:13:08.684891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.685328 (+   437us) service_pool.cc:224] Handling call
0709 12:13:10.042142 (+1356814us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54000c0b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:10.042347 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1724189952}
I20260709 12:13:10.057538 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:10.069186 11253 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:10.068140 11174 rpcz_store.cc:276] 0709 12:13:08.742488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.595271 (+852783us) service_pool.cc:224] Handling call
0709 12:13:09.595593 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.808407 (+212814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.054777 (+246370us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54000d8c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:10.055022 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405172992}
I20260709 12:13:10.084636 11179 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:13:10.085157 11253 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:13:10.085861 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1436) took 1020 ms. Trace:
I20260709 12:13:10.090443 11175 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:10.087452 11253 rpcz_store.cc:276] 0709 12:13:09.065396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.084653 (+1019257us) service_pool.cc:224] Handling call
0709 12:13:10.084943 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.085580 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.085806 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:10.095957 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1433) took 1066 ms. Trace:
I20260709 12:13:10.096590 11255 rpcz_store.cc:276] 0709 12:13:09.029067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.067959 (+1038892us) service_pool.cc:224] Handling call
0709 12:13:10.095213 (+ 27254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.095476 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.095515 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.095573 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:10.095632 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.095708 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.095879 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:10.107978 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:10.108417 13738 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.005s
I20260709 12:13:10.108752 11330 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:10.110564 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:10.079293 13769 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:13:09.989032 13554 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:10.118257 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.124166 11330 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:10.091413 13693 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:10.157980 13738 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.182713 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1428) took 1205 ms. Trace:
I20260709 12:13:10.183339 11249 rpcz_store.cc:276] 0709 12:13:08.977756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.402038 (+424282us) service_pool.cc:224] Handling call
0709 12:13:09.402426 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:09.412735 (+ 10309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:09.412790 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:09.435040 (+ 22250us) log.cc:844] Serialized 52 byte log entry
0709 12:13:09.839771 (+404731us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:09.839812 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.839884 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:10.182345 (+342461us) raft_consensus.cc:1685] finished
0709 12:13:10.182399 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.182632 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5568,"raft.queue_time_us":9905,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":961,"thread_start_us":376854,"threads_started":1,"wal-append.queue_time_us":516451}
I20260709 12:13:10.186852 13671 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:10.163664 13676 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:10.164120 13779 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:10.172610 13635 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:10.198666 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1893) took 1481 ms. Trace:
I20260709 12:13:10.175560 13685 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:10.175143 13446 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:10.200079 11331 rpcz_store.cc:276] 0709 12:13:08.717586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.443294 (+725708us) service_pool.cc:224] Handling call
0709 12:13:09.466177 (+ 22883us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.466381 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:09.575439 (+109058us) log.cc:844] Serialized 14 byte log entry
0709 12:13:09.928436 (+352997us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.198261 (+269825us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.198328 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:10.198378 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.198463 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.198604 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75325,"prepare.queue_time_us":139771,"prepare.run_cpu_time_us":6542,"prepare.run_wall_time_us":11332,"raft.queue_time_us":78158,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":7250,"thread_start_us":619974,"threads_started":2,"wal-append.queue_time_us":537712}
W20260709 12:13:10.207850 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 216415us
W20260709 12:13:10.219033 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.219898 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 185659us
W20260709 12:13:10.222801 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 165327us
I20260709 12:13:10.223810 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:10.180677 13738 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.208012 13480 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:10.208583 13779 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:10.170158 13690 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.236856 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:10.240129 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:10.210995 13685 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:10.242864 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.245929 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 500) took 1168 ms. Trace:
I20260709 12:13:10.246194 11935 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:13:10.251057 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 881941us
I20260709 12:13:10.251974 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:10.252899 13619 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:10.253360 13619 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:10.230270 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.261108 11256 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:10.251482 11927 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:10.267472 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 493) took 1480 ms. Trace:
I20260709 12:13:10.267866 11927 rpcz_store.cc:276] 0709 12:13:08.787152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.789129 (+  1977us) service_pool.cc:224] Handling call
0709 12:13:10.251031 (+1461902us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:10.251259 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.267118 (+ 15859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.267390 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10705152}
I20260709 12:13:10.269193 13634 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:10.269804 13753 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:10.270799 13740 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:10.271303 13740 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:10.271713 13740 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:10.274142 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.275199 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.276682 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.280927 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.286461 11256 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:10.287448 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1441) took 1218 ms. Trace:
I20260709 12:13:10.254748 13635 raft_consensus.cc:2764] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:13:10.289511 11932 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:10.290777 11256 rpcz_store.cc:276] 0709 12:13:09.069111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.285933 (+1216822us) service_pool.cc:224] Handling call
0709 12:13:10.286276 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.287151 (+   875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.287386 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:10.294267 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.297951 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.255708 13781 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:10.208407 13693 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:10.299685 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59622us
I20260709 12:13:10.276172 13690 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.307314 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.301169 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:10.313176 11933 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.315363 11256 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:10.386001 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 280) took 1245 ms. Trace:
I20260709 12:13:10.386516 11256 rpcz_store.cc:276] 0709 12:13:09.140990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.297780 (+1156790us) service_pool.cc:224] Handling call
0709 12:13:10.312726 (+ 14946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.385537 (+ 72811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.385918 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:10.319581 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1282) took 1097 ms. Trace:
I20260709 12:13:10.387548 11324 rpcz_store.cc:276] 0709 12:13:09.221793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.308804 (+ 87011us) service_pool.cc:224] Handling call
0709 12:13:10.319350 (+1010546us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b540026c5d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:10.319511 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108043136}
I20260709 12:13:10.322296 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.324859 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:10.246455 11928 rpcz_store.cc:276] 0709 12:13:09.077038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.078646 (+  1608us) service_pool.cc:224] Handling call
0709 12:13:09.153309 (+ 74663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.174209 (+ 20900us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:09.304184 (+129975us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.245322 (+941138us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b18003fe9e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:13:10.245485 (+   163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.245541 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:10.245600 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.245703 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.245858 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2465,"prepare.queue_time_us":157633,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":893566,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":109,"spinlock_wait_cycles":6366080,"thread_start_us":140072,"threads_started":2,"wal-append.queue_time_us":106655}
I20260709 12:13:10.327848 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.332957 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.312105 13691 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:13:10.335494 13619 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.312204 13779 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:10.336437 13766 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.504s	user 0.001s	sys 0.000s
I20260709 12:13:10.401175 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1422) took 1528 ms. Trace:
I20260709 12:13:10.401566 11248 rpcz_store.cc:276] 0709 12:13:08.872662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.308352 (+435690us) service_pool.cc:224] Handling call
0709 12:13:09.308666 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:09.309009 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:09.309039 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:09.316618 (+  7579us) log.cc:844] Serialized 26 byte log entry
0709 12:13:09.433284 (+116666us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:09.433336 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.433416 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:10.335869 (+902453us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:10.400874 (+ 65005us) raft_consensus.cc:1685] finished
0709 12:13:10.400898 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.401108 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1229,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":16329,"spinlock_wait_cycles":32610688,"thread_start_us":78783,"threads_started":1,"wal-append.queue_time_us":439601}
I20260709 12:13:10.278327 13640 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:13:10.278374 13371 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 12:13:10.300112 13623 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.005s
I20260709 12:13:10.397986 13779 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:10.405658 13779 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:10.405386 13623 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.396318 13666 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:10.407859 13623 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:10.394698 13739 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:10.408546 11254 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:13:10.408684 13739 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:10.409727 13739 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.410792 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1435) took 1381 ms. Trace:
I20260709 12:13:10.411331 11929 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:10.411159 11254 rpcz_store.cc:276] 0709 12:13:09.029605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.095277 (+1065672us) service_pool.cc:224] Handling call
0709 12:13:10.095595 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.409002 (+313407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.410548 (+  1546us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54002a67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:10.410735 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719143168}
I20260709 12:13:10.412029 11933 raft_consensus.cc:2410] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:10.412823 11254 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:10.412892 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:10.413365 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.413554 11933 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:10.339747 13693 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:10.303406 13781 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:10.340668 11250 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:10.354588 13738 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:10.354923 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.356492 13740 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.360100 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 327) took 1616 ms. Trace:
I20260709 12:13:10.424345 11181 rpcz_store.cc:276] 0709 12:13:08.743406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.821266 (+1077860us) service_pool.cc:224] Handling call
0709 12:13:10.066210 (+244944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.091746 (+ 25536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.359907 (+268161us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54000c0b50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:10.360037 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520245504}
I20260709 12:13:10.361716 11251 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:10.368211 13480 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.371191 13714 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.375391 11257 raft_consensus.cc:1217] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-8.5]   Dedup: 4.3->[6.4-8.5]
I20260709 12:13:10.436124 11257 pending_rounds.cc:85] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 3
I20260709 12:13:10.436465 11257 pending_rounds.cc:107] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:13:10.379559 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78459.5us
W20260709 12:13:10.389530 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64755.3us
I20260709 12:13:10.436779 11257 raft_consensus.cc:2889] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:10.437023 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 282) took 1076 ms. Trace:
I20260709 12:13:10.443166 11251 rpcz_store.cc:276] 0709 12:13:09.360986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.361231 (+1000245us) service_pool.cc:224] Handling call
0709 12:13:10.361528 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.436660 (+ 75132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.436936 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:10.389859 11935 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:10.391489 11330 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:10.393976 11322 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:10.309100 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:10.398118 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:10.448355 11331 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:10.448917 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 325) took 1706 ms. Trace:
I20260709 12:13:10.449359 11180 rpcz_store.cc:276] 0709 12:13:08.742719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.628967 (+886248us) service_pool.cc:224] Handling call
0709 12:13:09.629418 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.850579 (+221161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:09.850628 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:09.850672 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:09.850730 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.448720 (+597990us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54002b7854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:10.448762 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.448863 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508135168}
I20260709 12:13:10.440358 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1438) took 1373 ms. Trace:
I20260709 12:13:10.452298 11255 rpcz_store.cc:276] 0709 12:13:09.066952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.158347 (+1091395us) service_pool.cc:224] Handling call
0709 12:13:10.168734 (+ 10387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.169021 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:10.222038 (+ 53017us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.439839 (+217801us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.439913 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:10.439988 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.440085 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.440270 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1337,"prepare.queue_time_us":90766,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":83,"thread_start_us":260829,"threads_started":2}
I20260709 12:13:10.392994 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:10.306703 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 83007.1us
I20260709 12:13:10.343211 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 499) took 1505 ms. Trace:
I20260709 12:13:10.411113 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.427107 13510 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:10.427508 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.405509 13563 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:10.430070 11256 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:10.439209 13371 consensus_queue.cc:1243] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:13:10.399688 13627 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.000s
I20260709 12:13:10.369436 13673 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.412477 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.398869 11327 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:10.452191 11933 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:10.435247 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:10.516913 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 81812.1us
I20260709 12:13:10.447918 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:10.519542 11324 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:10.473484 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.418473 13173 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:10.439704 13676 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
W20260709 12:13:10.475328 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 166280us
I20260709 12:13:10.477115 11932 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:10.476009 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:10.486467 13634 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:10.532792 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:10.534718 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:10.535959 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:10.430519 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1427) took 1465 ms. Trace:
I20260709 12:13:10.433350 13781 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:10.489910 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 323) took 1747 ms. Trace:
I20260709 12:13:10.551548 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.551970 11176 rpcz_store.cc:276] 0709 12:13:08.742247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.526730 (+784483us) service_pool.cc:224] Handling call
0709 12:13:09.527021 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.967270 (+440249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.485469 (+518199us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54001f40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:10.489812 (+  4343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010998016}
I20260709 12:13:10.491807 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:10.509141 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:10.503134 13563 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:10.512374 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:10.513022 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.522024 13373 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:10.524479 13607 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:10.525530 13481 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:10.526487 13733 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:10.516240 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.530953 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:10.531953 13690 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:10.511247 13446 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:13:10.563329 13289 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:13:10.549892 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 295447us
I20260709 12:13:10.505235 11930 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:10.487972 13628 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.553002 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:10.570200 13648 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:10.536741 11252 rpcz_store.cc:276] 0709 12:13:08.964884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.356711 (+391827us) service_pool.cc:224] Handling call
0709 12:13:09.357064 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:09.383152 (+ 26088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:09.383177 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:09.383554 (+   377us) log.cc:844] Serialized 26 byte log entry
0709 12:13:09.779910 (+396356us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:10.164788 (+384878us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.331885 (+167097us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b1800425140. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:10.420966 (+ 89081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.421075 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:10.421132 (+    57us) raft_consensus.cc:1685] finished
0709 12:13:10.429583 (+  8451us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.430460 (+   877us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":440156,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2481,"raft.queue_time_us":65047,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":22386,"spinlock_wait_cycles":19629824,"thread_start_us":918539,"threads_started":3,"wal-append.queue_time_us":497469}
W20260709 12:13:10.511992 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.557947 11177 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:10.559718 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:10.511077 13166 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.577257 13686 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:10.475113 11926 rpcz_store.cc:276] 0709 12:13:08.810569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.868304 (+ 57735us) service_pool.cc:224] Handling call
0709 12:13:10.218015 (+1349711us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:10.315779 (+ 97764us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12175616}
W20260709 12:13:10.532897 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56965us
I20260709 12:13:10.584975 13373 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:10.618806 11254 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:13:10.510988 13627 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.532155 13173 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:10.484486 11323 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:10.440471 13480 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.565366 13635 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:10.516175 13785 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:13:10.563189 13554 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:10.532069 13708 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:13:10.570899 13676 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:13:10.564184 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51943.7us
I20260709 12:13:10.612179 11324 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
W20260709 12:13:10.711329 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.574149 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1437) took 1507 ms. Trace:
I20260709 12:13:10.613253 13666 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:13:10.582353 13622 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:10.578480 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47599.4us
I20260709 12:13:10.607363 13628 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.588522 13698 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.623993 13366 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:10.643369 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:10.646158 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:10.737046 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.737578 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.608089 13686 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:10.608716 13634 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:10.589613 13289 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:10.654726 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.620231 13789 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.619170 13373 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:10.620040 13685 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:10.701138 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.704627 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:10.797196 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92688.3us
I20260709 12:13:10.566538 13680 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:13:10.711797 11253 rpcz_store.cc:276] 0709 12:13:09.066275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.094646 (+1028371us) service_pool.cc:224] Handling call
0709 12:13:10.125910 (+ 31264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.126113 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:10.211124 (+ 85011us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.573765 (+362641us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.573825 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:10.573879 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.573955 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.574090 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9993,"prepare.queue_time_us":116228,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":117,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":95,"thread_start_us":411015,"threads_started":2}
I20260709 12:13:10.616619 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:10.614267 13166 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.663295 11928 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEARNER]: Advancing to term 6
I20260709 12:13:10.721575 11324 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:10.722086 11323 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:10.722505 11327 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:13:10.740126 11252 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:13:10.904744 11252 pending_rounds.cc:85] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 2
I20260709 12:13:10.905086 11252 pending_rounds.cc:107] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:10.699278 13635 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:10.637015 13563 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.754252 13686 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:10.712944 13739 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.703649 13554 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:10.717335 13622 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:10.706506 13371 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:13:10.905400 11252 raft_consensus.cc:2889] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:10.917218 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:10.754053 13740 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:10.769117 13289 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:10.856140 13373 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.752796 13714 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:10.934090 11253 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
W20260709 12:13:10.935491 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 426591us
I20260709 12:13:10.739638 13698 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:10.919469 13635 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:10.754174 13366 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:10.756853 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 142275us
I20260709 12:13:10.970753 11253 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:10.973193 11253 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:13:10.929736 13554 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:10.935190 13607 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.944496 13673 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:10.802217 13789 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:10.968674 13563 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:10.904217 13685 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.019438 13809 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:10.930173 13807 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 12:13:10.931515 13795 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 12:13:10.979022 13708 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.965952 13782 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:13:10.920219 13173 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.003587 13792 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:11.046959 11255 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:11.045845 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 439194us
I20260709 12:13:10.935642 13739 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.010807 13446 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:11.027577 13810 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.012311 13798 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.057343 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:11.063288 11222 leader_election.cc:336] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.707s (SENT)
I20260709 12:13:11.028471 13618 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.064582 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:11.052618 13801 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 12:13:11.077209 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 107721us
I20260709 12:13:11.012995 13607 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.052500 13743 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:11.059000 13806 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:13:11.084426 13805 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 12:13:11.085852 13480 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:11.086261 13797 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:13:11.086637 13812 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.145787 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 171456us
I20260709 12:13:11.051191 13708 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.091589 13166 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.092181 13813 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.093055 13799 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:13:11.163039 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.163726 11935 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:13:11.093986 13802 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.443s	user 0.000s	sys 0.001s
I20260709 12:13:11.072836 13792 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:11.071753 13798 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.056109 13627 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.101503 13814 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:11.101670 13619 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.063556 13446 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.065539 13810 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.182767 13817 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.183125 13789 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.183436 13817 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.184449 13810 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.106283 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:11.107113 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:11.186893 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 79880.4us
I20260709 12:13:11.187830 13765 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:11.189023 13765 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.122320 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59100.7us
W20260709 12:13:11.190066 11222 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.738s (SENT)
I20260709 12:13:11.190348 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 502) took 1711 ms. Trace:
I20260709 12:13:11.191619 13166 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:11.192891 13708 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.194051 13627 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:11.194536 13627 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:11.030941 13796 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:13:11.195456 13627 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.196918 13627 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.210850 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:11.211407 13371 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.211656 13739 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.211970 13371 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.212409 13808 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:13:11.212990 13366 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.200685 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:11.167325 13813 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:11.214720 13803 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:13:11.211787 11253 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:11.177017 13697 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.173271 13814 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:11.088236 13685 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.147913 11900 leader_election.cc:400] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5.
I20260709 12:13:11.235426 11900 leader_election.cc:403] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:13:11.060200 13690 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:13:11.153199 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:11.158901 11329 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:11.127274 13782 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.162130 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.135763 13481 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.074995 13618 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.166260 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.190805 11931 rpcz_store.cc:276] 0709 12:13:09.479040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.483360 (+  4320us) service_pool.cc:224] Handling call
0709 12:13:09.483711 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.416365 (+932654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.416409 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.416979 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.581263 (+164284us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:10.581334 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.581468 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.102349 (+520881us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b4800041658. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:11.189990 (+ 87641us) raft_consensus.cc:1685] finished
0709 12:13:11.190013 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.190280 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":791172,"raft.queue_time_us":823945,"raft.run_cpu_time_us":5717,"raft.run_wall_time_us":790311,"spinlock_wait_cycles":6758656,"thread_start_us":290157,"threads_started":2,"wal-append.queue_time_us":199952}
I20260709 12:13:11.178946 13743 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:11.251199 13690 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:11.251691 13690 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:11.251838 13798 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.252395 13618 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.252868 13743 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:11.253870 13828 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.252661 13690 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.189523 13765 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.254334 13446 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.254468 13828 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.255148 13690 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.256152 13765 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.256955 13828 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.254058 13743 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:11.259115 13693 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.838s	user 0.012s	sys 0.000s
W20260709 12:13:11.260947 13779 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.855s	user 0.005s	sys 0.005s
I20260709 12:13:11.202243 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.263713 11324 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:11.264906 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.265834 11927 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:13:11.240316 13698 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.254336 13733 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.270277 13733 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.272472 13733 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.273576 11325 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:13:11.275535 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1693) took 1259 ms. Trace:
I20260709 12:13:11.275982 11177 rpcz_store.cc:276] 0709 12:13:10.015890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.016046 (+   156us) service_pool.cc:224] Handling call
0709 12:13:11.275310 (+1259264us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54000cd854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:11.275464 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140193664}
I20260709 12:13:11.240505 13166 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.278383 13166 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.265575 11324 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:11.141485 13634 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:11.152859 13628 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.159710 13818 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:11.284869 13782 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.287251 13782 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.150774 13812 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.234619 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.257488 13823 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.236186 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:11.310981 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:11.236330 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 88562.3us
W20260709 12:13:11.311780 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:11.312644 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.237771 13817 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.219182 13813 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.283252 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1440) took 2215 ms. Trace:
W20260709 12:13:11.290571 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 99459.7us
W20260709 12:13:11.288440 11930 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.669s	user 0.005s	sys 0.006s
I20260709 12:13:11.322268 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:11.283877 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.323552 11329 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:11.140404 11255 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:11.326745 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.318392 11249 rpcz_store.cc:276] 0709 12:13:09.068338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.185921 (+1117583us) service_pool.cc:224] Handling call
0709 12:13:10.186433 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.279086 (+ 92653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.279120 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.288343 (+  9223us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.550701 (+262358us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:10.571590 (+ 20889us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.572214 (+   624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.572655 (+   441us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.283028 (+710373us) raft_consensus.cc:1685] finished
0709 12:13:11.283049 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.283184 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14920,"prepare.queue_time_us":19067,"prepare.run_wall_time_us":4124,"raft.queue_time_us":46565,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":17270,"spinlock_wait_cycles":8403200,"thread_start_us":327201,"threads_started":2,"wal-append.queue_time_us":646260}
W20260709 12:13:11.312016 13802 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:13:11.315277 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.331267 11330 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:11.318768 11325 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:11.147444 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:11.321251 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:11.314966 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 208748us
I20260709 12:13:11.333563 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.334704 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:11.335376 11254 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:11.337513 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.343072 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.343396 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:11.343935 11330 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:11.345300 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.345988 11931 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:11.347587 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.348207 11324 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:13:11.348249 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.332602 11326 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:11.349050 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.349763 11324 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:11.276928 13554 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.321990 13821 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:13:11.351045 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.361398 11324 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:11.361716 13821 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:11.362200 13821 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:11.362411 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.362813 13753 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:11.363104 11324 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:11.363272 13753 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:11.365834 13824 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:11.366626 13693 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.364463 13753 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.367204 13693 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.367815 13792 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.368126 13753 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:11.368531 13607 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.323083 13806 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.322001 11930 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:11.369374 13806 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.370110 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 511) took 1021 ms. Trace:
I20260709 12:13:11.370625 11930 rpcz_store.cc:276] 0709 12:13:10.348321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.410837 (+ 62516us) service_pool.cc:224] Handling call
0709 12:13:11.370033 (+959196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:11.371655 13806 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.371752 13807 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5.
I20260709 12:13:11.326879 11928 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:11.240154 13697 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.288961 11926 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
W20260709 12:13:11.350781 13781 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.002s
I20260709 12:13:11.353962 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:11.281668 13779 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.378366 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.374989 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:11.388084 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:11.288223 13481 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.381201 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.381613 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.345439 13817 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:11.382328 13166 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.382858 13824 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:11.384166 13289 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.419229 13289 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.389233 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:11.389776 13563 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.396561 11249 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:13:11.369482 13693 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.351876 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:11.396915 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:11.404327 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1416) took 2615 ms (client timeout 2197 ms). Trace:
I20260709 12:13:11.430289 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:11.319453 13823 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.420428 11248 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:11.412737 13634 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.413208 13830 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.386328 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:11.425197 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:11.389251 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.426662 11254 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:11.419884 13779 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:11.440371 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54139.5us
W20260709 12:13:11.441120 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51943us
I20260709 12:13:11.428838 13818 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:11.396334 13743 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:11.396474 13510 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:11.442196 13510 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:11.442497 13693 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:11.396904 13782 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.448118 13166 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:11.449874 13806 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:11.450284 13806 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:11.451042 13806 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.452260 13806 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:11.432260 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.397123 13792 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:13:11.429972 11250 rpcz_store.cc:269] 0709 12:13:08.789184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.955492 (+166308us) service_pool.cc:224] Handling call
0709 12:13:11.400166 (+2444674us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400069854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:11.404233 (+  4067us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3054788992}
I20260709 12:13:11.455782 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.456481 11324 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:11.457363 11250 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:13:11.377305 13373 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.441341 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.459887 11254 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:11.460652 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:11.461953 13373 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.462661 13373 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:11.463063 11254 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:13:11.462908 13166 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:11.463701 11250 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:11.463047 13373 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.464476 13373 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.465098 13743 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:11.466634 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.466720 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.467499 11254 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:11.467720 11926 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:11.467808 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.468446 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.468577 11180 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:11.469173 11254 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:11.464891 13796 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:11.470709 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:11.471046 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.471755 11254 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:11.472404 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.473656 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:11.474138 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.474403 11935 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:11.479686 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:11.480122 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.480283 11180 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.481180 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:11.481904 11935 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:11.482607 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.474573 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.483376 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.483856 13697 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.475293 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.484697 13618 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:11.485338 13618 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.483973 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:11.476332 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.477339 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.476756 11181 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:11.487927 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.488314 11175 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:11.477811 13628 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.478026 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.478458 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:11.403412 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 422) took 1102 ms. Trace:
I20260709 12:13:11.424420 13820 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.485538 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.487779 13618 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.484051 11179 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:11.491336 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:11.491698 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 66607.4us
I20260709 12:13:11.507066 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:11.477681 13831 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:11.492616 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:11.482785 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.433120 13289 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:11.511551 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.514829 11249 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:11.515038 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:11.434319 13563 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.511319 13792 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:11.515818 13806 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:11.520192 13563 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.519731 13792 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.519825 13831 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.443142 13510 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.520920 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.521595 11254 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 2.
I20260709 12:13:11.521991 13824 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.388115 13781 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.489239 13628 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.523448 11178 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:11.517040 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:11.523313 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.438540 13733 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:11.508255 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:11.518143 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.529153 13655 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:11.350697 13814 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.527753 13618 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.529563 13655 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.530602 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:11.530751 13655 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.499740 11928 rpcz_store.cc:276] 0709 12:13:10.301527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.581709 (+280182us) service_pool.cc:224] Handling call
0709 12:13:10.642152 (+ 60443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:11.380687 (+738535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.399762 (+ 19075us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:11.532148 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.532781 11928 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:11.499133 13810 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:11.533695 13810 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.534070 13810 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:11.522529 13831 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.518357 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:11.513074 13820 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.538028 13779 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:11.538610 13779 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.539040 13779 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:11.437650 13635 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.540035 13820 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.541833 13635 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.542465 13554 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.542734 11328 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:11.543177 13809 consensus_queue.cc:1175] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 5 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:11.544147 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:11.545781 13697 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:11.439899 13830 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:11.548317 13830 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.551064 11177 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:11.551549 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:11.537861 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.528823 13733 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.530248 13814 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.512174 13602 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.556945 11180 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:11.537320 11250 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:11.588650 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.498494 13765 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:11.593823 13733 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.594664 13733 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:11.595381 13733 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.506951 13698 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:11.571228 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:11.602116 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.602792 11180 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:11.603407 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:11.604390 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.575153 13823 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.512532 13817 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:11.529418 11249 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:11.582229 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:11.583820 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.530084 13618 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.589329 11250 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:11.605476 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:11.519733 13481 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.598403 13733 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.614588 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:11.563725 13563 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:11.613010 11253 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:11.570211 13806 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:11.618952 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.619309 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:11.619580 11249 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:11.621578 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:11.625934 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.626111 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:11.630708 11251 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.632540 11253 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:13:11.607620 13627 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:11.608497 13698 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.650928 13845 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.651455 13693 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:11.652770 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.610423 13765 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:11.616863 13289 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.647441 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.588464 13831 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:13:11.653818 11254 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:11.618508 13618 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.624131 13806 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.664270 11250 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:11.667111 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.675172 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:11.625322 13823 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.635133 13554 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:11.599651 13602 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.634920 13635 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:11.608875 13810 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.648536 13481 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:11.654218 13698 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.687875 13823 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.656527 13765 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.643199 13563 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:11.698611 13823 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.671727 13693 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:11.702723 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.681137 13806 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:11.717587 11935 raft_consensus.cc:2368] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8.
I20260709 12:13:11.651510 13845 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:11.705919 13765 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.721264 13845 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.727645 13765 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.730130 13765 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.732472 13366 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:11.734578 13366 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:11.735595 13366 consensus_queue.cc:1243] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:13:11.750780 13628 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.722968 13602 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.723721 13733 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.723721 11251 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:11.752410 13563 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.769896 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.770699 11931 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
W20260709 12:13:11.764168 13686 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.828s	user 0.006s	sys 0.008s
I20260709 12:13:11.805159 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.756562 11930 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:11.703091 13481 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:11.838346 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.838752 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:11.841799 11181 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:11.846369 11181 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:13:11.772679 13733 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.853785 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.856470 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:11.857386 11175 raft_consensus.cc:2410] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:13:11.862203 13846 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:13:11.862771 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.863430 11175 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:11.864234 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.865969 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:11.867588 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:11.868342 11931 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:11.870149 13622 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.871313 13618 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.873385 13481 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.885171 13618 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.877341 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.215296 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:11.839102 13781 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:11.889025 13693 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:11.890372 11182 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:11.886135 11926 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 12:13:11.804239 13830 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:11.807025 13686 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:11.911288 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.912650 11179 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:11.850831 13782 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:11.920277 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:11.912009 11926 pending_rounds.cc:85] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 4
I20260709 12:13:11.918951 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:11.924048 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.929550 13481 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:11.929865 13810 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:11.930189 13481 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.930661 13481 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:11.910202 13676 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.921571 11931 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7. Re-sending same reply.
I20260709 12:13:11.963029 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:11.934034 11926 pending_rounds.cc:107] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 4.5
W20260709 12:13:11.966728 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 347458us
I20260709 12:13:12.030987 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:12.039445 11931 raft_consensus.cc:2410] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:13:12.034415 13852 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:13:12.032300 13830 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:12.034480 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.033933 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:12.044386 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:12.045032 13686 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:12.045243 11253 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:12.045320 13655 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.005s
I20260709 12:13:12.046634 13655 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.047617 13698 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.049017 13655 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:12.050583 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:12.051183 11253 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.044523 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:12.053839 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:11.998971 13753 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.630s	user 0.016s	sys 0.000s
I20260709 12:13:12.054625 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:11.939433 13676 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.055785 11251 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:12.057458 13676 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:12.060348 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:11.936331 13622 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:12.030920 13373 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.000s
I20260709 12:13:12.031736 13554 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:12.029529 13289 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.078599 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1448) took 2586 ms. Trace:
I20260709 12:13:12.064445 13698 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:12.077260 11251 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:12.081053 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:12.079674 11256 rpcz_store.cc:276] 0709 12:13:09.492228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.396958 (+904730us) service_pool.cc:224] Handling call
0709 12:13:10.429746 (+ 32788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.487906 (+ 58160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.072314 (+1584408us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b5400086bd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:12.078538 (+  6224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17481088}
I20260709 12:13:11.999408 11928 raft_consensus.cc:2368] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7.
I20260709 12:13:12.077984 11253 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:12.080744 13850 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.077610 13830 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:12.090435 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:12.058259 11926 raft_consensus.cc:2889] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:12.072613 13373 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.086412 13481 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.087647 13289 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.066923 13648 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:12.089902 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.094400 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:12.094954 11181 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
W20260709 12:13:12.095666 13794 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.471s	user 0.011s	sys 0.002s
I20260709 12:13:12.096014 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.096897 11180 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:13:12.099200 13554 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:12.100142 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.067610 13655 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:12.078732 13622 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.104915 13622 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:12.105024 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:12.092386 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.031016 13779 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.101451 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:12.077551 13671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.550s	user 0.023s	sys 0.011s
I20260709 12:13:12.129727 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:12.130334 11322 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:12.125347 11173 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
W20260709 12:13:12.125854 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.126169 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:12.126482 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:12.132126 11323 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:13:12.083338 13782 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.132257 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:12.132581 11330 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:12.132880 11251 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:12.122150 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:12.134797 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.127552 13698 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:13:12.135629 13697 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:12.135950 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:12.132450 13618 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:13:12.137131 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:12.137737 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:12.139940 11251 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:12.136674 11182 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:12.141757 13792 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:12.143025 13792 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:12.143404 13792 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:12.144363 13792 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.137708 11256 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:12.146811 13782 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.148481 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 503) took 2669 ms. Trace:
I20260709 12:13:12.129863 13733 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:12.128448 13554 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:12.150262 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:12.154182 11932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 421) took 1865 ms (client timeout 1707 ms). Trace:
I20260709 12:13:12.154860 13779 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:12.154588 11932 rpcz_store.cc:269] 0709 12:13:10.288388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.297607 (+  9219us) service_pool.cc:224] Handling call
0709 12:13:12.153957 (+1856350us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b540059a6d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:12.154108 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410014080}
I20260709 12:13:12.155583 13850 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:12.156179 13850 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:12.158245 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:12.158818 13740 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:12.159264 13740 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:12.158962 13850 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.160199 13740 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.160945 13640 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:12.161362 13640 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:12.135155 13671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.607s	user 0.024s	sys 0.011s
I20260709 12:13:12.162451 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:12.162387 13640 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.163838 13640 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.166222 13554 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:12.166781 13793 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.542s	user 0.003s	sys 0.003s
I20260709 12:13:12.165480 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.167524 11179 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:12.167945 11256 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:12.168439 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:12.169152 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.169399 11179 raft_consensus.cc:2410] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 8 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:12.169296 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.175367 13806 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.177820 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.179960 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.204113 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.182514 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:12.204852 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.183010 11322 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:12.184659 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.148865 11934 rpcz_store.cc:276] 0709 12:13:09.479393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.479894 (+   501us) service_pool.cc:224] Handling call
0709 12:13:09.480265 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.273970 (+793705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.274013 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.274587 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.438466 (+163879us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:10.617380 (+178914us) log.cc:844] Serialized 14 byte log entry
0709 12:13:12.095054 (+1477674us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:12.148020 (+ 52966us) spinlock_profiling.cc:243] Waited 872 us on lock 0x7b18003945a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287
0709 12:13:12.148156 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.148227 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.148269 (+    42us) raft_consensus.cc:1685] finished
0709 12:13:12.148282 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.148423 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":803412,"prepare.queue_time_us":334764,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":1219581,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":9794,"spinlock_wait_cycles":38828160,"thread_start_us":474793,"threads_started":3,"wal-append.queue_time_us":302818}
I20260709 12:13:12.186898 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1447) took 2695 ms. Trace:
I20260709 12:13:12.205961 11179 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:12.149420 13782 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:12.161281 13740 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:12.188305 13554 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:12.177937 13792 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:12.193526 11182 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:12.199571 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:12.211565 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.207777 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:12.210310 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.212443 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.211824 13618 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:12.213986 13618 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.206185 11257 rpcz_store.cc:276] 0709 12:13:09.491808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.374728 (+882920us) service_pool.cc:224] Handling call
0709 12:13:10.375137 (+   409us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:13:10.441816 (+ 66679us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:10.507247 (+ 65431us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.874874 (+1367627us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b1800425560. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:13:12.063243 (+188369us) log.cc:844] Serialized 14 byte log entry
0709 12:13:12.063604 (+   361us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:12.064134 (+   530us) log.cc:844] Serialized 52 byte log entry
0709 12:13:12.064263 (+   129us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:12.064294 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.065367 (+  1073us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.186673 (+121306us) raft_consensus.cc:1685] finished
0709 12:13:12.186694 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.186836 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7145,"prepare.queue_time_us":65564,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":6430,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":161,"spinlock_wait_cycles":37751936,"thread_start_us":186943,"threads_started":1,"wal-append.queue_time_us":1661467}
W20260709 12:13:12.214949 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.214052 13666 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:12.215844 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:12.220147 11253 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:12.220402 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:12.221180 11179 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:12.217491 11257 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:12.210559 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:12.215910 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:12.217578 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:12.228282 11296 leader_election.cc:400] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8.
I20260709 12:13:12.228729 11296 leader_election.cc:403] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.229797 11934 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:13:12.282882 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.273214 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.213883 13830 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:12.249887 13698 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:12.289994 13698 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.290414 13698 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:12.286226 13289 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:12.250056 13635 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:12.227969 13666 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.294090 13635 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:12.294178 13666 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.295112 13635 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.227356 13782 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:12.282619 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:12.127751 13823 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.297951 13782 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:12.286623 11933 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:12.298755 13806 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:12.299033 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 529) took 1964 ms. Trace:
I20260709 12:13:12.299117 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:12.299513 13823 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:12.299468 11933 rpcz_store.cc:276] 0709 12:13:10.334362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.283340 (+1948978us) service_pool.cc:224] Handling call
0709 12:13:12.283675 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:12.298782 (+ 15107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.298977 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:12.214542 13618 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:12.300421 11181 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:12.227640 11928 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:12.284512 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 62610.6us
I20260709 12:13:12.298429 13782 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:12.308418 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:12.212896 13714 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:12.310292 11182 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:12.212188 13779 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:12.219322 13554 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:12.345472 11179 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:12.329991 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:12.347213 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:12.250813 13817 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:12.310060 13807 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:12.357864 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.358901 11179 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:12.305526 13289 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8.
I20260709 12:13:12.330844 13779 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.363392 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.364094 11179 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:12.365016 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:12.366493 11179 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:13:12.368352 11182 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:12.351363 13655 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:12.369928 13850 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:12.373142 13697 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:12.373719 13850 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:12.374176 13850 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:12.348865 13714 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:12.373034 11182 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:12.368855 11179 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:12.353847 13830 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:12.461136 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 234982us
I20260709 12:13:12.465974 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.362751 13817 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:12.369313 13779 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:12.371752 13655 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:12.483935 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 128769us
I20260709 12:13:12.488848 11257 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:12.496544 13289 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:12.500442 13289 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:12.503736 13289 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:12.374557 13697 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.618275 13623 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:12.618865 13623 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:12.619736 13623 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.615352 13635 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.622812 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1674) took 2612 ms. Trace:
I20260709 12:13:12.622637 13635 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.623165 11251 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:12.623217 11248 rpcz_store.cc:276] 0709 12:13:10.010682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.404263 (+393581us) service_pool.cc:224] Handling call
0709 12:13:10.404549 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:11.435548 (+1030999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.622514 (+1186966us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002a6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:12.622722 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2335773952}
I20260709 12:13:12.643028 13781 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:12.644026 13743 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:12.638566 13816 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:13:12.644609 13743 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:12.645848 13743 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:12.663661 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.483964 13554 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.534610 13714 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:12.691515 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 120665us
W20260709 12:13:12.693779 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 227883us
W20260709 12:13:12.702611 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.703653 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:12.704276 11251 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:12.607367 13289 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:13:12.710489 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:12.624576 13830 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.743966 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:12.744784 11249 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:12.666987 13618 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.744740 11251 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:12.680114 13640 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.747900 11327 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:12.750623 13627 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:12.639276 13817 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.749131 11250 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:12.708662 13743 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:12.751142 13627 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:12.771795 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.774236 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1699) took 1298 ms. Trace:
I20260709 12:13:12.776398 11177 rpcz_store.cc:276] 0709 12:13:11.475374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.477144 (+  1770us) service_pool.cc:224] Handling call
0709 12:13:11.477452 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:11.551490 (+ 74038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.773873 (+1222383us) spinlock_profiling.cc:243] Waited 73.2 ms on lock 0x7b5400363654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:12.774135 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168445312}
I20260709 12:13:12.683252 13655 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:12.747423 13869 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:12.779989 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.779919 13812 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:12.784180 13740 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:12.797852 11176 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:12.790740 13864 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 12:13:12.792784 13691 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:12.790442 13622 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.769138 13640 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:12.802143 13877 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:13:12.817456 11930 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.679s	user 0.012s	sys 0.004s
I20260709 12:13:12.826467 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1451) took 2209 ms. Trace:
I20260709 12:13:12.801571 13740 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.828197 11252 rpcz_store.cc:276] 0709 12:13:10.617044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.617579 (+   535us) service_pool.cc:224] Handling call
0709 12:13:10.617908 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.933577 (+315669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.826197 (+1892620us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400067a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:12.826380 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232920320}
I20260709 12:13:12.838568 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:12.787743 13869 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:12.844398 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.839576 11252 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
W20260709 12:13:12.834213 13870 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:13:12.845389 13691 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:12.803539 13825 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 12:13:12.848598 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60147.4us
I20260709 12:13:12.837108 13740 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:12.856964 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:12.858646 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.849251 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:12.858094 11176 raft_consensus.cc:2376] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6. Re-sending same reply.
I20260709 12:13:12.834257 11930 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:12.827229 13622 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:12.946094 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 787836us
W20260709 12:13:13.031744 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 100073us
I20260709 12:13:12.874188 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 437) took 1324 ms. Trace:
I20260709 12:13:12.920019 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:12.937378 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1904) took 1073 ms. Trace:
I20260709 12:13:13.094725 11931 rpcz_store.cc:276] 0709 12:13:11.549727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.058822 (+509095us) service_pool.cc:224] Handling call
0709 12:13:12.080078 (+ 21256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:12.125144 (+ 45066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:12.125175 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:12.125665 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:13:12.347838 (+222173us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:12.347881 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.347950 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.873911 (+525961us) raft_consensus.cc:1685] finished
0709 12:13:12.873930 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.874106 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31091,"raft.queue_time_us":32368,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":4296,"thread_start_us":176521,"threads_started":1,"wal-append.queue_time_us":319821}
I20260709 12:13:13.104599 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:13.114602 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:12.875093 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:13.093875 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:12.923774 13666 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.629s	user 0.007s	sys 0.004s
I20260709 12:13:13.091168 13173 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:12.923805 13782 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.000s
W20260709 12:13:12.903750 13698 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.000s
I20260709 12:13:12.955124 13673 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.998452 13816 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.873415 13881 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:12.873656 13883 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:13.112257 11174 rpcz_store.cc:276] 0709 12:13:11.855713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.855900 (+   187us) service_pool.cc:224] Handling call
0709 12:13:11.856372 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:11.856943 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:11.856993 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:11.857892 (+   899us) log.cc:844] Serialized 52 byte log entry
0709 12:13:12.058049 (+200157us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:12.064567 (+  6518us) log.cc:844] Serialized 14 byte log entry
0709 12:13:12.928746 (+864179us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b180046fea0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:13:12.929022 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.929143 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.929214 (+    71us) raft_consensus.cc:1685] finished
0709 12:13:12.929338 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.929537 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3343,"prepare.queue_time_us":8823,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":881247,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132,"spinlock_wait_cycles":28722816,"thread_start_us":138029,"threads_started":1,"wal-append.queue_time_us":239421}
I20260709 12:13:13.122835 11254 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:12.978891 13823 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:12.902561 13869 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.109303 13820 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 12:13:13.134522 13806 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:12.930943 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1905) took 1074 ms. Trace:
I20260709 12:13:13.134208 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 429) took 1979 ms. Trace:
I20260709 12:13:13.059465 13885 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.999210 13812 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:13.155423 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.875769 13618 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:13.123823 13743 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:13.119494 13876 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.065390 13676 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.135s
I20260709 12:13:13.220114 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:13.318372 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 213900us
I20260709 12:13:13.319095 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:13.319674 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:13.320286 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:13.320539 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:13.126158 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:13.224975 11175 rpcz_store.cc:276] 0709 12:13:11.856034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.864960 (+  8926us) service_pool.cc:224] Handling call
0709 12:13:11.865293 (+   333us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:11.885089 (+ 19796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:11.885123 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:11.885750 (+   627us) log.cc:844] Serialized 52 byte log entry
0709 12:13:12.062181 (+176431us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:12.081595 (+ 19414us) log.cc:844] Serialized 14 byte log entry
0709 12:13:12.930501 (+848906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.930614 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.930670 (+    56us) raft_consensus.cc:1685] finished
0709 12:13:12.930689 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.930867 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":873739,"prepare.queue_time_us":19513,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":892477,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":164,"thread_start_us":176041,"threads_started":1,"wal-append.queue_time_us":176346,"wal-append.run_cpu_time_us":2790,"wal-append.run_wall_time_us":1067602}
I20260709 12:13:12.998731 13289 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:12.922266 13635 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.197564 13623 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:12.921799 13691 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.216522 13883 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.220772 13823 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.349402 13821 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.350533 13883 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:13.139506 13666 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.199662 13782 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:13.253917 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 142864us
I20260709 12:13:13.213492 13881 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:13.212621 13673 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.241225 13806 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:13.347657 11323 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:13.349481 13900 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.350222 13823 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:13.350859 13821 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:13.201932 13698 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.217391 13816 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.352998 11254 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:13.362969 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.363306 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 240110us
I20260709 12:13:13.366376 11249 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:13.344607 13869 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:13.389979 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.390713 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:13.307891 13812 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:13.392760 11254 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:13.365674 13591 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:13.240139 13885 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.365824 13690 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:13.362596 13817 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.355540 13743 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:13.365013 13806 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.231701 11927 rpcz_store.cc:276] 0709 12:13:11.155122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.160285 (+  5163us) service_pool.cc:224] Handling call
0709 12:13:11.173075 (+ 12790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:11.266278 (+ 93203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.113840 (+1847562us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b5400690ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:13.130886 (+ 17046us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91258624}
I20260709 12:13:13.353142 13623 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.397063 13591 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:13.406147 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 486220us
I20260709 12:13:13.383991 13738 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:13.316601 13876 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.350306 13901 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:13:13.389176 13779 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.902s	user 0.013s	sys 0.000s
I20260709 12:13:13.346439 13289 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.387262 13900 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.558249 11929 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:13:13.515069 13740 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.004s
I20260709 12:13:13.419894 13806 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:13.599498 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 208835us
I20260709 12:13:13.599153 13753 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.549779 11253 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:13.600425 13753 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.600809 13738 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:13.440388 13893 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:13.599537 13591 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:13.542634 13671 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.006s	sys 0.010s
I20260709 12:13:13.602000 13738 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.602120 13893 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:13.533924 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 181850us
I20260709 12:13:13.603219 13738 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:13.603766 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.433012 13821 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.604511 13821 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.605041 13893 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:13.575845 13623 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:13.512192 11248 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:13.536092 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:13.606958 11327 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:13.602649 13671 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.006s	sys 0.011s
W20260709 12:13:13.432221 13627 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.000s
I20260709 12:13:13.607996 13627 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.411885 13622 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.607214 13901 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:13.610136 13627 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:13.484949 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.426833 13782 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.434303 13881 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.612432 13881 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:13.613780 13821 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:13.571631 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1452) took 2954 ms (client timeout 1945 ms). Trace:
W20260709 12:13:13.617919 11255 rpcz_store.cc:269] 0709 12:13:10.617343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.617577 (+   234us) service_pool.cc:224] Handling call
0709 12:13:13.571405 (+2953828us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400277c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:13.571539 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713770368}
I20260709 12:13:13.610929 13901 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.619262 13901 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:13.619854 13289 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.549952 13902 raft_consensus.cc:1064] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:13:13.577152 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:13.576156 13850 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.200s	user 0.020s	sys 0.002s
I20260709 12:13:13.622157 13850 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.622939 13902 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.624745 13850 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.407136 13690 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.625985 13690 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:13.394373 13869 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.415067 13817 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.598881 13779 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.690910 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:13.597920 11182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.000s	sys 0.008s
W20260709 12:13:13.691798 11182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.001s	sys 0.013s
W20260709 12:13:13.646324 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.662629 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.576747 13903 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.228s	user 0.000s	sys 0.002s
I20260709 12:13:13.612594 13876 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.435672 13816 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.705906 13812 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:13.566000 13899 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:13:13.706730 13743 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.732956 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:13.686383 13634 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.690465 13824 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.684950 13827 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:13.690167 13666 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.641116 13821 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.599192 13740 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.757570 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:13.783980 11323 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:13.764170 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.768666 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.771436 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 450922us
I20260709 12:13:13.779275 11174 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:13:13.753328 11935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.013s	sys 0.008s
I20260709 12:13:13.732946 13876 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:13.787322 11935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.013s	sys 0.008s
I20260709 12:13:13.607225 13900 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:13.734833 13289 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:13.637501 13591 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.629542 13901 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.781493 13881 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.800799 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:13.817590 11175 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
W20260709 12:13:13.813010 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.819013 11328 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:13.787918 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:13.789696 13850 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:13:13.800426 13876 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.827766 13876 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.810094 11327 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:13.759120 13634 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:13.780448 13824 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.779320 13827 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:13.866390 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.789608 13673 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:13.837549 13830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.014s	sys 0.018s
W20260709 12:13:13.880242 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 279784us
I20260709 12:13:13.829860 13881 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.838686 11323 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:13.792896 13635 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 12:13:13.888185 13371 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.836352 13623 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:13.940070 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49917.5us
I20260709 12:13:13.835507 13698 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.879654 13912 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:13.944382 13909 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:13:13.945219 13914 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:13:13.893726 13830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.014s	sys 0.019s
I20260709 12:13:13.787462 13821 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:13.943461 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.954226 11181 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 334) took 1922 ms (client timeout 1716 ms). Trace:
W20260709 12:13:13.981721 11181 rpcz_store.cc:269] 0709 12:13:12.032058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.095712 (+ 63654us) service_pool.cc:224] Handling call
0709 12:13:13.954003 (+1858291us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54002e4fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:13.954141 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468282112}
I20260709 12:13:13.973608 13885 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:13.952653 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.971359 11179 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 4. Current term is 6. Ops: []
W20260709 12:13:13.947345 13905 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
W20260709 12:13:13.976738 11180 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1897) took 2133 ms (client timeout 1597 ms). Trace:
I20260709 12:13:13.944450 13481 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.946658 13827 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.945219 13869 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:13.946709 13824 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:13.988245 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.987702 11180 rpcz_store.cc:269] 0709 12:13:11.838349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.838502 (+   153us) service_pool.cc:224] Handling call
0709 12:13:13.970871 (+2132369us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54000c0b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:13.971231 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591907072}
I20260709 12:13:13.991969 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:13.996136 11323 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:13.987934 13691 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.008s	sys 0.020s
I20260709 12:13:13.986369 13817 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:13.992089 13887 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.986006 13917 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:13.973768 13623 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:13.993405 13779 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:14.001287 13691 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.009s	sys 0.020s
W20260709 12:13:14.003613 13896 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:13:14.010893 13887 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:14.023290 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.036392 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 336) took 2003 ms. Trace:
I20260709 12:13:14.016292 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 301) took 1310 ms. Trace:
I20260709 12:13:14.037850 11249 rpcz_store.cc:276] 0709 12:13:12.705415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.770198 (+ 64783us) service_pool.cc:224] Handling call
0709 12:13:12.784683 (+ 14485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.391849 (+607166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:14.016067 (+624218us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54001ce8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:14.016233 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337274112}
I20260709 12:13:14.023573 11330 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:14.024045 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.028062 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.042275 11324 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:14.043169 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.040597 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.057559 11934 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:14.056406 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:14.068959 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.070465 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:14.036954 11182 rpcz_store.cc:276] 0709 12:13:12.032761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.432161 (+399400us) service_pool.cc:224] Handling call
0709 12:13:12.432534 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:12.992421 (+559887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:12.992454 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:12.993087 (+   633us) log.cc:844] Serialized 52 byte log entry
0709 12:13:13.692456 (+699369us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:13.941867 (+249411us) log.cc:844] Serialized 14 byte log entry
0709 12:13:13.974375 (+ 32508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.974461 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:14.036080 (+ 61619us) raft_consensus.cc:1685] finished
0709 12:13:14.036111 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:14.036306 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":573362,"prepare.queue_time_us":123449,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":614961,"raft.run_cpu_time_us":578,"raft.run_wall_time_us":40378,"thread_start_us":895584,"threads_started":2,"wal-append.queue_time_us":819346}
I20260709 12:13:14.071308 11933 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:14.059716 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:14.073163 11932 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:14.073928 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.058118 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:14.075083 11180 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:14.075867 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:14.075148 11326 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:14.076627 11331 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:14.059188 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:14.078383 11174 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:13.978940 13912 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:14.079929 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.083266 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1705) took 2601 ms. Trace:
I20260709 12:13:14.083709 11178 rpcz_store.cc:276] 0709 12:13:11.481999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.482131 (+   132us) service_pool.cc:224] Handling call
0709 12:13:11.482460 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:11.527000 (+ 44540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:14.082976 (+2555976us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b540016f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:14.083199 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93506432}
I20260709 12:13:14.058588 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:14.018761 13897 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.667s	user 0.002s	sys 0.000s
I20260709 12:13:14.011931 11323 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:14.087671 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:14.031344 13817 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.022848 13635 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.023061 13623 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:14.092942 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.081837 13591 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:14.014350 13865 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.643s	user 0.003s	sys 0.000s
I20260709 12:13:14.040226 13740 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:14.092968 11175 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:14.114097 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:14.015724 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 514) took 3522 ms. Trace:
I20260709 12:13:14.022959 13827 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:14.096938 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:14.092777 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:14.024360 13917 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:14.047014 13908 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
I20260709 12:13:14.047395 13923 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Committing config change with OpId 6.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:14.082223 13806 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:14.119146 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.050480 13827 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:14.298171 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.052578 13806 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.051421 11323 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:13:15.053284 13827 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:14.119561 13648 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.053975 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.054503 13933 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.054672 11323 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:15.055167 13933 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.057271 13933 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:14.420886 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.173609 11180 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:14.278522 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.058596 13635 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.059239 13887 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.059623 11930 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:15.059850 11931 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:14.141978 11328 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:14.160190 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.164410 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:14.583665 13881 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:14.185755 11174 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:14.438297 13619 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.061645 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1704) took 2242 ms. Trace:
I20260709 12:13:14.189931 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:14.275169 11249 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:14.177795 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.112318 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.062831 11181 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:14.198909 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:15.063781 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 911281us
I20260709 12:13:15.113319 11328 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:15.113425 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1920) took 1058 ms. Trace:
I20260709 12:13:15.113958 11181 rpcz_store.cc:276] 0709 12:13:14.054674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.093326 (+ 38652us) service_pool.cc:224] Handling call
0709 12:13:15.113357 (+1020031us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:14.353920 13850 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.199852 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.110914 11252 rpcz_store.cc:276] 0709 12:13:12.819536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.847541 (+ 28005us) service_pool.cc:224] Handling call
0709 12:13:12.847941 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:12.857787 (+  9846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:12.857831 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:12.871242 (+ 13411us) log.cc:844] Serialized 52 byte log entry
0709 12:13:13.126256 (+255014us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:13.126300 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.126374 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:14.440600 (+1314226us) raft_consensus.cc:1685] finished
0709 12:13:14.440628 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.061567 (+620939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9271,"raft.queue_time_us":9986,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":11878,"thread_start_us":196305,"threads_started":1,"wal-append.queue_time_us":661118}
I20260709 12:13:15.050419 13821 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:14.141572 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:14.602790 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1468) took 1498 ms (client timeout 1898 ms). Trace:
I20260709 12:13:15.070132 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:15.118726 11253 rpcz_store.cc:269] 0709 12:13:13.104622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.104779 (+   157us) service_pool.cc:224] Handling call
0709 12:13:14.602532 (+1497753us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400109354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:14.602702 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525299584}
W20260709 12:13:15.076725 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 2.209s (SENT) [suppressed 24 similar messages]
I20260709 12:13:14.528551 11331 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:15.120513 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1307) took 1099 ms. Trace:
W20260709 12:13:15.120579 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43918.8us
I20260709 12:13:15.121060 11331 rpcz_store.cc:276] 0709 12:13:14.021143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.027217 (+  6074us) service_pool.cc:224] Handling call
0709 12:13:15.120441 (+1093224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:15.121704 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.122304 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:14.264626 11933 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:15.123216 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 446) took 1063 ms. Trace:
I20260709 12:13:15.123625 11933 rpcz_store.cc:276] 0709 12:13:14.060247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.070139 (+  9892us) service_pool.cc:224] Handling call
0709 12:13:15.123160 (+1053021us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:15.052677 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.053975 13827 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.094300 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1916) took 1041 ms. Trace:
W20260709 12:13:15.096457 13743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.010s	sys 0.001s
W20260709 12:13:15.134585 13743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.010s	sys 0.002s
W20260709 12:13:14.288976 13883 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.938s	user 0.009s	sys 0.000s
W20260709 12:13:14.292702 13823 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.904s	user 0.009s	sys 0.000s
W20260709 12:13:14.253311 13909 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 12:13:15.060011 13902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.001s	sys 0.008s
I20260709 12:13:15.137866 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:15.138509 13902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.001s	sys 0.009s
I20260709 12:13:14.113377 13671 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:15.138902 11253 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:14.120169 13922 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.600675 13481 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.106452 11182 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:15.146154 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1923) took 1090 ms. Trace:
W20260709 12:13:15.146591 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.146729 11182 rpcz_store.cc:276] 0709 12:13:14.055589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.089688 (+ 34099us) service_pool.cc:224] Handling call
0709 12:13:15.146054 (+1056366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:14.192831 13640 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:14.117080 13634 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:14.115695 13901 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.108790 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1919) took 1054 ms. Trace:
I20260709 12:13:14.241448 13781 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:14.140432 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.159857 13739 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:14.190222 13779 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:15.114195 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.061903 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:15.156349 11222 connection.cc:256] RPC call timeout handler was delayed by -1.8494411220001439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 12:13:14.141093 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:15.156783 11222 connection.cc:256] RPC call timeout handler was delayed by -1.8089411220000784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 12:13:15.156661 11328 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:15.157418 11330 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:15.158034 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1316) took 1017 ms. Trace:
I20260709 12:13:15.158226 11330 rpcz_store.cc:276] 0709 12:13:14.140546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.140743 (+   197us) service_pool.cc:224] Handling call
0709 12:13:15.157989 (+1017246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.158864 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.115334 11250 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:14.177796 13698 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.092339 13936 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.162758 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.062286 11929 rpcz_store.cc:276] 0709 12:13:10.493012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.607703 (+114691us) service_pool.cc:224] Handling call
0709 12:13:11.273228 (+665525us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:11.273482 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.598932 (+2325450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:14.015442 (+416510us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b54006c5054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:14.015646 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5180161152}
I20260709 12:13:15.118907 11329 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 4: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:15.125066 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.054129 13648 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:14.280692 13891 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.916s	user 0.000s	sys 0.006s
I20260709 12:13:14.190330 13816 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.960s
I20260709 12:13:15.139083 13671 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:14.481773 13690 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.855s	user 0.011s	sys 0.000s
I20260709 12:13:14.138253 13817 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.107120 11926 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:15.148180 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.148356 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:15.159590 13881 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:15.159722 13623 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:15.177793 11331 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:15.162422 13912 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.164054 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1320) took 1022 ms. Trace:
I20260709 12:13:15.180446 11330 rpcz_store.cc:276] 0709 12:13:14.142549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.162504 (+1019955us) service_pool.cc:224] Handling call
0709 12:13:15.163988 (+  1484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.179589 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 443) took 1127 ms. Trace:
I20260709 12:13:15.183804 13937 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:15.183548 11926 rpcz_store.cc:276] 0709 12:13:14.052057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.066766 (+ 14709us) service_pool.cc:224] Handling call
0709 12:13:14.278275 (+211509us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:15.179519 (+901244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51866368}
I20260709 12:13:15.177703 13690 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.185195 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.145998 13373 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.186269 11926 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:15.167483 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.187101 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:15.186802 13623 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.168130 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:15.187638 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.156122 13781 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.188553 11926 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:15.188804 13623 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.189208 13623 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.152688 13812 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:15.137925 13883 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.170047 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1317) took 1029 ms. Trace:
I20260709 12:13:15.163832 13698 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.134378 13827 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.160080 13739 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.151556 13481 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.125129 11927 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:15.148160 13823 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.139838 13922 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.155331 13740 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 12:13:14.281258 13913 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 12:13:15.186499 13690 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.168490 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.218623 11931 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:15.218918 11932 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:15.219241 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 439) took 1168 ms. Trace:
I20260709 12:13:15.181617 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.120250 13619 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.165097 13779 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:15.126446 13850 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.192394 13881 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.209009 13739 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.191607 13883 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:15.210820 13889 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.885s	user 0.007s	sys 0.005s
I20260709 12:13:15.228820 13823 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.166299 13936 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.229671 13635 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.231555 13619 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.231863 13936 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:15.211830 13698 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.212946 13648 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:15.232977 13895 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.846s	user 0.002s	sys 0.014s
I20260709 12:13:15.234795 13481 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.190066 13781 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.193066 13937 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.219619 11931 rpcz_store.cc:276] 0709 12:13:14.050701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.072199 (+ 21498us) service_pool.cc:224] Handling call
0709 12:13:15.219177 (+1146978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.189612 13623 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.241434 13817 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.241966 13922 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.242100 13881 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.243616 13779 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.244267 13827 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.244626 13753 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:15.245798 13884 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 2.274s	user 0.010s	sys 0.012s
I20260709 12:13:15.251194 13850 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.246810 13937 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.248188 13917 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:15.248769 13888 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.902s	user 0.000s	sys 0.011s
W20260709 12:13:15.247700 13714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.208s	user 0.014s	sys 0.017s
W20260709 12:13:15.253841 13714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.224s	user 0.015s	sys 0.017s
W20260709 12:13:15.289402 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.121428 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:15.112706 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1921) took 1057 ms. Trace:
I20260709 12:13:15.134375 11180 rpcz_store.cc:276] 0709 12:13:14.053203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.057802 (+  4599us) service_pool.cc:224] Handling call
0709 12:13:15.094224 (+1036422us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:15.299017 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 1.17742e+06us
I20260709 12:13:15.298476 11178 rpcz_store.cc:276] 0709 12:13:14.054977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.117766 (+ 62789us) service_pool.cc:224] Handling call
0709 12:13:15.112650 (+994884us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.115198 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.301831 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 350) took 1104 ms. Trace:
I20260709 12:13:15.063256 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1918) took 1009 ms. Trace:
I20260709 12:13:15.302484 11181 rpcz_store.cc:276] 0709 12:13:14.196896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.114947 (+918051us) service_pool.cc:224] Handling call
0709 12:13:15.301502 (+186555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.302799 11175 rpcz_store.cc:276] 0709 12:13:14.053963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.058787 (+  4824us) service_pool.cc:224] Handling call
0709 12:13:15.063187 (+1004400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.303929 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:15.304618 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.128522 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.305522 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 344) took 3255 ms. Trace:
I20260709 12:13:15.305797 11175 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:13:15.306089 11179 rpcz_store.cc:276] 0709 12:13:12.049528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.466305 (+416777us) service_pool.cc:224] Handling call
0709 12:13:13.971084 (+1504779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.986427 (+ 15343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.304966 (+1318539us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54000c0b50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:15.305269 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3459811712}
I20260709 12:13:15.306767 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.307098 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.307403 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:14.297766 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1715) took 2173 ms. Trace:
I20260709 12:13:15.152998 11174 rpcz_store.cc:276] 0709 12:13:14.054322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.058893 (+  4571us) service_pool.cc:224] Handling call
0709 12:13:15.108723 (+1049830us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.308878 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:15.310110 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.310196 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.308765 11177 rpcz_store.cc:276] 0709 12:13:12.124292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.777847 (+653555us) service_pool.cc:224] Handling call
0709 12:13:12.785365 (+  7518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:13.030610 (+245245us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:13:13.030657 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:13.031244 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:13:13.116769 (+ 85525us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:13.116820 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.116938 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:14.018177 (+901239us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:14.248130 (+229953us) raft_consensus.cc:1685] finished
0709 12:13:14.248164 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:14.297667 (+ 49503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244756,"raft.queue_time_us":245406,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"spinlock_wait_cycles":180997504,"thread_start_us":85179,"threads_started":1,"wal-append.queue_time_us":277217}
I20260709 12:13:15.311474 11174 raft_consensus.cc:2410] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 8 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:15.311834 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:15.301908 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.009s	sys 0.009s
W20260709 12:13:15.312708 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.010s	sys 0.010s
I20260709 12:13:15.300709 11078 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:15.315217 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.304695 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.315721 11078 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:15.311949 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.317581 11182 raft_consensus.cc:2368] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
I20260709 12:13:15.318137 11177 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:15.318835 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.319259 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.319763 11173 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.320483 11177 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 3: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.321112 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.308545 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.322121 11174 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:15.322734 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1925) took 1162 ms. Trace:
I20260709 12:13:15.323199 11180 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:15.308086 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.325711 11176 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.326444 11179 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.326546 11180 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:15.321280 11182 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:13:15.327328 11182 pending_rounds.cc:85] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 3
I20260709 12:13:15.327749 11182 pending_rounds.cc:107] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:13:15.328197 11182 raft_consensus.cc:2889] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:15.329115 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1922) took 1273 ms. Trace:
I20260709 12:13:15.323166 11174 rpcz_store.cc:276] 0709 12:13:14.160506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.312280 (+1151774us) service_pool.cc:224] Handling call
0709 12:13:15.322648 (+ 10368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.329528 11182 rpcz_store.cc:276] 0709 12:13:14.055290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.320164 (+1264874us) service_pool.cc:224] Handling call
0709 12:13:15.320714 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.328831 (+  8117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.329035 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.324802 11181 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:13:15.325318 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.336252 11175 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:15.336102 11078 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 3, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 3 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:15.346118 13949 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:15.347263 13949 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.154662 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.374683 11326 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
W20260709 12:13:15.374951 13554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.453s	user 0.011s	sys 0.014s
W20260709 12:13:15.375439 13554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.453s	user 0.011s	sys 0.014s
I20260709 12:13:15.376991 13373 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.161394 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.156814 11254 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:15.185367 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1927) took 2479 ms. Trace:
I20260709 12:13:15.178347 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.190728 11324 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:15.189917 11327 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:15.190383 11329 rpcz_store.cc:276] 0709 12:13:14.140949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.141283 (+   334us) service_pool.cc:224] Handling call
0709 12:13:15.169992 (+1028709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.416416 11325 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:15.443003 13690 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.444015 13690 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.445170 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:15.446380 11251 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:15.446085 13733 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:15.447660 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.449896 11175 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:15.450241 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.451406 11253 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:15.452505 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.408346 11322 rpcz_store.cc:276] 0709 12:13:12.706329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.706538 (+   209us) service_pool.cc:224] Handling call
0709 12:13:12.706842 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:12.740707 (+ 33865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:12.740738 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:12.741205 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:13:15.072062 (+2330857us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b180035ba80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:15.151039 (+ 78977us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:15.151094 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.151281 (+   187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:15.184917 (+ 33636us) raft_consensus.cc:1685] finished
0709 12:13:15.184938 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.185284 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19250,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"spinlock_wait_cycles":31993216,"thread_start_us":78748,"threads_started":1,"wal-append.queue_time_us":2437238}
I20260709 12:13:15.455948 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.456785 11322 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:15.181617 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.458534 11331 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:13:15.459110 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1319) took 1317 ms. Trace:
I20260709 12:13:15.459419 11331 rpcz_store.cc:276] 0709 12:13:14.141659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.178374 (+1036715us) service_pool.cc:224] Handling call
0709 12:13:15.459006 (+280632us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.460919 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:15.461683 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.463192 13690 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.464758 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:15.181730 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.409617 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.470386 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:15.471778 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.471879 13806 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:15.472512 13806 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:15.472653 11254 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 2.
I20260709 12:13:15.474118 13949 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:15.476042 11324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1311) took 1434 ms (client timeout 1878 ms). Trace:
I20260709 12:13:15.467878 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 296) took 2764 ms. Trace:
W20260709 12:13:15.477936 11327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1313) took 1392 ms (client timeout 1679 ms). Trace:
W20260709 12:13:15.478416 11327 rpcz_store.cc:269] 0709 12:13:14.085618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.085840 (+   222us) service_pool.cc:224] Handling call
0709 12:13:15.477561 (+1391721us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54000c71d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:15.477763 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292777856}
I20260709 12:13:15.479913 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.481017 11327 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:13:15.222754 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 86616.4us
I20260709 12:13:15.225090 11927 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:15.481875 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.482841 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 438) took 1432 ms. Trace:
I20260709 12:13:15.483397 11927 rpcz_store.cc:276] 0709 12:13:14.050288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.079353 (+ 29065us) service_pool.cc:224] Handling call
0709 12:13:15.482475 (+1403122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:15.157438 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5402922289999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
I20260709 12:13:15.484016 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:15.485288 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3575605650000853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 12:13:15.485620 13602 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:15.486697 13602 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:15.374724 11249 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
W20260709 12:13:15.476554 11324 rpcz_store.cc:269] 0709 12:13:14.041868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.042883 (+  1015us) service_pool.cc:224] Handling call
0709 12:13:15.475546 (+1432663us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001c43d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:15.475818 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2637418752}
I20260709 12:13:15.491204 11249 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:15.388494 11178 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:15.378044 13373 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.383147 13481 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:15.494674 13481 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:15.494885 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1315) took 1356 ms. Trace:
I20260709 12:13:15.495217 13481 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:15.496003 11173 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:15.496424 13373 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.396695 13554 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.498158 13554 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.498878 11254 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:15.499902 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1304) took 1479 ms. Trace:
I20260709 12:13:15.397919 13591 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.500371 11325 rpcz_store.cc:276] 0709 12:13:14.020334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.023361 (+  3027us) service_pool.cc:224] Handling call
0709 12:13:14.023617 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.443620 (+1420003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.499511 (+ 55891us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54000e6854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:15.499804 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3117235072}
I20260709 12:13:15.398365 13714 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:15.501304 13591 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.399940 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:15.505939 13714 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:15.498730 13812 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:15.506541 13714 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:15.507025 13812 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:15.507645 11181 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:15.508082 11181 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:15.495425 11326 rpcz_store.cc:276] 0709 12:13:14.138920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.139099 (+   179us) service_pool.cc:224] Handling call
0709 12:13:14.139625 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.375446 (+1235821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.494509 (+119063us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400322dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:15.494767 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817101184}
I20260709 12:13:15.402631 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 544) took 2581 ms. Trace:
I20260709 12:13:15.509927 13908 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:15.510078 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.406842 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1702) took 2811 ms. Trace:
I20260709 12:13:15.411655 11250 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:15.405565 11079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 167) took 15055 ms (client timeout 15000 ms). Trace:
I20260709 12:13:15.409541 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 413) took 1221 ms. Trace:
I20260709 12:13:15.450390 11175 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:15.181769 11328 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
W20260709 12:13:15.541920 11282 heartbeater.cc:646] Failed to heartbeat to 127.9.198.254:36219 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.198.254:36219 timed out after 15.000s (SENT)
I20260709 12:13:15.541707 13671 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.542434 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1324) took 1394 ms. Trace:
I20260709 12:13:15.543421 13671 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:15.543234 11328 rpcz_store.cc:276] 0709 12:13:14.147623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.113977 (+966354us) service_pool.cc:224] Handling call
0709 12:13:15.542159 (+428182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.483094 11329 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:15.231577 11930 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:15.484937 13807 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:15.562891 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 440) took 1511 ms. Trace:
I20260709 12:13:15.563424 13807 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:15.563220 13481 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.511817 11256 rpcz_store.cc:276] 0709 12:13:12.595281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.595958 (+   677us) service_pool.cc:224] Handling call
0709 12:13:12.616151 (+ 20193us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:12.740338 (+124187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:12.740387 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:12.748841 (+  8454us) log.cc:844] Serialized 52 byte log entry
0709 12:13:12.826086 (+ 77245us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:12.826138 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.826207 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:15.245179 (+2418972us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:15.405998 (+160819us) raft_consensus.cc:1685] finished
0709 12:13:15.406056 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.406523 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125109,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5590,"spinlock_wait_cycles":38757504,"thread_start_us":162539,"threads_started":2,"wal-append.queue_time_us":125984}
I20260709 12:13:15.564103 13807 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:15.485800 11222 connection.cc:256] RPC call timeout handler was delayed by -1.131894819000081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 12:13:15.564929 13806 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:15.565819 11222 connection.cc:256] RPC call timeout handler was delayed by -1.1264948190000723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 12:13:15.566392 13806 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:15.566789 13481 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.566866 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:15.378211 11078 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 5, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 5 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:15.568274 13952 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:15.569279 13952 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:15.510291 11935 rpcz_store.cc:276] 0709 12:13:12.820764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.821584 (+   820us) service_pool.cc:224] Handling call
0709 12:13:12.821945 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:13.067113 (+245168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:13.067146 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:13.067886 (+   740us) log.cc:844] Serialized 26 byte log entry
0709 12:13:13.788154 (+720268us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:13.788195 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.788273 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:15.401722 (+1613449us) raft_consensus.cc:1685] finished
0709 12:13:15.401797 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.402205 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31961,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":907993,"threads_started":2,"wal-append.queue_time_us":735432}
I20260709 12:13:15.571036 13655 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.570487 11078 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 3 to 4, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 4 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:15.570833 13952 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.571812 13655 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.489049 11255 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:15.477895 11248 rpcz_store.cc:276] 0709 12:13:12.702926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.703048 (+   122us) service_pool.cc:224] Handling call
0709 12:13:12.703360 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.606474 (+903114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.467193 (+1860719us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400108954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:15.467599 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859567872}
I20260709 12:13:15.453307 11253 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:15.409842 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1700) took 2875 ms. Trace:
I20260709 12:13:15.575155 11257 rpcz_store.cc:276] 0709 12:13:12.534127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.534307 (+   180us) service_pool.cc:224] Handling call
0709 12:13:12.534680 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:12.784434 (+249754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:12.784462 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:12.784937 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:13:13.015205 (+230268us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:13.015264 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.015376 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:15.210238 (+2194862us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:15.409145 (+198907us) raft_consensus.cc:1685] finished
0709 12:13:15.409208 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.409647 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135737,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9007,"spinlock_wait_cycles":262131072,"thread_start_us":435755,"threads_started":2,"wal-append.queue_time_us":404445}
I20260709 12:13:15.481519 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 419) took 1201 ms. Trace:
I20260709 12:13:15.412304 13809 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.509106 11181 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.577167 11327 rpcz_store.cc:276] 0709 12:13:14.280188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.479466 (+1199278us) service_pool.cc:224] Handling call
0709 12:13:15.481447 (+  1981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.577909 11181 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:15.578379 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.578877 13591 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.579191 11327 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:15.508392 13714 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.580061 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:15.580523 13714 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:15.500372 13602 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.545099 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:15.512230 11079 rpcz_store.cc:269] 0709 12:13:00.277907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.293289 (+ 15382us) service_pool.cc:224] Handling call
0709 12:13:15.333547 (+15040258us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:06.215295 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:06.216405 (+  1110us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:06.216473 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:13:06.216502 (+    29us) tablet.cc:662] Decoding operations
0709 12:13:06.218462 (+  1960us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:06.218626 (+   164us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:06.218674 (+    48us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:13:06.220709 (+  2035us) tablet.cc:701] Row locks acquired
0709 12:13:06.220786 (+    77us) write_op.cc:260] PREPARE: finished
0709 12:13:06.221065 (+   279us) write_op.cc:270] Start()
0709 12:13:06.221299 (+   234us) write_op.cc:276] Timestamp: P: 1783599186220966 usec, L: 0
0709 12:13:06.221336 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:13:06.384343 (+163007us) log.cc:844] Serialized 5196 byte log entry
0709 12:13:07.701250 (+1316907us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b1800251200. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:08.164397 (+463147us) op_driver.cc:464] REPLICATION: finished
0709 12:13:08.551492 (+387095us) write_op.cc:301] APPLY: starting
0709 12:13:08.563053 (+ 11561us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:08.603715 (+ 40662us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:08.603769 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:08.607929 (+  4160us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:08.608274 (+   345us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:13:08.608326 (+    52us) write_op.cc:312] APPLY: finished
0709 12:13:08.608676 (+   350us) log.cc:844] Serialized 95 byte log entry
0709 12:13:08.608857 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:08.609006 (+   149us) write_op.cc:454] Released schema lock
0709 12:13:08.609080 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:08.642527 (+ 33447us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:08.645585 (+  3058us) write_op.cc:454] Released schema lock
0709 12:13:09.036243 (+390658us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e932a 00007f3ec666ef14 00007f3ec666eec3 00007f3ec666ee38 00007f3ec666edf3 00007f3ec666e0dc 00007f3ec6697e16 00007f3ec667fcbb 00007f3ec666a39d 00007f3ec666dd75
Metrics: {"mutex_wait_us":8323,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":329218,"apply.run_cpu_time_us":4432,"apply.run_wall_time_us":98559,"num_ops":13,"prepare.queue_time_us":88691,"prepare.run_cpu_time_us":21327,"prepare.run_wall_time_us":518547,"raft.queue_time_us":292188,"raft.run_cpu_time_us":23059,"raft.run_wall_time_us":1011265,"replication_time_us":1903375,"spinlock_wait_cycles":176265344,"thread_start_us":685186,"threads_started":4,"wal-append.queue_time_us":366789,"wal-append.run_cpu_time_us":11879,"wal-append.run_wall_time_us":1918396}]]}
I20260709 12:13:15.530110 11326 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:15.584803 13908 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:15.530691 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.545969 11331 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:15.554325 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:15.556762 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.509863 13554 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:15.557289 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:15.561749 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78098.5us
I20260709 12:13:15.559837 13850 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:15.380890 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60595.1us
I20260709 12:13:15.563500 11930 rpcz_store.cc:276] 0709 12:13:14.051056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.066689 (+ 15633us) service_pool.cc:224] Handling call
0709 12:13:15.562640 (+1495951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.580411 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.589757 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 420) took 1270 ms. Trace:
I20260709 12:13:15.606182 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:15.606678 11329 rpcz_store.cc:276] 0709 12:13:14.280449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.481377 (+1200928us) service_pool.cc:224] Handling call
0709 12:13:15.550770 (+ 69393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.607314 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:15.608153 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57262.1us
I20260709 12:13:15.621510 11322 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:15.607690 11078 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 4, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 4 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:15.588461 13953 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:15.588575 13954 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:15.608884 13923 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:15.592075 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:15.589720 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.633222 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:15.638132 13640 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:15.653887 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.654254 13640 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:15.611359 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 421) took 1330 ms. Trace:
I20260709 12:13:15.585456 13908 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:15.594638 13937 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:15.612037 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.643615 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1321) took 1500 ms. Trace:
I20260709 12:13:15.585462 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.585796 13807 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:15.652842 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 71096us
I20260709 12:13:15.615178 11178 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:13:15.616021 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:15.654798 11927 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:13:15.660392 11177 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.616632 13812 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.632825 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:15.608798 11329 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:13:15.530228 11323 rpcz_store.cc:276] 0709 12:13:14.188361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.178069 (+989708us) service_pool.cc:224] Handling call
0709 12:13:15.409209 (+231140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.623564 11327 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:15.661504 11181 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:15.626528 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:15.409955 13933 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.662483 13881 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:15.651702 13953 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:15.662803 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 406) took 1514 ms. Trace:
I20260709 12:13:15.663071 13933 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.663576 11222 leader_election.cc:400] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7.
I20260709 12:13:15.664115 11222 leader_election.cc:403] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:13:15.664774 13953 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:15.665390 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 410) took 1477 ms. Trace:
I20260709 12:13:15.608219 11173 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:15.665795 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.628211 13850 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.665861 11323 rpcz_store.cc:276] 0709 12:13:14.187501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.664148 (+1476647us) service_pool.cc:224] Handling call
0709 12:13:15.664657 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.665024 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:15.665058 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:15.665106 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:15.665138 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.665198 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.665314 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.667176 13809 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.668140 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 412) took 1480 ms. Trace:
I20260709 12:13:15.668396 11928 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:15.662519 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1323) took 1517 ms. Trace:
I20260709 12:13:15.668720 13937 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7.
I20260709 12:13:15.669293 13933 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.669235 11329 rpcz_store.cc:276] 0709 12:13:14.147233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.607824 (+1460591us) service_pool.cc:224] Handling call
0709 12:13:15.608404 (+   580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:15.662233 (+ 53829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.662445 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.654780 13640 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:15.671511 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 415) took 1482 ms. Trace:
I20260709 12:13:15.412626 13623 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.633692 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.654302 11328 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:15.610692 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:15.654814 11331 rpcz_store.cc:276] 0709 12:13:14.280708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.460349 (+1179641us) service_pool.cc:224] Handling call
0709 12:13:15.611288 (+150939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.668542 11323 rpcz_store.cc:276] 0709 12:13:14.188072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.666944 (+1478872us) service_pool.cc:224] Handling call
0709 12:13:15.667429 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.667773 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:15.667799 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:15.667877 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:15.667926 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.667978 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.668079 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.590866 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.653702 11255 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:15.674752 11325 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.629545 13655 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:15.410005 13942 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.620451 11935 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:15.626053 11175 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.652964 11078 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 1 to 5, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 5 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:15.652055 13954 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.663265 11327 rpcz_store.cc:276] 0709 12:13:14.147943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.622664 (+1474721us) service_pool.cc:224] Handling call
0709 12:13:15.623195 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.662565 (+ 39370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.662755 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.671891 11329 rpcz_store.cc:276] 0709 12:13:14.188968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.670373 (+1481405us) service_pool.cc:224] Handling call
0709 12:13:15.670837 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.671141 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:15.671170 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:15.671215 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:15.671266 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.671323 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.671455 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.656003 11322 rpcz_store.cc:276] 0709 12:13:14.144084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.455347 (+1311263us) service_pool.cc:224] Handling call
0709 12:13:15.643514 (+188167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:15.676818 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 66354.9us
I20260709 12:13:15.678546 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 407) took 1498 ms. Trace:
I20260709 12:13:15.653151 13554 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.681362 11328 rpcz_store.cc:276] 0709 12:13:14.186425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.653698 (+1467273us) service_pool.cc:224] Handling call
0709 12:13:15.654100 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.675444 (+ 21344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.675701 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.684733 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.667205 13812 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:15.686527 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60306.3us
I20260709 12:13:15.687772 11326 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:15.688457 13825 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:15.688973 13753 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:15.689066 11175 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:15.688345 11323 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:15.691885 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.673002 13850 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.680150 13923 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:15.685889 11253 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 3.
I20260709 12:13:15.694537 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 417) took 1505 ms. Trace:
I20260709 12:13:15.696565 11323 rpcz_store.cc:276] 0709 12:13:14.189596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.676461 (+1486865us) service_pool.cc:224] Handling call
0709 12:13:15.688146 (+ 11685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.693084 (+  4938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.694477 (+  1393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.695228 13807 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:15.689237 13825 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:15.678844 13953 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:15.696038 11181 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:15.693922 11078 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 3 to 4, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 4 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:15.706610 11327 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:13:15.707087 11327 pending_rounds.cc:85] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:13:15.707401 11327 pending_rounds.cc:107] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:13:15.707630 11327 raft_consensus.cc:2889] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:15.704602 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:15.709477 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.697989 13923 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:15.711611 13923 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:15.712224 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 79562.6us
I20260709 12:13:15.698028 13881 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.687858 13554 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.716837 11257 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:15.716235 11078 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:15.716741 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:15.722229 11331 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:15.723120 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 416) took 1533 ms. Trace:
I20260709 12:13:15.723537 11331 rpcz_store.cc:276] 0709 12:13:14.189293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.680486 (+1491193us) service_pool.cc:224] Handling call
0709 12:13:15.717320 (+ 36834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.722761 (+  5441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.723023 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.724638 11330 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:15.717871 13881 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.710245 13933 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 12:13:15.733954 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.734714 13733 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:15.726634 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1322) took 1576 ms. Trace:
I20260709 12:13:15.711189 13953 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.713069 13942 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.743705 13942 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:15.743705 13953 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:15.744663 13779 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:15.745067 13937 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:15.747061 13554 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:15.748521 13881 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.752254 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:15.754347 11900 leader_election.cc:400] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
I20260709 12:13:15.754784 11900 leader_election.cc:403] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:15.755532 13942 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:15.732481 13591 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:15.739646 11254 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:15.745148 13779 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:15.762045 13779 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:15.763131 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.765333 13942 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
W20260709 12:13:15.766305 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.767853 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 411) took 1549 ms. Trace:
I20260709 12:13:15.744001 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:15.768242 11330 rpcz_store.cc:276] 0709 12:13:14.187781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.181155 (+993374us) service_pool.cc:224] Handling call
0709 12:13:15.725341 (+544186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:15.783493 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:15.786597 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54774.7us
W20260709 12:13:15.794775 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.737478 11324 rpcz_store.cc:276] 0709 12:13:14.144428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.490320 (+1345892us) service_pool.cc:224] Handling call
0709 12:13:15.490865 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.491461 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:15.676199 (+184738us) log.cc:844] Serialized 14 byte log entry
0709 12:13:15.720467 (+ 44268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:15.720608 (+   141us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:15.720695 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.720789 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.720954 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9122,"prepare.queue_time_us":224038,"raft.queue_time_us":26338,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":180,"thread_start_us":122117,"threads_started":2}
I20260709 12:13:15.795636 13954 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:15.803586 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.805945 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:15.807579 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:15.808693 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.809438 11935 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:15.811190 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:15.811905 11935 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:15.813951 11254 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:15.816329 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:15.817130 11175 raft_consensus.cc:2376] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8. Re-sending same reply.
W20260709 12:13:15.818138 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 183487us
I20260709 12:13:15.818441 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.819754 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:15.793533 13881 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:13:15.827481 13881 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:15.829002 13942 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:15.829492 13881 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:15.830461 13942 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:15.830857 13942 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:15.828789 11181 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:13:15.828115 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.831897 13942 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.832674 11173 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.833159 13942 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:15.827661 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.834623 11178 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:15.834610 13825 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.830646 13809 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:15.830698 13881 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:15.840176 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.790149 13937 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:15.840220 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.828065 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:15.845180 11177 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:15.828855 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:15.851645 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:15.853662 13690 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:15.854125 13690 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:15.854502 13690 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:15.856808 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:15.857980 13671 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:15.860776 11928 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:15.833467 11255 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:15.848686 13591 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:13:15.863137 13591 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:15.863566 13591 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:15.864522 13591 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.873111 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:15.873432 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.872627 11329 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Committing config change with OpId 6.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:15.873873 11173 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:15.873157 11327 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:15.876144 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.876971 11327 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:15.877255 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.877882 11331 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:15.878079 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.879024 11326 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:15.880043 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.882126 11324 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:15.882279 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.883409 11330 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:15.884203 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.885166 11323 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:15.885816 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 441) took 1834 ms. Trace:
I20260709 12:13:15.890267 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:15.890434 11182 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:15.890918 11325 raft_consensus.cc:2376] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8. Re-sending same reply.
I20260709 12:13:15.890584 11928 rpcz_store.cc:276] 0709 12:13:14.051386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.079405 (+ 28019us) service_pool.cc:224] Handling call
0709 12:13:15.399474 (+1320069us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:15.885619 (+486145us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540053a7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:15.885755 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378967168}
I20260709 12:13:15.885946 13908 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.892887 11325 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:15.887255 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.894048 11322 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:15.895877 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.896824 11324 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 3.
I20260709 12:13:15.887544 11181 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:15.899390 13690 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:15.887938 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.888324 11326 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:15.888957 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.889447 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:15.902264 11323 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:15.903805 13908 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:15.890069 11327 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:15.904593 11327 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:15.905120 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:15.906073 11928 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:15.905463 11327 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.907083 11327 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:15.908748 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:15.909476 11257 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:15.911269 11256 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:15.912763 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:15.913312 11254 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:15.920920 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:15.922145 11256 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:15.924185 13690 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:15.926352 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 98514.3us
I20260709 12:13:15.931922 13954 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:15.942473 13954 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:15.942935 13954 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:15.871258 13779 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:15.948068 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 119271us
I20260709 12:13:15.983979 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:15.989415 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:15.990314 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 170681us
I20260709 12:13:15.825623 13640 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:15.995884 11253 raft_consensus.cc:1217] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[5.4-5.4]   Dedup: 5.4->[]
I20260709 12:13:16.004590 11253 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:13:16.006605 11255 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:15.987474 13942 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:13:16.009603 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 257420us
W20260709 12:13:16.016475 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 209067us
I20260709 12:13:15.995582 13591 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:16.030241 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.068879 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:16.076082 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.053609 13942 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:16.082224 11257 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:16.099247 11255 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:16.101578 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 76455.6us
I20260709 12:13:16.069432 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:16.063910 13937 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:16.112076 11253 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:16.062199 13807 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:13:16.114614 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:16.105780 13952 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:16.121850 11255 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:16.122550 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.126698 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:16.128429 11928 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:16.129977 13942 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.131738 13967 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:13:16.132406 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.132988 13964 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:16.134334 13640 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:16.140026 13963 raft_consensus.cc:1064] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:13:16.140468 13554 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:16.144910 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:16.150565 13807 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:16.151978 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.152077 13779 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:16.141201 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.156059 11253 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:16.167306 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.153422 13952 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.168025 11256 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:16.176901 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50379.4us
I20260709 12:13:16.167598 11327 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
W20260709 12:13:16.168243 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 89007.7us
I20260709 12:13:16.175061 11254 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
I20260709 12:13:16.178323 11327 pending_rounds.cc:85] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:13:16.178632 11327 pending_rounds.cc:107] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:13:16.179008 11327 raft_consensus.cc:2889] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:16.179649 13952 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:16.176462 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:16.168802 13602 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:16.182859 13602 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:16.183317 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.183544 13554 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:16.184006 13554 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:16.183823 13602 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.184433 13554 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.185226 13602 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.185851 11257 raft_consensus.cc:2376] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5. Re-sending same reply.
I20260709 12:13:16.181774 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:16.168078 13690 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:16.185241 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.188906 13953 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:13:16.180051 13825 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:16.197249 13937 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.198642 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:16.199326 13648 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:16.199589 11928 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:16.200551 13954 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.185916 13714 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:16.201905 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:16.208916 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.198123 13825 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:16.219808 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.193884 13623 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:16.231113 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 43559.3us
I20260709 12:13:16.185978 13967 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:16.237939 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:16.194779 13640 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:16.244853 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63190.2us
I20260709 12:13:16.228055 13690 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:16.201920 13714 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:16.211995 13908 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:16.231803 13953 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:16.232506 13825 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:16.250494 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.227234 13648 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:16.273267 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:16.255501 13923 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.003s
I20260709 12:13:16.250358 13908 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:16.254125 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:16.278673 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.251338 13953 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:16.281473 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.285738 13740 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:16.255915 13623 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:16.296090 11251 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:13:16.286149 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.297292 11257 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:16.287465 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.287099 13648 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.287781 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:16.288172 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:16.288358 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 86552us
I20260709 12:13:16.255034 13964 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:16.287101 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.288456 13634 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:13:16.301255 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51030.2us
I20260709 12:13:16.298954 11249 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:16.304919 11256 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:16.307852 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 249804us
I20260709 12:13:16.289096 13908 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.314724 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.318367 11255 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:16.318701 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.277861 13923 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.272935 13690 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.314013 13806 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:16.309491 13671 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:13:16.322604 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:16.323071 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:16.323530 11928 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:16.313864 13714 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.275048 13812 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:13:16.328729 13806 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:16.295874 13953 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.331391 11253 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:16.336404 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80315.8us
I20260709 12:13:16.300616 13964 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:16.340559 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:16.343807 11175 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:16.339501 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:16.345723 13954 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:16.346279 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.346966 11252 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:16.345748 13634 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:16.347977 13634 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.348294 13634 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:16.302891 13779 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:13:16.277856 13965 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 12:13:16.352281 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47753.9us
I20260709 12:13:16.355223 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.356302 11254 raft_consensus.cc:2410] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:13:16.358167 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.358489 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:16.351976 13881 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.013s	sys 0.033s
I20260709 12:13:16.335793 13923 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:16.367319 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.368247 11252 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:16.350385 13964 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:16.366819 13881 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.013s	sys 0.033s
I20260709 12:13:16.374671 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:16.382592 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.387681 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:16.388652 11327 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:16.390466 11254 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:16.372335 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:16.396471 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:16.397302 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.397377 11927 raft_consensus.cc:2410] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:13:16.398845 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.399524 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.398106 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44819.3us
I20260709 12:13:16.405205 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:16.374614 13602 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:13:16.389961 13591 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:16.395236 13923 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:16.457185 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:16.458374 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:16.426304 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.456427 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.392454 13740 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:16.400935 13881 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:16.406800 11928 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:16.484748 11175 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:16.463362 13974 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:16.457618 13923 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:16.510867 11930 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:13:16.511587 13963 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.467552 13591 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:16.517997 13966 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:13:16.519598 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:16.488292 13671 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:16.521097 11932 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Committing config change with OpId 6.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:16.463959 13967 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.528378 13952 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.382129 13623 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:16.533205 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 132453us
W20260709 12:13:16.534065 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.535082 11927 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:16.466189 13881 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:16.494897 13740 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.543305 11256 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:16.604403 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.605196 11256 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.549562 11327 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:16.561365 13965 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:16.562139 13591 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:16.535719 13733 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:16.495054 13974 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.534499 13923 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:16.573909 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 250928us
I20260709 12:13:16.509342 13554 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.571887 13640 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:16.628428 11250 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:13:16.603566 13881 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.679140 11256 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:16.682428 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.685314 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.619027 13974 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:16.692086 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79666.5us
W20260709 12:13:16.679553 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 391819us
W20260709 12:13:16.701125 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.712061 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.629982 13965 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.662457 13634 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.732437 13954 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:16.732978 13954 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:16.734161 13954 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.735666 13954 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:16.737761 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.741328 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.744321 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 468246us
I20260709 12:13:16.744660 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.749749 11935 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:16.752509 11251 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.755591 11249 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:16.755594 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 340538us
W20260709 12:13:16.753862 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 377240us
I20260709 12:13:16.711897 13779 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:13:16.764370 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.765901 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.771028 11254 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:13:16.770938 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.763931 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:16.791443 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 378095us
I20260709 12:13:16.759953 13733 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:16.799566 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.800846 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:16.802747 11253 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Committing config change with OpId 6.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:16.796039 13640 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:16.814316 11326 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:16.797900 13690 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:16.819554 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.821280 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.822549 13806 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:16.825809 13986 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:16.826483 13986 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.827899 11928 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:16.828791 13986 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:16.829591 13673 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:13:16.830449 13673 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:16.830798 13673 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:16.831504 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.831965 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:16.832414 13869 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:13:16.833235 11181 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:13:16.833966 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.834770 11249 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:16.809319 13952 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:16.834852 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:16.835644 11175 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:16.835691 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.837014 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.816012 13648 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:16.837980 13987 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:16.838538 13997 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:16.839008 13997 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:16.839493 13997 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.828846 13740 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:16.831596 13673 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.838012 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:16.858347 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.860442 11254 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
W20260709 12:13:16.866640 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.841327 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:16.870539 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:16.873679 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67927.2us
W20260709 12:13:16.889706 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.835554 13640 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:16.904259 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.839018 13714 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:16.906165 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.840592 13591 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:16.908413 11175 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:16.911368 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.808656 13733 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:16.873049 13908 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.002s
I20260709 12:13:16.920100 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.864673 13554 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:16.859970 13690 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:16.924609 11249 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:16.925472 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.927333 11331 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:13:16.933749 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 167190us
I20260709 12:13:16.927889 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:16.928489 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:16.919338 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:16.930336 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:16.926818 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:16.935827 11324 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:16.918761 13806 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:16.937402 13806 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:16.918979 13671 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:16.870200 13602 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:13:16.937420 11326 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:16.937865 13806 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:16.944857 11325 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:16.945230 11250 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:16.946038 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.913558 13714 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.933435 13673 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:16.968605 11249 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:13:16.953737 13809 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:13:16.973444 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:16.952755 13714 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:16.929080 13908 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:16.973428 13993 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 12:13:16.981971 13992 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:13:16.985720 13983 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:16.986145 11250 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:16.985841 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:16.985512 13923 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:16.944314 13869 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
W20260709 12:13:17.010251 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.975260 13634 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:17.018385 13983 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:17.022629 13988 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:13:17.024627 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.025463 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.027809 13881 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.028614 13690 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.048379 11251 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:17.038127 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 196863us
I20260709 12:13:17.020330 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:17.038661 13733 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:17.052222 14001 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:13:17.090248 13983 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.082852 13554 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.054108 13965 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.091763 13983 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:17.054278 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 93077us
I20260709 12:13:17.091990 13554 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.092753 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.078915 13954 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:17.094125 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.095566 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.095736 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:17.054627 13640 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.096980 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:17.055882 13908 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.057843 11178 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:17.097535 11324 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:17.056587 13953 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.712s	user 0.005s	sys 0.004s
I20260709 12:13:17.059994 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.062354 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:17.067869 11329 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
W20260709 12:13:17.070225 13964 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.685s	user 0.006s	sys 0.005s
I20260709 12:13:17.087380 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:17.081655 13937 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:17.090397 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.094214 13554 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:17.094014 13954 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:17.058249 13825 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.783s	user 0.010s	sys 0.004s
I20260709 12:13:17.100265 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:17.067894 13923 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:17.106681 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.127223 11324 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:17.106911 11248 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:17.118163 11255 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:17.117559 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:17.124022 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:17.127794 11255 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:17.128172 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:17.124662 13964 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:17.128846 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:17.117398 13953 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.106251 13671 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:17.125797 13648 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:17.131467 13997 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:17.132915 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.134552 13806 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.060099 13974 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.137089 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.137825 13972 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:17.139317 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:17.140123 13881 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:17.140879 14006 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.002s
I20260709 12:13:17.086279 13591 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:17.144654 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.145969 11326 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:17.149736 11322 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:17.153270 11257 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:17.106601 13965 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.146466 11255 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.200301 11255 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:17.172845 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 87243.1us
I20260709 12:13:17.173347 11328 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:17.203375 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.204130 11254 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:17.174253 11325 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:13:17.175523 11330 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:17.178866 13733 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.179922 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.180483 13952 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:17.180601 13953 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:17.181901 13809 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.183063 13806 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:17.184844 14007 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:13:17.210206 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:17.186151 14016 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:17.187944 13937 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:17.211140 13937 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:17.213078 11222 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:13:17.213871 13952 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.201222 13673 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:17.216707 13673 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.216835 11326 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:17.217195 13673 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:17.190754 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:17.219417 13602 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:17.219805 13602 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:17.220293 13602 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:17.220582 13671 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.143129 13825 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.221421 13733 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:17.222723 14000 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.169s	user 0.000s	sys 0.002s
I20260709 12:13:17.223522 13825 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.223948 13733 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.193771 11323 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:13:17.198052 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80575.1us
I20260709 12:13:17.207077 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:17.226884 13974 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:17.227496 13974 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:17.227859 13974 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.228180 13974 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:17.228632 13965 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:17.229499 13965 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:17.229828 13965 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:17.230778 13965 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.191550 13964 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.213521 11222 leader_election.cc:403] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:13:17.219856 13954 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:17.242561 11255 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 12:13:17.243012 11255 pending_rounds.cc:85] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 8
I20260709 12:13:17.243251 11255 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 12:13:17.243510 11255 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:17.249135 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 109923us
I20260709 12:13:17.236691 13965 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:17.276903 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50854us
W20260709 12:13:17.277038 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 66941.8us
I20260709 12:13:17.284667 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:17.259930 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 324544us
W20260709 12:13:17.294919 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.297281 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.298197 11324 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:17.304550 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.305709 14014 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:17.306350 14014 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.306820 14014 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:17.257838 13964 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6.
I20260709 12:13:17.313522 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:17.314627 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:17.236806 13714 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.274570 14016 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:17.326012 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 48091us
I20260709 12:13:17.280722 13733 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.312528 13640 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.326624 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:17.315732 13977 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:13:17.331470 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 118613us
I20260709 12:13:17.332888 11331 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:17.339017 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.343406 11325 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:13:17.347779 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:17.348913 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.317049 13986 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:13:17.313350 13972 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:13:17.352497 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:17.333262 13809 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:17.332888 13825 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.363408 13997 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.363880 13825 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.369812 13937 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.373904 13942 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.375500 14016 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.376441 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:17.356659 14003 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.303s	user 0.003s	sys 0.000s
I20260709 12:13:17.377789 13990 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:13:17.378921 13990 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:17.379251 13990 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:17.380079 13990 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.381254 13990 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:17.357893 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.383395 13648 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:17.342479 13952 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:17.319250 13591 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.361725 14018 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:13:17.412324 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.362489 13984 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:13:17.362922 13714 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:17.421861 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95157.3us
W20260709 12:13:17.388777 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.387398 11324 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:17.426146 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:17.429222 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:17.337594 13640 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.381347 13602 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.422366 13933 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:17.435025 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 84507.9us
I20260709 12:13:17.435174 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.437578 11327 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:17.439829 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.427531 13648 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:13:17.434275 13954 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:17.434005 13986 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:17.435253 13963 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:17.466711 13989 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:17.484279 11327 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:17.481309 13673 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.465989 14032 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:13:17.482203 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:17.516109 13963 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:17.437251 13933 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:17.517452 13933 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:17.517334 13963 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.516409 13989 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:17.493522 11323 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:17.463327 14021 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.524010 14021 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:17.525385 11330 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:17.526443 14021 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:17.498205 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58454.7us
I20260709 12:13:17.527467 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.454828 13714 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.498944 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:17.533747 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:17.518986 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 166373us
I20260709 12:13:17.535123 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:17.535615 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:17.525013 13989 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.461767 13986 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.539124 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:17.487737 13733 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:17.538004 13671 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:17.547894 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.549530 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:17.545264 13993 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.559001 14013 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:17.541363 13997 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:17.541815 13986 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:17.563581 11330 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:17.560817 11328 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:17.544816 13733 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.543118 13809 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:17.559571 14013 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:17.543114 13779 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
W20260709 12:13:17.668342 14029 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:13:17.548960 13806 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:17.648456 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 104457us
W20260709 12:13:17.686529 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.552454 13825 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.672410 13937 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:17.675513 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:17.676355 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:17.677121 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:17.678246 14013 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.679179 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.702759 11178 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:17.642793 13997 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:17.672266 11078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 241) took 20114 ms (client timeout 15000 ms). Trace:
I20260709 12:13:17.681399 13954 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:17.683049 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.684206 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.684469 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.557210 13671 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:17.669831 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.669319 14019 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.222s
I20260709 12:13:17.670150 14038 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:17.565191 13923 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 12:13:17.705257 11328 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:17.792672 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:17.819653 11927 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:17.783624 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:17.712007 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:17.823551 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.793111 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.792083 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:17.731293 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:17.712257 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 149155us
I20260709 12:13:17.829049 11182 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:13:17.789994 13983 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.000s
I20260709 12:13:17.648212 13733 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:17.831768 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 188612us
I20260709 12:13:17.836542 13779 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 12:13:17.845328 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 306262us
I20260709 12:13:17.852222 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:17.852260 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:17.836982 11926 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:17.775488 13933 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.853797 14021 raft_consensus.cc:2764] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:13:17.855899 13933 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:17.711506 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.741199 13997 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:17.857123 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:17.859674 13671 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:17.833199 13954 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:17.843536 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:17.830960 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 144451us
I20260709 12:13:17.690716 13806 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.865926 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.899019 11179 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:17.855861 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:17.902019 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46228.1us
I20260709 12:13:17.692691 13809 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:17.900492 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:17.917789 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.919073 11927 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:17.900312 13806 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:17.955936 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.981096 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:17.982111 11927 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:17.923207 14014 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.003s
I20260709 12:13:17.990346 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:17.992405 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 372) took 1003 ms. Trace:
I20260709 12:13:17.994958 11175 rpcz_store.cc:276] 0709 12:13:16.989062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:16.989468 (+   406us) service_pool.cc:224] Handling call
0709 12:13:17.040606 (+ 51138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:17.127422 (+ 86816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:17.127475 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:17.154777 (+ 27302us) log.cc:844] Serialized 26 byte log entry
0709 12:13:17.331495 (+176718us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:17.331532 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:17.331596 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:17.991338 (+659742us) raft_consensus.cc:1685] finished
0709 12:13:17.991357 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:17.992345 (+   988us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84047,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":22169,"thread_start_us":152397,"threads_started":2,"wal-append.queue_time_us":230043}
I20260709 12:13:17.909806 13825 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.896690 13648 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:13:17.911621 13807 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:17.705049 11078 rpcz_store.cc:269] 0709 12:12:57.550453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.551046 (+   593us) service_pool.cc:224] Handling call
0709 12:13:17.664906 (+20113860us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:16.123688 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:16.124449 (+   761us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:16.124516 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:13:16.124535 (+    19us) tablet.cc:662] Decoding operations
0709 12:13:16.125832 (+  1297us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:16.125925 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:16.125964 (+    39us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:13:16.126996 (+  1032us) tablet.cc:701] Row locks acquired
0709 12:13:16.127052 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:13:16.127262 (+   210us) write_op.cc:270] Start()
0709 12:13:16.127445 (+   183us) write_op.cc:276] Timestamp: P: 1783599196127185 usec, L: 0
0709 12:13:16.127463 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:13:16.128244 (+   781us) log.cc:844] Serialized 3632 byte log entry
0709 12:13:16.839556 (+711312us) op_driver.cc:464] REPLICATION: finished
0709 12:13:17.141686 (+302130us) write_op.cc:301] APPLY: starting
0709 12:13:17.141818 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:17.142084 (+   266us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:17.142225 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:17.231178 (+ 88953us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:17.231505 (+   327us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:13:17.231551 (+    46us) write_op.cc:312] APPLY: finished
0709 12:13:17.231929 (+   378us) log.cc:844] Serialized 71 byte log entry
0709 12:13:17.232125 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:17.232249 (+   124us) write_op.cc:454] Released schema lock
0709 12:13:17.232316 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:17.234592 (+  2276us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:17.234651 (+    59us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2658094,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":245110,"apply.run_cpu_time_us":4198,"apply.run_wall_time_us":110508,"num_ops":9,"prepare.queue_time_us":295345,"prepare.run_cpu_time_us":50984,"prepare.run_wall_time_us":251234,"raft.queue_time_us":382457,"raft.run_cpu_time_us":25394,"raft.run_wall_time_us":116537,"replication_time_us":711906,"thread_start_us":728139,"threads_started":4,"wal-append.queue_time_us":281647}]]}
I20260709 12:13:17.910854 13983 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:17.990437 13591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.000s	sys 0.014s
I20260709 12:13:18.030630 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.031525 11178 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:18.013201 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:18.019205 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.019567 11927 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:13:18.021683 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:18.022236 11257 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:18.022521 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:18.034962 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:18.023170 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:18.035738 11256 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:18.019552 11931 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:17.971148 13640 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:18.019554 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:18.036625 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:17.998073 11928 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:17.934078 13933 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:18.037304 11251 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:18.030570 11324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.000s	sys 0.065s
I20260709 12:13:18.067643 14016 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:18.070236 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:18.071228 11324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.000s	sys 0.065s
I20260709 12:13:18.076303 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:18.030968 13591 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.000s	sys 0.014s
W20260709 12:13:18.082981 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 230782us
I20260709 12:13:18.084131 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:18.070600 11255 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:18.089861 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:18.030391 13983 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:18.087806 11180 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
W20260709 12:13:18.094887 11296 connection.cc:256] RPC call timeout handler was delayed by -2.3300258290000784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:13:18.095329 11296 connection.cc:256] RPC call timeout handler was delayed by -2.277825828999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 12:13:18.096042 11296 connection.cc:256] RPC call timeout handler was delayed by -2.0879258290001417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 12:13:18.096361 11296 connection.cc:256] RPC call timeout handler was delayed by -2.0681258290001097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
W20260709 12:13:18.096638 11296 connection.cc:256] RPC call timeout handler was delayed by -1.9664258289999452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 12:13:18.096910 11296 connection.cc:256] RPC call timeout handler was delayed by -1.8836258290000387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 12:13:18.097190 11296 connection.cc:256] RPC call timeout handler was delayed by -1.7756258290000915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
W20260709 12:13:18.097502 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6919258289999561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
W20260709 12:13:18.097836 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6559258290001253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22870000000000001s for a response
W20260709 12:13:18.098197 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1555258289999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2843s for a response
W20260709 12:13:18.098477 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1060708560000876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 12:13:18.098752 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0452683449998403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 12:13:18.009526 13825 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.008739 14014 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.070573 13933 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:18.075757 14016 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:18.107712 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:18.124698 13640 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:18.125011 13933 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.126029 11930 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:18.126500 13933 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:18.127113 13740 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:13:18.129684 14040 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:13:18.124775 11929 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:13:18.132182 13986 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.136075 13986 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:18.139043 11929 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:13:18.144574 14039 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:13:18.122748 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:18.137296 13933 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.150280 13933 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.153162 13673 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:18.100253 11926 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:18.101253 13983 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:18.106163 14033 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.460s	user 0.001s	sys 0.000s
I20260709 12:13:18.205165 11325 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:13:18.207321 11330 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:18.146831 14016 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:18.146186 13591 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.069s
W20260709 12:13:18.215342 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92857.4us
I20260709 12:13:18.133180 13997 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.216646 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:18.217885 14032 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:18.218734 13942 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:18.219846 11174 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:18.220587 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 442) took 1145 ms. Trace:
I20260709 12:13:18.199539 14021 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:18.220989 11322 rpcz_store.cc:276] 0709 12:13:17.075114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.208171 (+133057us) service_pool.cc:224] Handling call
0709 12:13:17.214045 (+  5874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:17.214478 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:17.214513 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:17.215056 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:13:17.429153 (+214097us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:17.429201 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:17.429269 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:18.220344 (+791075us) raft_consensus.cc:1685] finished
0709 12:13:18.220372 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:18.220525 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13598,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":168649,"threads_started":1,"wal-append.queue_time_us":297954}
I20260709 12:13:18.222114 14021 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:18.218648 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 114248us
I20260709 12:13:18.224004 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:18.210364 13952 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.225095 13952 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.200212 13972 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:18.226080 13972 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.222635 14021 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:18.218092 13997 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:18.106007 14018 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:18.214342 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:18.242434 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:18.251796 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.257858 11330 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:18.270112 11174 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:18.218770 14032 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:18.271965 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:18.272717 11329 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:18.210659 13640 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:18.288228 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:18.270174 14045 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:13:18.210568 13733 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.294122 11180 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:18.297840 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.298130 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.298029 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:18.299212 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:18.362027 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.270462 14051 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:18.326803 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:18.277814 14032 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:18.455175 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.249431 14018 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:18.531726 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.006s	sys 0.008s
W20260709 12:13:18.557250 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.007s	sys 0.009s
W20260709 12:13:18.557770 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 658203us
I20260709 12:13:18.291137 13954 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.616724 11174 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:18.617141 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.617893 11175 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:18.635175 11173 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.027s	sys 0.055s
W20260709 12:13:18.648310 11173 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.028s	sys 0.055s
I20260709 12:13:18.504606 13933 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:18.543846 13984 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.011s	sys 0.003s
W20260709 12:13:18.570945 11932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.006s	sys 0.009s
I20260709 12:13:18.449518 13993 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.169s
W20260709 12:13:18.663501 11932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.006s	sys 0.010s
I20260709 12:13:18.672930 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:18.668401 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.674619 11182 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:18.675803 11178 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:18.666538 11253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.011s	sys 0.003s
W20260709 12:13:18.716777 11328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.020s	sys 0.034s
I20260709 12:13:18.723014 11329 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
W20260709 12:13:18.721163 11253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.012s	sys 0.003s
W20260709 12:13:18.726363 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53568.2us
W20260709 12:13:18.663007 13984 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.012s	sys 0.003s
I20260709 12:13:18.754619 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.754853 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:18.740586 11179 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 5: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:18.786356 11176 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:18.681107 13974 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.452s	user 0.010s	sys 0.003s
W20260709 12:13:18.748476 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 524545us
I20260709 12:13:18.744100 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:18.791169 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:18.791913 11328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.024s	sys 0.034s
I20260709 12:13:18.785607 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:18.683627 13990 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.294s	user 0.006s	sys 0.006s
W20260709 12:13:18.722507 13806 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.008s
W20260709 12:13:18.762126 14047 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.400s	user 0.000s	sys 0.001s
I20260709 12:13:18.668219 13967 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:18.806502 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.819897 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:18.821446 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:18.763443 13602 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:18.808528 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:18.761088 14043 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:13:18.811548 11182 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:18.812944 11174 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:18.761787 14063 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:18.762300 13965 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.484s	user 0.012s	sys 0.000s
I20260709 12:13:18.741094 14019 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:18.809152 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.826649 11178 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:18.741726 14014 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:18.835011 11325 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:13:18.835177 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:18.839082 11322 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:18.848371 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:18.850138 11180 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:18.821521 13640 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.865367 11329 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:18.866850 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:18.857694 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:18.873981 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 66208.4us
W20260709 12:13:18.891291 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 56198.4us
I20260709 12:13:18.892091 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:18.892891 11182 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:18.893290 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:18.805879 13974 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.904819 11254 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
W20260709 12:13:18.911801 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:18.912305 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:18.912787 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.913933 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:18.914537 11255 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:18.915292 13989 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:18.915688 13989 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:18.919145 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:18.915877 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:18.922837 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:18.925094 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:18.926522 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:18.926975 11255 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
W20260709 12:13:18.928246 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.927220 11934 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:18.822304 13806 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.867192 13825 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:18.894549 11330 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:18.966856 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:18.922024 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:18.969980 14069 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:18.916591 13989 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.922478 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:18.970593 14069 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:18.971030 14069 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:18.971097 11928 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:18.837327 14063 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:18.974587 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:18.974776 11174 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:18.900408 13671 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:18.975749 11256 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:18.971251 13989 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:18.980386 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:18.982470 11181 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:18.828775 13967 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.787635 14032 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.929122 13997 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:18.929296 14019 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:18.987033 11252 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:13:18.977662 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 84427us
I20260709 12:13:18.993975 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1965) took 1567 ms. Trace:
I20260709 12:13:18.899412 13640 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:18.994437 11324 rpcz_store.cc:276] 0709 12:13:17.426335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.426484 (+   149us) service_pool.cc:224] Handling call
0709 12:13:17.426821 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:17.428981 (+  2160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:17.429015 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:17.431286 (+  2271us) log.cc:844] Serialized 26 byte log entry
0709 12:13:18.076177 (+644891us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:18.076224 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:18.081240 (+  5016us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:18.993756 (+912516us) raft_consensus.cc:1685] finished
0709 12:13:18.993777 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:18.993913 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1973,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":5765,"thread_start_us":644493,"threads_started":1,"wal-append.queue_time_us":851254}
I20260709 12:13:18.996860 14071 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 12:13:19.009842 14066 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:13:18.985610 13671 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:19.030746 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:19.034801 14050 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:18.982046 13825 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:13:18.998121 13997 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:19.054379 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1939) took 1034 ms. Trace:
I20260709 12:13:19.010815 14018 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:19.029935 14048 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.379s	user 0.000s	sys 0.001s
I20260709 12:13:19.013520 14049 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:19.075153 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.011677 13954 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.075965 11248 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:19.001768 14019 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:13:19.073134 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.077490 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1731) took 1078 ms. Trace:
I20260709 12:13:19.074095 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:19.064630 11173 rpcz_store.cc:276] 0709 12:13:18.020311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.021333 (+  1022us) service_pool.cc:224] Handling call
0709 12:13:18.021698 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:18.022145 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:18.648915 (+626770us) log.cc:844] Serialized 14 byte log entry
0709 12:13:18.851618 (+202703us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:18.910326 (+ 58708us) log.cc:844] Serialized 26 byte log entry
0709 12:13:18.910494 (+   168us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:18.910537 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:18.910624 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:19.054151 (+143527us) raft_consensus.cc:1685] finished
0709 12:13:19.054174 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:19.054311 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":846005,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":81420,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":42351,"spinlock_wait_cycles":66681344,"thread_start_us":721452,"threads_started":2,"wal-append.queue_time_us":285635}
I20260709 12:13:19.080976 11257 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:19.074334 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.074824 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:19.082145 11252 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:19.072412 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.083166 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.074685 11927 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:19.078994 11934 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:19.066434 14050 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:19.088357 11147 connection.cc:256] RPC call timeout handler was delayed by -1.7059083180001835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260709 12:13:19.089670 11147 connection.cc:256] RPC call timeout handler was delayed by -1.6681083180001224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 12:13:19.080247 14056 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:13:19.075584 14049 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:19.090926 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:19.114014 11147 connection.cc:256] RPC call timeout handler was delayed by -1.2073083180000594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 12:13:19.114506 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0300083180000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24100000000000002s for a response
I20260709 12:13:19.109915 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.115473 11248 raft_consensus.cc:2368] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7.
I20260709 12:13:19.118245 11173 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:19.080562 11253 rpcz_store.cc:276] 0709 12:13:17.999506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.999685 (+   179us) service_pool.cc:224] Handling call
0709 12:13:18.023478 (+ 23793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:18.030942 (+  7464us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:18.155376 (+124434us) log.cc:844] Serialized 14 byte log entry
0709 12:13:19.076361 (+920985us) log.cc:844] Serialized 14 byte log entry
0709 12:13:19.077132 (+   771us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:19.077197 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:19.077249 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.077316 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:19.077431 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53716,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":8109,"raft.queue_time_us":16319,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":33759,"thread_start_us":982884,"threads_started":3,"wal-append.queue_time_us":854039}
W20260709 12:13:19.121248 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 206508us
I20260709 12:13:19.088290 13602 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.133586 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1357) took 1596 ms. Trace:
I20260709 12:13:19.113865 14050 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:19.122450 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:19.132164 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:19.087596 13954 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.124125 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.134054 11327 rpcz_store.cc:276] 0709 12:13:17.537666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.537817 (+   151us) service_pool.cc:224] Handling call
0709 12:13:17.545538 (+  7721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:17.551921 (+  6383us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:17.551967 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:17.643378 (+ 91411us) log.cc:844] Serialized 26 byte log entry
0709 12:13:18.103902 (+460524us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:18.103943 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:18.104022 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:19.133358 (+1029336us) raft_consensus.cc:1685] finished
0709 12:13:19.133378 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:19.133523 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":933,"raft.queue_time_us":7686,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7914,"thread_start_us":309632,"threads_started":1,"wal-append.queue_time_us":608443}
I20260709 12:13:19.135993 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:19.120388 14049 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:19.112550 13974 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:19.130632 14044 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.178s	user 0.001s	sys 0.001s
I20260709 12:13:19.078087 14018 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:13:19.147998 11253 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:19.087388 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:19.187718 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55633us
I20260709 12:13:19.121604 14075 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:19.193091 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70754us
W20260709 12:13:19.142325 14062 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.155s	user 0.002s	sys 0.000s
I20260709 12:13:19.210908 11250 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:19.123044 13671 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.138577 13997 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.219746 11182 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:19.220364 11930 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:19.221094 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:19.138953 13602 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.197045 14067 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:19.193620 13993 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:19.201140 14077 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:19.201545 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:19.208895 13690 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:19.209842 14019 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.185882 13806 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:19.223070 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:19.228883 11180 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 4: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:19.246634 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:19.258329 11253 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:19.261385 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 352620us
I20260709 12:13:19.226894 14077 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:19.266624 11933 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:19.266942 14077 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:19.267390 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:19.267526 14050 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.268810 13807 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:19.192400 13954 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.260952 14067 raft_consensus.cc:2764] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:13:19.285001 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:19.285333 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:19.286546 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 75082.6us
I20260709 12:13:19.287379 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:19.314644 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:19.355383 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.272614 13806 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:19.449932 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 160162us
I20260709 12:13:19.236404 13993 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.223706 14075 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.238708 14019 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.477566 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:19.479261 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:19.481079 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 260091us
I20260709 12:13:19.497670 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:19.502075 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46002.9us
I20260709 12:13:19.501513 11182 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:19.475803 13967 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:19.511827 13983 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.263s	user 0.013s	sys 0.000s
I20260709 12:13:19.513312 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 456) took 1616 ms. Trace:
I20260709 12:13:19.516193 13602 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:19.516675 14067 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:19.556005 14067 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:19.556452 14067 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:19.517030 14086 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:19.558097 14086 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:19.517052 13972 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.518009 13733 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:19.475433 14014 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:13:19.519970 11173 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
W20260709 12:13:19.522810 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.561718 11253 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:19.562330 11250 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:13:19.562772 14021 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.297s	user 0.012s	sys 0.000s
I20260709 12:13:19.563423 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:19.563196 14021 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.564146 11250 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
W20260709 12:13:19.525789 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.526497 13809 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.464099 13806 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.565939 13809 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:19.567039 13806 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.570061 13967 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.570605 13967 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:19.571545 14074 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.433s	user 0.001s	sys 0.000s
I20260709 12:13:19.572778 13967 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.573307 13967 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:19.528095 13952 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.553833 13983 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.554751 11926 rpcz_store.cc:276] 0709 12:13:17.897006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.898792 (+  1786us) service_pool.cc:224] Handling call
0709 12:13:18.028157 (+129365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:18.204614 (+176457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.512931 (+1308317us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b5400617950. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:19.513190 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55125760}
I20260709 12:13:19.558995 14075 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.563262 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.573612 13993 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:19.573912 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1756) took 1291 ms. Trace:
I20260709 12:13:19.559043 14086 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:19.576279 14081 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.298s	user 0.000s	sys 0.000s
W20260709 12:13:19.576673 14064 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.433s	user 0.000s	sys 0.001s
I20260709 12:13:19.504845 14084 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.501281 14063 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:19.473598 14018 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:13:19.591711 11253 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:19.591109 13967 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.593909 11176 rpcz_store.cc:276] 0709 12:13:18.282040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.803293 (+521253us) service_pool.cc:224] Handling call
0709 12:13:18.824281 (+ 20988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:18.834648 (+ 10367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:18.834676 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:18.850946 (+ 16270us) log.cc:844] Serialized 26 byte log entry
0709 12:13:18.965418 (+114472us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:18.965484 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:18.967999 (+  2515us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:19.573701 (+605702us) raft_consensus.cc:1685] finished
0709 12:13:19.573720 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:19.573854 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7144,"raft.queue_time_us":15000,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":43722,"thread_start_us":98619,"threads_started":1,"wal-append.queue_time_us":211541}
W20260709 12:13:19.589562 14016 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.374s	user 0.014s	sys 0.000s
I20260709 12:13:19.525475 14019 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:19.598240 13989 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 12:13:19.599436 13983 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.630316 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:19.632488 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 123333us
I20260709 12:13:19.608625 14016 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.637456 14016 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.602655 14088 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:13:19.613952 13954 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.640620 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1755) took 1358 ms. Trace:
I20260709 12:13:19.652280 11178 rpcz_store.cc:276] 0709 12:13:18.281763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.737085 (+455322us) service_pool.cc:224] Handling call
0709 12:13:18.826188 (+ 89103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:18.851889 (+ 25701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.640392 (+788503us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54001f36d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:19.640534 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160338816}
I20260709 12:13:19.605129 14084 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:19.648881 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.655264 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.649201 11255 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:19.655654 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.650161 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:19.651374 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.653972 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1360) took 1538 ms. Trace:
I20260709 12:13:19.639128 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:19.655026 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:19.658077 11178 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:19.657249 11328 rpcz_store.cc:276] 0709 12:13:18.115530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.115745 (+   215us) service_pool.cc:224] Handling call
0709 12:13:18.116210 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:18.137988 (+ 21778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:18.138038 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:18.138718 (+   680us) log.cc:844] Serialized 26 byte log entry
0709 12:13:18.803474 (+664756us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:18.803524 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:18.803624 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:19.653747 (+850123us) raft_consensus.cc:1685] finished
0709 12:13:19.653769 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:19.653913 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1523,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"thread_start_us":664245,"threads_started":1,"wal-append.queue_time_us":953122}
I20260709 12:13:19.660759 11331 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:19.665364 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:19.670028 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1967) took 2130 ms. Trace:
I20260709 12:13:19.670483 11323 rpcz_store.cc:276] 0709 12:13:17.528827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.537676 (+  8849us) service_pool.cc:224] Handling call
0709 12:13:17.550854 (+ 13178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:17.564253 (+ 13399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:17.564285 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:17.642932 (+ 78647us) log.cc:844] Serialized 26 byte log entry
0709 12:13:18.840679 (+1197747us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b18003d2640. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:19.094624 (+253945us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:19.094702 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.094808 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:19.659247 (+564439us) raft_consensus.cc:1685] finished
0709 12:13:19.659269 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:19.659464 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4414,"raft.queue_time_us":13072,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":74935,"spinlock_wait_cycles":39813888,"thread_start_us":253673,"threads_started":1,"wal-append.queue_time_us":1546462}
I20260709 12:13:19.674899 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:19.677920 11330 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:13:19.652814 14021 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.679857 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:19.683243 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.678522 11330 pending_rounds.cc:85] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 3
I20260709 12:13:19.685124 11330 pending_rounds.cc:107] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:13:19.685443 11330 raft_consensus.cc:2889] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:19.686088 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 47053.7us
W20260709 12:13:19.684377 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 91065.7us
I20260709 12:13:19.687000 11326 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:19.687963 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:19.688529 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:19.690271 11176 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:19.692054 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.692864 11927 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:19.688243 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:19.694622 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:19.694326 14077 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.696107 11178 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:19.684834 14075 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:19.696955 14075 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:19.697887 14075 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.699831 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:19.700426 11330 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:19.700496 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1926) took 2052 ms. Trace:
I20260709 12:13:19.701067 14084 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:19.701946 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.702615 11326 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:19.703430 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.703603 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:19.704263 11256 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
W20260709 12:13:19.704593 14069 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.000s
I20260709 12:13:19.675159 14014 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.706290 14014 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:19.706743 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1935) took 1840 ms (client timeout 2150 ms). Trace:
W20260709 12:13:19.707136 11179 rpcz_store.cc:269] 0709 12:13:17.866541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.930032 (+ 63491us) service_pool.cc:224] Handling call
0709 12:13:19.706549 (+1776517us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54000d7854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:19.706687 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704159104}
I20260709 12:13:19.708465 14014 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.708586 11179 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:19.709097 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:19.710201 14086 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:19.675376 14094 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.711527 14094 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.709048 14014 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:19.701895 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:19.770306 11327 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:19.700966 11181 rpcz_store.cc:276] 0709 12:13:17.664883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.665059 (+   176us) service_pool.cc:224] Handling call
0709 12:13:19.700274 (+2035215us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003480d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:19.700409 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2675158400}
I20260709 12:13:19.704499 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:19.697391 11178 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:19.659107 14067 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.764520 13983 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:19.666051 14016 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:13:19.769337 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 81445.5us
I20260709 12:13:19.776052 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.776257 13974 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:13:19.774722 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63505.1us
I20260709 12:13:19.788195 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1762) took 1032 ms. Trace:
I20260709 12:13:19.790205 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.705161 14069 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.820554 11927 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:13:19.779903 14092 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.146s	user 0.000s	sys 0.005s
I20260709 12:13:19.821940 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.822835 11326 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:19.800597 14016 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.823856 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.824105 13952 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:19.824844 11326 raft_consensus.cc:2410] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:19.765851 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.823594 11178 rpcz_store.cc:276] 0709 12:13:18.755704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.696941 (+941237us) service_pool.cc:224] Handling call
0709 12:13:19.697236 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:19.787830 (+ 90594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.788122 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:19.826889 14016 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:19.819610 14077 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:19.819808 14089 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:13:19.816851 11330 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:19.828265 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:19.828948 11330 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:19.829023 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:19.829208 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:19.829736 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.830525 11330 raft_consensus.cc:2410] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:19.830807 11178 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:13:19.830942 14067 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:19.765805 13671 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:13:19.831255 14088 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:19.831991 11178 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:19.721699 14094 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:19.822134 14069 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:19.711551 14086 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:19.791838 13983 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:19.833397 13983 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:19.834529 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:19.835150 14049 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.003s
I20260709 12:13:19.835831 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.836629 11935 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:13:19.829874 11248 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:19.835563 14049 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:19.774824 13997 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:19.837874 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.848330 11931 raft_consensus.cc:2410] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:19.850087 11322 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:19.831256 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.793813 11181 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:13:19.851301 11330 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:19.842365 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.826767 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:19.853513 11927 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:19.856753 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.845371 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:19.858666 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:19.851187 11181 pending_rounds.cc:85] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 2
I20260709 12:13:19.859687 11181 pending_rounds.cc:107] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:13:19.860026 11181 raft_consensus.cc:2889] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:19.860384 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:19.858731 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.862040 11935 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:19.877211 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1947) took 1059 ms. Trace:
I20260709 12:13:19.877840 11181 rpcz_store.cc:276] 0709 12:13:18.817380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.771862 (+954482us) service_pool.cc:224] Handling call
0709 12:13:19.793312 (+ 21450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:19.876848 (+ 83536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.877129 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:19.801658 14084 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:19.906607 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 77651.2us
I20260709 12:13:19.911793 11181 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:13:19.856640 14077 raft_consensus.cc:2764] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:13:19.918558 11327 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:19.873404 14083 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:19.937790 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:19.939234 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:19.946318 14087 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:13:19.956569 13640 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:19.958681 14082 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:19.869956 13989 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:13:19.959959 13967 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:19.973593 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.869041 13983 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:19.974516 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:19.988533 11181 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:20.063289 13807 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:20.106463 11327 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:13:20.106618 11935 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
W20260709 12:13:20.106482 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 246185us
W20260709 12:13:20.024865 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64325.1us
I20260709 12:13:20.107417 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1950) took 1113 ms. Trace:
W20260709 12:13:20.105825 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.113360 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:20.057888 13740 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.114722 13640 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:20.115113 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.116310 13640 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:20.115196 14082 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:20.116834 13640 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.115748 11181 rpcz_store.cc:276] 0709 12:13:18.993908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.913713 (+919805us) service_pool.cc:224] Handling call
0709 12:13:19.988159 (+ 74446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:20.107021 (+118862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.107338 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:20.114380 13740 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.118958 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:20.119333 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:20.119484 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:20.123317 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:20.124502 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:19.964854 14088 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:20.132031 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.219240 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:20.218945 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.122712 11181 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:20.126658 11147 leader_election.cc:400] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7.
I20260709 12:13:20.220680 11147 leader_election.cc:403] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:13:20.221010 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:20.221756 11181 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:20.224745 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:20.221756 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95329.3us
I20260709 12:13:20.100240 14013 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:20.224442 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 101196us
I20260709 12:13:20.227260 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.105819 14077 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:19.912460 14084 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.208666 13967 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:20.118391 13640 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:20.241250 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.219384 13997 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:20.220835 14038 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.055279 13963 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.182s
I20260709 12:13:20.222698 14083 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.118420 14067 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.125s
I20260709 12:13:20.225180 13733 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:20.221446 14055 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:13:20.247339 14014 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:20.239569 13809 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:20.251262 13967 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:20.125859 13807 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.256390 13954 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:20.254222 14096 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:20.284322 11181 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:20.269260 14086 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:20.261047 13997 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:20.250815 14019 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:13:20.294533 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:20.241403 14013 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:20.243805 14050 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:13:20.300154 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:20.229228 14088 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.282298 13809 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.250319 14077 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.314649 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:20.289553 14049 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:20.271831 14038 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.291038 14096 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:20.292207 14086 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:20.298174 13997 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:20.318100 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.295997 14014 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:20.278426 14083 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.331779 11248 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:20.292408 14101 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7.
I20260709 12:13:20.316013 13809 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:20.343605 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:20.344477 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.344460 11248 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:20.348651 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 235716us
I20260709 12:13:20.291383 14082 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.351413 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.351926 14082 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:20.354267 14104 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:13:20.355194 14103 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:13:20.329649 14086 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:20.356127 13967 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.357573 13967 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:20.361307 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 132718us
I20260709 12:13:20.362984 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.366309 14014 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:20.346163 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 2.056s (SENT) [suppressed 9 similar messages]
I20260709 12:13:20.369249 14084 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:20.369788 11933 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:20.371011 14083 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:20.371560 14083 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.372140 14038 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:20.372033 14083 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.373698 14088 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:20.374379 14088 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:20.376044 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.376693 11256 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:20.378094 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1769) took 1471 ms. Trace:
W20260709 12:13:20.388900 11176 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.000s
I20260709 12:13:20.393647 11176 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:13:20.354146 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:20.323385 14096 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:20.378505 11178 rpcz_store.cc:276] 0709 12:13:18.907489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.841420 (+933931us) service_pool.cc:224] Handling call
0709 12:13:19.870384 (+ 28964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.881349 (+ 10965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.881394 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:19.881992 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:13:20.229232 (+347240us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:20.229298 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.229402 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.377878 (+148476us) raft_consensus.cc:1685] finished
0709 12:13:20.377899 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.378032 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58371,"raft.queue_time_us":13799,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":6627,"thread_start_us":319259,"threads_started":1,"wal-append.queue_time_us":388002}
W20260709 12:13:20.423979 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.428372 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.428905 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1771) took 1432 ms. Trace:
I20260709 12:13:20.429294 11256 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:20.429685 11176 rpcz_store.cc:276] 0709 12:13:18.996924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.670945 (+674021us) service_pool.cc:224] Handling call
0709 12:13:20.411619 (+740674us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:20.432188 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 86056.8us
I20260709 12:13:20.426452 11250 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:20.432260 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.435031 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.435595 11176 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:13:20.440519 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1737) took 2393 ms (client timeout 2212 ms). Trace:
W20260709 12:13:20.437146 14105 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:13:20.367861 13997 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:20.442592 11256 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:20.435174 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72133.9us
I20260709 12:13:20.441577 11248 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:20.443585 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:20.441027 11254 rpcz_store.cc:269] 0709 12:13:18.034456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.034698 (+   242us) service_pool.cc:224] Handling call
0709 12:13:20.427429 (+2392731us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54001cfcd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:20.427603 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873351552}
I20260709 12:13:20.444252 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.445554 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.446719 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.447481 13993 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:20.441671 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 376) took 1232 ms. Trace:
I20260709 12:13:20.450850 11176 rpcz_store.cc:276] 0709 12:13:19.207969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.430830 (+1222861us) service_pool.cc:224] Handling call
0709 12:13:20.431313 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:20.440620 (+  9307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.440851 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:20.449108 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.457176 11256 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:20.459822 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1757) took 2088 ms. Trace:
I20260709 12:13:20.458864 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.435900 14067 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:20.460376 11175 rpcz_store.cc:276] 0709 12:13:18.362230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.790639 (+428409us) service_pool.cc:224] Handling call
0709 12:13:18.816818 (+ 26179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.260835 (+444017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.261428 (+   593us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:19.264997 (+  3569us) log.cc:844] Serialized 26 byte log entry
0709 12:13:19.497193 (+232196us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:19.633348 (+136155us) log.cc:844] Serialized 14 byte log entry
0709 12:13:19.634471 (+  1123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.440554 (+806083us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b54003485d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:20.440619 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.440674 (+    55us) raft_consensus.cc:1685] finished
0709 12:13:20.450521 (+  9847us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.450731 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":130949,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":15512,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":37046,"spinlock_wait_cycles":178782976,"thread_start_us":500322,"threads_started":3,"wal-append.queue_time_us":319453}
I20260709 12:13:20.467443 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:20.492914 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.439666 13807 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:20.493727 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:20.494000 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.454545 13983 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:20.450275 14113 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:20.499249 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:20.468562 14125 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:20.487083 14067 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:20.503079 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:20.494879 13942 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:13:20.508062 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.509217 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:20.512691 14077 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:20.514382 11250 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:20.514596 13809 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.499928 13954 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:20.515652 13806 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:20.516013 11256 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:20.516342 14094 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.517278 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:20.517107 13690 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:20.517554 14032 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:20.492880 14084 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:20.518862 14115 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:20.519021 13997 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:20.495710 13740 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:20.523113 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 377) took 1315 ms. Trace:
I20260709 12:13:20.601848 14094 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.605592 11173 rpcz_store.cc:276] 0709 12:13:19.208142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.275798 (+ 67656us) service_pool.cc:224] Handling call
0709 12:13:20.522926 (+1247128us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54002a0154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:20.523042 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538441600}
I20260709 12:13:20.531714 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1938) took 2511 ms. Trace:
I20260709 12:13:20.607215 11177 rpcz_store.cc:276] 0709 12:13:18.019945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.020276 (+   331us) service_pool.cc:224] Handling call
0709 12:13:18.020620 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:18.021005 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:18.021038 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:19.713813 (+1692775us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b2c00013008. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:13:19.819245 (+105432us) log.cc:844] Serialized 26 byte log entry
0709 12:13:19.994836 (+175591us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:20.097184 (+102348us) log.cc:844] Serialized 14 byte log entry
0709 12:13:20.097404 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.097487 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.525592 (+428105us) raft_consensus.cc:1685] finished
0709 12:13:20.531499 (+  5907us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.531652 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100731,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":7063,"raft.queue_time_us":67271,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5042,"spinlock_wait_cycles":34804864,"thread_start_us":175057,"threads_started":1,"wal-append.queue_time_us":471432}
I20260709 12:13:20.608877 11249 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:20.532552 14054 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:20.505492 14088 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:13:20.619258 11249 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:20.599997 14049 raft_consensus.cc:2764] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:20.596246 14084 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:20.621968 11173 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:20.622236 14084 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:20.623533 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1765) took 1807 ms. Trace:
I20260709 12:13:20.623929 11179 rpcz_store.cc:276] 0709 12:13:18.816321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.709401 (+893080us) service_pool.cc:224] Handling call
0709 12:13:19.774654 (+ 65253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.824470 (+ 49816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.824503 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:19.825113 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:13:19.917720 (+ 92607us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:19.917762 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.623101 (+705339us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b54000d8754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:20.623157 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.623215 (+    58us) raft_consensus.cc:1685] finished
0709 12:13:20.623237 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.623466 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114025216,"thread_start_us":92200,"threads_started":1,"wal-append.queue_time_us":457976}
I20260709 12:13:20.514953 14083 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:20.585368 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 68186.3us
W20260709 12:13:20.586186 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:20.597123 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61653.6us
I20260709 12:13:20.530912 14125 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.535198 14067 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:20.608721 14094 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:20.532766 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.611047 11175 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:13:20.640618 11257 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:20.640954 11176 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:20.630414 11249 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:13:20.536355 14038 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:20.631114 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 144064us
W20260709 12:13:20.632565 11174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1766) took 1803 ms (client timeout 2068 ms). Trace:
W20260709 12:13:20.647169 11174 rpcz_store.cc:269] 0709 12:13:18.834290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.847995 (+ 13705us) service_pool.cc:224] Handling call
0709 12:13:20.632364 (+1784369us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540016f8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:20.632499 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256362240}
I20260709 12:13:20.633504 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:20.648396 11324 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:20.636436 11256 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:20.538005 14113 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:20.638505 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:20.633289 14115 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:20.534499 13983 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:13:20.663385 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:20.645902 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:20.664305 11327 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:20.664494 11322 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:20.665191 13965 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:20.655251 13809 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:20.639285 14125 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:20.657763 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:20.632189 13740 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:20.669319 13740 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.662789 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:20.674607 11325 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:20.679620 11252 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:13:20.681015 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.682145 11252 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:20.600641 13806 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:20.684192 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.684720 13806 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:20.657356 14123 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 12:13:20.665807 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.653978 13954 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:20.668004 13690 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:20.685117 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:20.641674 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 378) took 1433 ms. Trace:
I20260709 12:13:20.690699 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:20.825217 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 134627us
I20260709 12:13:20.826480 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:20.691424 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:20.828461 11934 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:20.692658 11929 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:20.692533 14115 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:20.718608 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:20.746639 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.756484 11249 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:20.830616 11176 rpcz_store.cc:276] 0709 12:13:19.208302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.595341 (+1387039us) service_pool.cc:224] Handling call
0709 12:13:20.640782 (+ 45441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:20.641366 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.641576 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:20.843827 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:20.844691 11254 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:20.846856 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:20.842994 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:20.853089 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 168041us
I20260709 12:13:20.666569 13965 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:20.863323 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:20.687774 13807 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:20.855435 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.879153 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.880865 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:20.881930 11329 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:20.867102 11251 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:13:20.658100 14038 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.822841 13690 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.823626 13740 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:20.661921 13640 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:20.863304 14021 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.023s	sys 0.044s
I20260709 12:13:20.858791 14053 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:20.898818 14054 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:20.864902 13967 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.006s
I20260709 12:13:20.862685 14108 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:20.906057 11254 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:20.916736 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53040.7us
I20260709 12:13:20.948587 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:20.949716 11248 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:20.858371 14122 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:20.901399 14053 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.896317 14050 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:20.903112 14021 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.023s	sys 0.044s
I20260709 12:13:20.952696 11325 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:20.899333 13690 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:20.896983 14070 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:20.892624 14014 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:20.953734 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 553) took 1092 ms. Trace:
W20260709 12:13:20.945941 14086 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 12:13:20.892407 14121 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:20.896533 13997 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:13:20.898900 13806 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:13:20.916153 14082 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:13:20.965363 14083 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:20.955096 14049 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:13:21.015338 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:20.915091 14054 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.955662 14085 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:13:21.011674 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:20.994907 14053 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:21.005852 11931 rpcz_store.cc:276] 0709 12:13:19.861525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.861725 (+   200us) service_pool.cc:224] Handling call
0709 12:13:19.908657 (+ 46932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.915373 (+  6716us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:19.915403 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:20.120581 (+205178us) log.cc:844] Serialized 26 byte log entry
0709 12:13:20.447236 (+326655us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:20.447307 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.448239 (+   932us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.953442 (+505203us) raft_consensus.cc:1685] finished
0709 12:13:20.953470 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.953658 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59800,"raft.queue_time_us":2815,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":10476,"thread_start_us":201877,"threads_started":1,"wal-append.queue_time_us":249598}
I20260709 12:13:21.036547 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.036547 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:21.045915 11174 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:21.040143 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.034345 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:21.005476 14050 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:21.038208 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 90812.8us
I20260709 12:13:20.963454 14063 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:13:21.024633 14096 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.003s
I20260709 12:13:21.018591 14121 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:13:21.047392 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.018371 14122 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:21.062407 14121 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:21.063298 14122 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.064270 14019 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.963635 13965 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.064795 14019 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.033720 13740 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:13:21.065913 11179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fd765747f9648049b905c70696b206d"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:21.066651 11179 raft_consensus.cc:606] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.022420 14014 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:21.067003 14019 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.067714 14134 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:21.063840 14094 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.068414 14134 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.068600 14094 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.067791 14014 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.069996 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:21.070899 14134 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.071393 14134 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.071827 14134 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.072348 14094 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.072952 14094 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:21.074019 14094 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:21.067682 14019 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:21.074980 14019 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:21.075676 14102 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.844s	user 0.005s	sys 0.005s
I20260709 12:13:21.077008 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1364) took 1411 ms. Trace:
I20260709 12:13:21.078008 14019 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.078642 14019 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:21.078980 14019 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:21.079942 14019 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.079572 14050 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.081430 14019 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:21.061899 14143 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.077358 11328 rpcz_store.cc:276] 0709 12:13:19.665365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.676554 (+ 11189us) service_pool.cc:224] Handling call
0709 12:13:19.677426 (+   872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.707489 (+ 30063us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:19.707517 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:19.707992 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:13:19.858937 (+150945us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:19.858974 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.859042 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:21.074988 (+1215946us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:21.076734 (+  1746us) raft_consensus.cc:1685] finished
0709 12:13:21.076755 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:21.076949 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20016,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":21616896,"thread_start_us":150547,"threads_started":1,"wal-append.queue_time_us":324188}
I20260709 12:13:21.018778 14086 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.035420 14082 consensus_queue.cc:1243] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:21.046885 13967 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.057811 14070 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:21.057072 14139 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.130640 11174 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:21.122586 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.117071 14096 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.139910 14096 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:21.141656 11181 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:21.152711 14118 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.153679 14118 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.156044 13806 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.156656 13806 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.156908 13690 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:20.998710 14113 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:21.158444 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:21.162763 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.099721 14083 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:13:21.176608 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.169960 14140 raft_consensus.cc:2764] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:13:21.181521 14133 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:13:21.181939 13690 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:21.182878 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 119358us
W20260709 12:13:21.183390 14131 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:13:21.183625 14140 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:21.183799 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.184195 14140 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.184141 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:21.184653 14129 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:13:21.184801 14140 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:21.184965 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.185714 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.121367 13967 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.189601 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:21.172050 14125 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.002s
I20260709 12:13:21.190397 11249 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:21.196247 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.196424 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:21.197364 11251 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:21.197861 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 150758us
I20260709 12:13:21.200093 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:21.200107 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.201011 11254 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:21.174502 14134 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.200008 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:21.202147 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.193677 13690 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:21.116339 14143 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.190378 14038 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:21.211668 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:21.212886 14151 raft_consensus.cc:2764] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:13:21.204222 14108 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:21.217797 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.147361 14139 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.250700 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.200623 14125 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.218719 13690 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:21.216212 13806 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.217862 14049 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:13:21.259788 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 59783.7us
I20260709 12:13:21.262346 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.264065 14139 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.262348 13690 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.265524 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.255005 14108 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:21.223896 14075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.019s	sys 0.010s
I20260709 12:13:21.274408 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.276091 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.259284 14049 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.277839 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.271015 13806 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:13:21.275014 14075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.019s	sys 0.031s
I20260709 12:13:21.272110 14139 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:21.278825 13972 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.298378 11178 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:13:21.279045 14049 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:21.285995 14125 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:21.281347 14141 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:21.280460 14086 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.281221 13806 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.283378 14159 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:21.289324 14139 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:21.283972 14149 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:21.287010 14148 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:21.312657 13972 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:13:21.321355 11930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 560) took 2402 ms (client timeout 1503 ms). Trace:
I20260709 12:13:21.315366 14141 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:21.322144 11930 rpcz_store.cc:269] 0709 12:13:18.918980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.921734 (+  2754us) service_pool.cc:224] Handling call
0709 12:13:21.319806 (+2398072us) spinlock_profiling.cc:243] Waited 59.7 ms on lock 0x7b5400667bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:21.321243 (+  1437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137294592}
I20260709 12:13:21.322706 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 557) took 1059 ms. Trace:
I20260709 12:13:21.323194 14141 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.317848 14139 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:21.325809 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1741) took 2359 ms (client timeout 2226 ms). Trace:
W20260709 12:13:21.326269 11255 rpcz_store.cc:269] 0709 12:13:18.966135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.966607 (+   472us) service_pool.cc:224] Handling call
0709 12:13:21.325590 (+2358983us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b5400068454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:21.325733 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515604736}
I20260709 12:13:21.327446 14053 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.323199 11928 rpcz_store.cc:276] 0709 12:13:20.263347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.266469 (+  3122us) service_pool.cc:224] Handling call
0709 12:13:20.288347 (+ 21878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:20.373998 (+ 85651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:20.374034 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:20.374631 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 12:13:20.649119 (+274488us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:20.658259 (+  9140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.322258 (+663999us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b54003b5454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:21.322312 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:21.322367 (+    55us) raft_consensus.cc:1685] finished
0709 12:13:21.322385 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:21.322640 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6899,"raft.queue_time_us":4908,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":24608,"spinlock_wait_cycles":117925760,"thread_start_us":263483,"threads_started":1,"wal-append.queue_time_us":482671}
I20260709 12:13:21.316557 13806 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:21.347960 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 375) took 2140 ms (client timeout 2056 ms). Trace:
W20260709 12:13:21.351688 11182 rpcz_store.cc:269] 0709 12:13:19.207801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.286991 (+ 79190us) service_pool.cc:224] Handling call
0709 12:13:21.347686 (+2060695us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b540016f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:21.347863 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484354816}
I20260709 12:13:21.342037 13967 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.320031 14148 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:21.331332 14159 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:21.371446 14148 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:21.378350 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 100556us
I20260709 12:13:21.379418 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.380932 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.382000 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.382805 11252 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:21.384711 13967 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.387107 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:21.391126 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 552) took 1540 ms. Trace:
I20260709 12:13:21.391571 11927 rpcz_store.cc:276] 0709 12:13:19.850840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.871525 (+ 20685us) service_pool.cc:224] Handling call
0709 12:13:19.904273 (+ 32748us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.904761 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.904800 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:21.098344 (+1193544us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b4c001976c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec49b8a20 00007f3ec49c9a29 00007f3ec498f644 00007f3ec49b6f2d 00007f3ec49b6b93 00007f3ec66e718b 00007f3ec66e7cb5 00007f3ec66e9004 00007f3ec66e8729 00007f3ec66edd96 00007f3ec66e2adb
0709 12:13:21.116483 (+ 18139us) log.cc:844] Serialized 564 byte log entry
0709 12:13:21.201957 (+ 85474us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:21.202041 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.203421 (+  1380us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:21.390790 (+187369us) raft_consensus.cc:1685] finished
0709 12:13:21.390811 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:21.391061 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9196416,"thread_start_us":51189,"threads_started":1,"wal-append.queue_time_us":145594}
I20260709 12:13:21.383757 14160 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:21.383570 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.350531 14054 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:21.395656 11251 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:21.410567 14054 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:21.394505 14113 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:13:21.411717 14093 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.561s	user 0.010s	sys 0.024s
W20260709 12:13:21.394286 14067 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.745s	user 0.013s	sys 0.000s
I20260709 12:13:21.416230 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:21.431972 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:21.413548 14158 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:13:21.434212 14113 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.377437 14159 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:21.443977 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:21.445103 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.464016 14151 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.446177 14071 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.433233 14067 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.479671 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.478600 11252 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:21.480963 14169 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:13:21.470160 14113 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:13:21.481725 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.502779 13967 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:21.503042 14049 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:21.479280 14084 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.856s	user 0.013s	sys 0.001s
I20260709 12:13:21.506994 14084 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.482389 14118 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:21.509406 14084 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:21.483589 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:21.526829 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:21.527772 11252 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
W20260709 12:13:21.529134 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 881728us
I20260709 12:13:21.529871 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.447415 14140 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:21.533314 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49815.5us
I20260709 12:13:21.534698 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.479425 14054 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:13:21.534049 14113 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.525002 14085 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:21.556746 14085 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:21.557652 14085 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.559420 14067 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:21.547181 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.561223 14053 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.548307 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:21.534051 14146 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:21.565274 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 1989) took 1033 ms. Trace:
I20260709 12:13:21.565405 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.566371 11329 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:21.566308 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:21.549525 14096 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.567683 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.549935 14140 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.551292 14070 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.551836 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 379) took 2337 ms. Trace:
I20260709 12:13:21.599274 11173 rpcz_store.cc:276] 0709 12:13:19.214306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.606520 (+1392214us) service_pool.cc:224] Handling call
0709 12:13:20.606885 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:20.622412 (+ 15527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.551566 (+929154us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b54002e40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:21.551750 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34189184}
I20260709 12:13:21.552264 14128 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:21.552665 13983 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:21.506505 14071 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.562502 11325 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:21.567036 11176 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:21.488312 11076 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:13:21.565771 11330 rpcz_store.cc:276] 0709 12:13:20.531880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.532661 (+   781us) service_pool.cc:224] Handling call
0709 12:13:20.533104 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:20.533457 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:20.533790 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 12:13:21.508963 (+975173us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b18003ada40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:13:21.509204 (+   241us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:21.518270 (+  9066us) log.cc:844] Serialized 26 byte log entry
0709 12:13:21.518425 (+   155us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:21.518461 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.518525 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:21.555016 (+ 36491us) raft_consensus.cc:1685] finished
0709 12:13:21.565069 (+ 10053us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:21.565216 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":919912,"prepare.queue_time_us":589,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":977772,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":27538,"spinlock_wait_cycles":6629120,"thread_start_us":94621,"threads_started":1,"wal-append.queue_time_us":416887}
I20260709 12:13:21.596778 11182 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:21.597312 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:21.606567 11930 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:21.567632 14096 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.565542 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:21.595955 14069 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:21.618270 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:21.609156 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:21.536854 14148 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.602312 13983 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:21.602357 13740 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:21.621018 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.673616 11254 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:21.604535 13974 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.768s	user 0.010s	sys 0.016s
I20260709 12:13:21.644542 11930 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:21.691689 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.626372 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:21.627712 14069 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:21.613600 11076 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 4 to 5, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 5 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:21.692387 11927 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:21.629300 11926 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
W20260709 12:13:21.629899 14121 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.567s	user 0.013s	sys 0.000s
W20260709 12:13:21.603754 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 55606.2us
I20260709 12:13:21.695863 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:21.639707 11329 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:21.647343 14032 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:21.647500 13983 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.650806 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1772) took 2653 ms. Trace:
I20260709 12:13:21.703361 11181 rpcz_store.cc:276] 0709 12:13:18.997268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.323467 (+1326199us) service_pool.cc:224] Handling call
0709 12:13:20.323830 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:21.142135 (+818305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.650472 (+508337us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b54002576d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:21.650736 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880172800}
W20260709 12:13:21.652242 14078 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 2.449s	user 0.011s	sys 0.000s
I20260709 12:13:21.608471 14084 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:13:21.654624 13965 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:21.662322 14143 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.663905 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1952) took 2582 ms. Trace:
I20260709 12:13:21.631767 14049 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.609866 14128 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:21.610364 14053 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:13:21.666743 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.716677 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:21.670814 13807 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:21.671066 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.670924 13806 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.671903 13640 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:21.666950 14071 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.665491 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 338) took 1221 ms. Trace:
I20260709 12:13:21.674706 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:21.673426 13740 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:21.682654 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:21.691143 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.693217 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:21.730880 11928 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:21.705706 11178 rpcz_store.cc:276] 0709 12:13:19.081903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.427511 (+1345608us) service_pool.cc:224] Handling call
0709 12:13:20.454122 (+ 26611us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:21.304045 (+849923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.663614 (+359569us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:21.663839 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940914944}
I20260709 12:13:21.714951 11179 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:21.714818 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:21.732236 11929 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:21.703058 14032 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.671811 14127 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:21.716061 14159 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:21.719264 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:21.724058 14143 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:21.671769 14139 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:21.726084 11248 rpcz_store.cc:276] 0709 12:13:20.444411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.444603 (+   192us) service_pool.cc:224] Handling call
0709 12:13:21.665275 (+1220672us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54002a49d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:21.665397 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157496064}
I20260709 12:13:21.750281 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.679689 11182 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:21.725194 11076 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:21.750281 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:21.751721 11076 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:21.728588 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53997.2us
I20260709 12:13:21.729662 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:21.753736 11173 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:21.693316 13954 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:21.726760 13807 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:21.713543 11254 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:21.692015 13974 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.855s	user 0.010s	sys 0.016s
I20260709 12:13:21.728677 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:21.719034 11330 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:21.734378 13740 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:21.705701 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 464) took 2022 ms. Trace:
I20260709 12:13:21.746351 14032 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:21.755682 11930 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:21.794929 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:21.705916 14121 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:21.771062 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51894.8us
I20260709 12:13:21.796377 11256 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:21.772368 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 465) took 1115 ms. Trace:
I20260709 12:13:21.745119 11927 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:21.774266 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:21.776669 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:21.748332 11329 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:21.798015 14148 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.744204 13690 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:21.762936 14096 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:21.808694 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.844657 13806 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:21.834875 14061 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:21.745828 14127 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:21.845921 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.846848 11254 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:21.812294 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:21.829160 14069 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:21.845973 14185 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:13:21.771977 11076 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.820230 14053 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:21.851410 14053 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.851774 14053 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:21.848742 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51753.1us
I20260709 12:13:21.805244 11326 rpcz_store.cc:276] 0709 12:13:20.657082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.657286 (+   204us) service_pool.cc:224] Handling call
0709 12:13:20.661253 (+  3967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:20.661490 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:20.663090 (+  1600us) log.cc:844] Serialized 14 byte log entry
0709 12:13:21.772046 (+1108956us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:21.772109 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:21.772152 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.772211 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:21.772314 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":588255,"prepare.queue_time_us":1988,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":586849,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":524105,"threads_started":2,"wal-append.queue_time_us":375792}
I20260709 12:13:21.855515 14159 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:21.809648 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:21.855907 14143 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:13:21.835577 14177 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 12:13:21.828116 14172 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.272s	user 0.001s	sys 0.001s
W20260709 12:13:21.859998 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.859391 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:21.887917 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:21.841346 11929 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:21.865561 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.890885 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.867043 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.889740 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.847355 14061 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:21.775211 14134 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.888839 11933 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:21.775934 13954 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:21.898257 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:21.859488 13690 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.921738 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.923451 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.860983 13993 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:21.926213 11076 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:21.934333 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:21.934964 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:21.861929 14143 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:13:21.862280 14084 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:21.948424 11177 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:21.944885 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 55242.6us
I20260709 12:13:21.952828 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:21.952414 11076 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 4, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 4 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:21.948712 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.934082 13690 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:21.948055 13967 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.925691 11181 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:21.789721 11926 rpcz_store.cc:276] 0709 12:13:19.682740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.684830 (+  2090us) service_pool.cc:224] Handling call
0709 12:13:21.565400 (+1880570us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:21.703082 (+137682us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157046016}
I20260709 12:13:21.898375 14159 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:13:22.010531 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 57782.8us
I20260709 12:13:21.991734 11076 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 6, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 6 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:13:22.016347 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:21.944120 13989 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.847519 14128 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.026490 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:22.027166 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 450) took 4236 ms. Trace:
I20260709 12:13:21.950234 11254 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:21.955104 13993 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:22.032374 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:21.952442 14041 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:22.029428 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:22.027604 11932 rpcz_store.cc:276] 0709 12:13:17.782289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.790096 (+  7807us) service_pool.cc:224] Handling call
0709 12:13:17.790576 (+   480us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:17.818482 (+ 27906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:17.818512 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:17.827774 (+  9262us) log.cc:844] Serialized 52 byte log entry
0709 12:13:18.669309 (+841535us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:18.933922 (+264613us) log.cc:844] Serialized 14 byte log entry
0709 12:13:21.951093 (+3017171us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b54006912a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6667733 00007f3ec66ecdbf 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c
0709 12:13:21.951149 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.951249 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:21.951295 (+    46us) raft_consensus.cc:1685] finished
0709 12:13:22.018009 (+ 66714us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:22.018172 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104515,"prepare.queue_time_us":384997,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":96,"raft.queue_time_us":109437,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":26967,"spinlock_wait_cycles":72078592,"thread_start_us":1440359,"threads_started":3,"wal-append.queue_time_us":950552,"wal-append.run_cpu_time_us":5854,"wal-append.run_wall_time_us":1726290}
I20260709 12:13:22.028297 13990 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.019213 14117 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:22.017285 14134 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:22.036686 11076 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:22.042073 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:21.935532 14127 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.030083 14140 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:21.973799 14084 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.046622 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:22.047247 11256 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:22.036605 13990 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.051637 13993 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.056180 13990 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.037021 14117 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:22.058995 14117 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.060043 14117 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:22.051846 14165 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.664s	user 0.005s	sys 0.000s
I20260709 12:13:22.052178 14139 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.063686 11076 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 5, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 5 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:22.066582 14083 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.067046 14084 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.067106 14083 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.034816 14174 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:13:22.033073 14146 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:13:22.069512 11255 pending_rounds.cc:85] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 4
I20260709 12:13:22.069857 11255 pending_rounds.cc:107] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:13:22.070086 11255 raft_consensus.cc:2889] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:22.070174 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:22.071534 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:22.072080 11929 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:22.040086 14041 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.074225 11256 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:22.075749 14041 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:22.065356 11076 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 4 to 5, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 5 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:22.027556 14061 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.080583 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 386) took 1646 ms. Trace:
I20260709 12:13:22.081036 11175 rpcz_store.cc:276] 0709 12:13:20.434543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.508954 (+ 74411us) service_pool.cc:224] Handling call
0709 12:13:22.080372 (+1571418us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b54000cdd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:22.080524 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39291264}
I20260709 12:13:22.084118 11180 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:22.085825 14121 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.087366 13954 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:22.078498 11076 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 3, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:22.089365 11076 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 5, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 5 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:22.094509 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:22.058127 14183 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:13:22.096463 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:22.097360 11322 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:22.101161 11929 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:22.103861 11076 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 4, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 4 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:22.107503 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:22.138003 11148 connection.cc:256] RPC call timeout handler was delayed by -1.0519111779999548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 12:13:22.128595 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.108541 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:22.139762 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:22.118670 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.143345 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.143913 11175 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:22.145119 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:22.146049 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.144932 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.152019 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:22.154284 11331 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:13:22.154687 11331 pending_rounds.cc:85] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:13:22.154901 11331 pending_rounds.cc:107] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:13:22.155118 11331 raft_consensus.cc:2889] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:22.155275 11323 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:22.155618 11323 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:22.156291 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.145016 11076 catalog_manager.cc:5697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:22.069463 14083 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.139778 11180 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:22.100011 14120 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.179220 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.180059 11179 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:22.176445 11931 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:13:22.183868 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:22.185125 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:22.186944 11251 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:22.156386 11323 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.200747 11323 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:22.147965 14041 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:22.113134 11926 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:22.146363 14018 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:22.170869 13908 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:13:22.192364 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 87491.3us
I20260709 12:13:22.226498 11323 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:22.198714 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:22.228209 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:22.191980 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:22.202324 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:22.103507 14053 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.229501 13908 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:22.234295 13908 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:22.235411 14053 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.210095 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:22.235607 14032 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.211820 11256 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
W20260709 12:13:22.214993 14176 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.657s	user 0.000s	sys 0.002s
I20260709 12:13:22.175953 14014 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:22.219431 11076 catalog_manager.cc:5697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 3, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:22.220427 14013 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.231974 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:22.236285 11325 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 4.
I20260709 12:13:22.237244 14139 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:22.257243 14014 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.191638 14019 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:22.202804 14184 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:13:22.239300 14018 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.241247 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:22.260042 11076 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 5, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 5 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:22.261215 11324 raft_consensus.cc:2410] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:13:22.247575 11327 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:22.252709 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:22.261868 11076 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 5, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 5 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:22.250027 14159 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:13:22.267470 13807 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.270046 13807 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:22.254478 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:22.281534 11331 raft_consensus.cc:2410] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:13:22.256266 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.284125 11175 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:22.277143 14032 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:22.253464 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:22.289409 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:22.290428 14159 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:22.291090 14159 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:22.291441 14159 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:22.291806 14159 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:22.293344 11248 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:22.293814 14140 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:22.294317 14140 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:22.295166 14140 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.296330 14140 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:22.297742 14146 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:22.298187 14146 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:22.298554 14146 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:22.300172 14175 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.742s	user 0.000s	sys 0.001s
I20260709 12:13:22.189757 14120 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.303732 14120 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:22.304497 11326 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:22.305066 11256 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:13:22.305456 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 392) took 1097 ms. Trace:
I20260709 12:13:22.306005 11174 rpcz_store.cc:276] 0709 12:13:21.207937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.208464 (+   527us) service_pool.cc:224] Handling call
0709 12:13:21.223363 (+ 14899us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:21.269138 (+ 45775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:21.269178 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:21.278541 (+  9363us) log.cc:844] Serialized 26 byte log entry
0709 12:13:21.445794 (+167253us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:21.466210 (+ 20416us) log.cc:844] Serialized 14 byte log entry
0709 12:13:21.473090 (+  6880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.473180 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:22.299426 (+826246us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:22.305078 (+  5652us) raft_consensus.cc:1685] finished
0709 12:13:22.305214 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:22.305379 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44559,"prepare.queue_time_us":20547,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":3252,"raft.queue_time_us":49350,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":4053,"spinlock_wait_cycles":97253632,"thread_start_us":139225,"threads_started":1,"wal-append.queue_time_us":246390}
I20260709 12:13:22.269935 13740 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.293900 14139 consensus_queue.cc:1243] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:13:22.283778 13690 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.308458 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:22.267601 14018 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.310068 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:22.312717 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:22.318598 11174 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:22.357604 11256 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:13:22.317348 14139 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:22.365448 14110 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 2.034s	user 0.000s	sys 0.014s
W20260709 12:13:22.355247 13809 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.010s	sys 0.019s
I20260709 12:13:22.380673 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:22.381424 11174 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:13:22.378954 13809 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.011s	sys 0.020s
I20260709 12:13:22.395520 11175 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:22.382334 13908 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.418987 13740 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.420864 11173 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:22.429725 13908 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.434762 13690 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.444726 11934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:13:22.445497 11934 raft_consensus.cc:606] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:13:22.445865 11934 raft_consensus.cc:614] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:22.446326 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 332) took 2799 ms. Trace:
I20260709 12:13:22.446702 11253 rpcz_store.cc:276] 0709 12:13:19.647085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.684174 (+ 37089us) service_pool.cc:224] Handling call
0709 12:13:19.768589 (+ 84415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.778848 (+ 10259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.778879 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:19.790449 (+ 11570us) log.cc:844] Serialized 26 byte log entry
0709 12:13:19.987890 (+197441us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b18000dcb40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:20.230679 (+242789us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:20.441121 (+210442us) log.cc:844] Serialized 14 byte log entry
0709 12:13:20.494134 (+ 53013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.494226 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:22.364596 (+1870370us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:22.446112 (+ 81516us) raft_consensus.cc:1685] finished
0709 12:13:22.446131 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:22.446255 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9801,"prepare.queue_time_us":144258,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4641,"raft.queue_time_us":36681,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":15794,"spinlock_wait_cycles":64421504,"thread_start_us":472331,"threads_started":3,"wal-append.queue_time_us":498771}
I20260709 12:13:22.449931 14094 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:22.451529 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:22.454190 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:22.455461 14162 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:13:22.389446 14127 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:22.468804 11256 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:22.459261 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:22.476823 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:22.481478 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.479260 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:22.525588 11176 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:22.460166 13972 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.528017 13972 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:22.530499 13972 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:22.531884 11934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:13:22.532579 11934 raft_consensus.cc:606] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:13:22.532923 11934 raft_consensus.cc:614] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:22.536460 11934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:13:22.536980 11934 raft_consensus.cc:606] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:22.541801 11174 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:22.543146 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:22.566668 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.717135 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1746) took 1104 ms. Trace:
W20260709 12:13:22.718616 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.005s	sys 0.009s
W20260709 12:13:22.719355 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.005s	sys 0.009s
I20260709 12:13:22.717602 11251 rpcz_store.cc:276] 0709 12:13:21.599469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.666475 (+ 67006us) service_pool.cc:224] Handling call
0709 12:13:22.703986 (+1037511us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540010b154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:22.704172 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861103360}
I20260709 12:13:22.725327 11251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:13:22.726114 11251 raft_consensus.cc:606] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:22.726605 11251 raft_consensus.cc:614] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:22.746224 11251 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:13:22.747115 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1757) took 1075 ms. Trace:
I20260709 12:13:22.747583 11251 rpcz_store.cc:276] 0709 12:13:21.671630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.745555 (+1073925us) service_pool.cc:224] Handling call
0709 12:13:22.745974 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.746827 (+   853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.747053 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:22.748639 14186 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.920s	user 0.006s	sys 0.000s
I20260709 12:13:22.749159 11251 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:13:22.750174 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1759) took 1078 ms. Trace:
I20260709 12:13:22.754705 11251 rpcz_store.cc:276] 0709 12:13:21.672191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.748481 (+1076290us) service_pool.cc:224] Handling call
0709 12:13:22.748851 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.749802 (+   951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.750100 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:22.785669 11932 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.035s	sys 0.058s
W20260709 12:13:22.786885 11932 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.035s	sys 0.058s
W20260709 12:13:22.787259 11178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.006s	sys 0.025s
I20260709 12:13:22.790459 11253 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:13:22.807070 11178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.006s	sys 0.025s
W20260709 12:13:22.799568 14128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.003s	sys 0.011s
I20260709 12:13:22.789345 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1562) took 1158 ms. Trace:
I20260709 12:13:22.810920 11257 rpcz_store.cc:276] 0709 12:13:21.630844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.306355 (+675511us) service_pool.cc:224] Handling call
0709 12:13:22.306621 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:22.306808 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:22.316646 (+  9838us) log.cc:844] Serialized 14 byte log entry
0709 12:13:22.788454 (+471808us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b180045e6e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:13:22.788682 (+   228us) log.cc:844] Serialized 14 byte log entry
0709 12:13:22.788888 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:22.788971 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:22.789036 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.789137 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:22.789284 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":423256,"prepare.queue_time_us":493690,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2734,"raft.queue_time_us":5096,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":20206,"spinlock_wait_cycles":71441664}
I20260709 12:13:22.814230 11257 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:22.814463 14070 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:22.815016 14070 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:22.815138 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1764) took 1053 ms. Trace:
I20260709 12:13:22.815635 11257 rpcz_store.cc:276] 0709 12:13:21.770330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.813061 (+1042731us) service_pool.cc:224] Handling call
0709 12:13:22.813431 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:22.814842 (+  1411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.815073 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:22.817696 11257 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:13:22.817173 14070 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.813267 11173 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:22.818535 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1563) took 1046 ms. Trace:
I20260709 12:13:22.820475 14070 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:23.096784 11257 rpcz_store.cc:276] 0709 12:13:21.771921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.816889 (+1044968us) service_pool.cc:224] Handling call
0709 12:13:22.817251 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:22.818229 (+   978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.818469 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:22.880209 11934 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:22.942548 13974 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.156s
W20260709 12:13:23.044121 11182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.005s	sys 0.012s
W20260709 12:13:23.044313 11900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.004s	sys 0.028s
I20260709 12:13:22.912202 14120 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:23.099162 11900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.004s	sys 0.029s
W20260709 12:13:23.006880 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.941100 11935 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:22.878175 14067 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:23.100744 11182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.006s	sys 0.013s
I20260709 12:13:23.100522 14067 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:22.912339 11249 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:13:22.852006 11180 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.007s	sys 0.031s
W20260709 12:13:23.102974 11180 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.007s	sys 0.047s
W20260709 12:13:23.038573 11927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.004s	sys 0.012s
W20260709 12:13:22.927078 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 110752us
W20260709 12:13:23.103696 11927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.013s
W20260709 12:13:23.094969 11930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.019s	sys 0.013s
W20260709 12:13:23.104492 11930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.019s	sys 0.014s
W20260709 12:13:23.099597 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 675332us
W20260709 12:13:22.851908 11296 connection.cc:256] RPC call timeout handler was delayed by -2.2909979290000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:13:23.105329 11296 connection.cc:256] RPC call timeout handler was delayed by -2.2909979290000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:13:23.105757 11296 connection.cc:256] RPC call timeout handler was delayed by -2.2648979289999716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 12:13:23.067600 11176 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.001s
W20260709 12:13:23.099618 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 947622us
I20260709 12:13:23.106580 11176 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:23.107851 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:22.841555 11181 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:23.094719 11174 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:13:23.089965 11251 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:23.110684 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1761) took 1438 ms. Trace:
I20260709 12:13:23.111052 11251 rpcz_store.cc:276] 0709 12:13:21.672747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.760404 (+1087657us) service_pool.cc:224] Handling call
0709 12:13:23.089617 (+329213us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:23.110450 (+ 20833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.110631 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:23.106117 11296 connection.cc:256] RPC call timeout handler was delayed by -2.2432979289999366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 12:13:23.113497 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.098244 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:23.106576 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:22.983498 14194 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
W20260709 12:13:22.820636 14128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.005s	sys 0.012s
W20260709 12:13:23.113482 11296 connection.cc:256] RPC call timeout handler was delayed by -2.2432979289999366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 12:13:23.127875 11296 connection.cc:256] RPC call timeout handler was delayed by -2.1901979289998508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260709 12:13:23.129374 11296 connection.cc:256] RPC call timeout handler was delayed by -2.1478979290000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 12:13:23.130110 11296 connection.cc:256] RPC call timeout handler was delayed by -2.1478979290000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 12:13:23.130771 11296 connection.cc:256] RPC call timeout handler was delayed by -2.0461979289998453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 12:13:23.131335 11296 connection.cc:256] RPC call timeout handler was delayed by -1.9849979289999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260709 12:13:23.120291 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.132546 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:23.133605 11249 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:23.134425 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:22.982733 14196 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.285s	user 0.000s	sys 0.000s
I20260709 12:13:23.126070 11935 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:23.122313 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:23.122668 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:23.126960 11934 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
W20260709 12:13:23.118542 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.112418 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:23.120734 14127 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.000s
I20260709 12:13:23.310510 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:23.311168 11328 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:23.311509 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 189270us
W20260709 12:13:23.117105 14141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.009s	sys 0.008s
I20260709 12:13:23.128770 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:23.312615 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:23.306960 11257 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:23.109093 13965 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:23.125855 14085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.565s	user 0.015s	sys 0.040s
W20260709 12:13:23.136505 11296 connection.cc:256] RPC call timeout handler was delayed by -1.9282979289998821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 12:13:23.328030 11296 connection.cc:256] RPC call timeout handler was delayed by -1.7824979289998737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2068s for a response
W20260709 12:13:23.328383 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6528979289998915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
W20260709 12:13:23.328701 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6474979289998828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 12:13:23.328991 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6474979289998828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 12:13:23.329289 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6402979289998711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
W20260709 12:13:23.329597 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3045979290000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2599s for a response
W20260709 12:13:23.330076 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3045979290000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2599s for a response
W20260709 12:13:23.330499 11296 connection.cc:256] RPC call timeout handler was delayed by -1.091297928999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2836s for a response
I20260709 12:13:23.315954 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:23.317317 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1550) took 2122 ms. Trace:
I20260709 12:13:23.093757 14014 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:23.314414 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:23.322340 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:23.121868 14050 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:23.121923 14148 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.342435 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:23.339932 11252 rpcz_store.cc:276] 0709 12:13:21.194877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.528590 (+333713us) service_pool.cc:224] Handling call
0709 12:13:21.535988 (+  7398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:21.543371 (+  7383us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:21.605123 (+ 61752us) log.cc:844] Serialized 14 byte log entry
0709 12:13:21.720175 (+115052us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:21.732377 (+ 12202us) log.cc:844] Serialized 52 byte log entry
0709 12:13:21.732562 (+   185us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:21.735242 (+  2680us) log.cc:844] Serialized 14 byte log entry
0709 12:13:21.746347 (+ 11105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.746463 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:22.747881 (+1001418us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:23.317034 (+569153us) raft_consensus.cc:1685] finished
0709 12:13:23.317065 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:23.317246 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14723,"prepare.queue_time_us":66389,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":2283,"raft.queue_time_us":16100,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":35567,"spinlock_wait_cycles":11776768,"thread_start_us":46706,"threads_started":1,"wal-append.queue_time_us":133170}
I20260709 12:13:23.350323 11329 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
W20260709 12:13:23.316308 14141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.009s	sys 0.009s
I20260709 12:13:23.161989 13974 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.099617 14120 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:23.327814 14085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.767s	user 0.015s	sys 0.040s
I20260709 12:13:23.326485 14128 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.334380 14140 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:23.131778 14055 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.235s
I20260709 12:13:23.347312 14127 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:23.416141 14127 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:23.379113 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:23.399227 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.037s	sys 0.070s
W20260709 12:13:23.421211 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.037s	sys 0.070s
W20260709 12:13:23.421931 11076 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.100s	sys 0.198s
W20260709 12:13:23.422698 11076 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.100s	sys 0.198s
W20260709 12:13:23.399636 11932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.016s	sys 0.045s
W20260709 12:13:23.407720 11933 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.016s	sys 0.046s
I20260709 12:13:23.421123 11249 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:23.423269 11932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.016s	sys 0.045s
W20260709 12:13:23.423739 11933 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.016s	sys 0.047s
I20260709 12:13:23.425045 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 561) took 1360 ms. Trace:
I20260709 12:13:23.354476 14071 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:23.425558 11932 rpcz_store.cc:276] 0709 12:13:22.065093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.098073 (+ 32980us) service_pool.cc:224] Handling call
0709 12:13:22.154032 (+ 55959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:22.154218 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:22.804398 (+650180us) log.cc:844] Serialized 14 byte log entry
0709 12:13:23.424617 (+620219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:23.424686 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:23.424746 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.424834 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:23.424975 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":561964,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":50,"thread_start_us":1243226,"threads_started":2}
W20260709 12:13:23.372045 13809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.008s	sys 0.021s
I20260709 12:13:23.333223 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:23.448894 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:23.451359 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 118186us
I20260709 12:13:23.403072 14143 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:23.473714 11928 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.007s	sys 0.057s
W20260709 12:13:23.480322 11928 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.007s	sys 0.057s
I20260709 12:13:23.480487 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:23.444972 13809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.009s	sys 0.022s
I20260709 12:13:23.426669 14085 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:23.351364 13965 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:23.371857 14148 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:23.433000 14071 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:23.394584 13974 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:23.511970 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.466621 14121 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:23.511102 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.519927 11252 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:23.522197 11322 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:23.431542 14128 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:23.482944 14055 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:13:23.534163 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 399814us
I20260709 12:13:23.427532 14140 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:23.558403 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.559747 11322 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:23.563208 11248 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1751) took 1893 ms (client timeout 1624 ms). Trace:
W20260709 12:13:23.563699 11248 rpcz_store.cc:269] 0709 12:13:21.669846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.750026 (+ 80180us) service_pool.cc:224] Handling call
0709 12:13:23.562965 (+1812939us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54001cf2d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:23.563123 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247060352}
W20260709 12:13:23.566391 14203 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:13:23.508042 13809 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:23.573649 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 468341us
W20260709 12:13:23.585542 11250 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.000s	sys 0.003s
W20260709 12:13:23.586107 11250 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.000s	sys 0.003s
I20260709 12:13:23.492810 14143 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:23.567791 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:23.572875 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 260382us
W20260709 12:13:23.637399 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69698.4us
W20260709 12:13:23.595753 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 457418us
I20260709 12:13:23.594122 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:23.566946 14069 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:23.603754 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.550153 14055 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:23.610230 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:23.625566 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:23.625690 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.564492 14085 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:13:23.632313 11248 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:13:23.678900 11248 pending_rounds.cc:85] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 3
I20260709 12:13:23.679277 11248 pending_rounds.cc:107] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:13:23.679605 11248 raft_consensus.cc:2889] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:23.630425 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 474) took 1520 ms. Trace:
I20260709 12:13:23.681187 11928 rpcz_store.cc:276] 0709 12:13:22.110081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.110645 (+   564us) service_pool.cc:224] Handling call
0709 12:13:22.110996 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.161878 (+ 50882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.161906 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.162344 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 12:13:22.304467 (+142123us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b4800041658. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:22.503045 (+198578us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:23.480803 (+977758us) log.cc:844] Serialized 14 byte log entry
0709 12:13:23.604748 (+123945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.604835 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:23.604884 (+    49us) raft_consensus.cc:1685] finished
0709 12:13:23.626232 (+ 21348us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:23.630360 (+  4128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81,"prepare.queue_time_us":213374,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":27,"raft.queue_time_us":141583,"raft.run_cpu_time_us":3377,"raft.run_wall_time_us":168310,"spinlock_wait_cycles":20590848,"thread_start_us":1296129,"threads_started":3,"wal-append.queue_time_us":604475}
W20260709 12:13:23.587007 11926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 562) took 1518 ms (client timeout 1981 ms). Trace:
W20260709 12:13:23.691589 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:23.692185 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.641965 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.693379 11322 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:23.674088 14120 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:23.674434 14207 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:13:23.676378 14148 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:23.610092 14127 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:23.700402 11322 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:23.630745 14067 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:23.584605 13809 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:23.707069 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:23.708575 11322 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:23.674389 14213 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:13:23.723955 11329 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:13:23.777379 14141 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:23.777992 14141 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:23.778654 14141 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:23.784719 13690 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 12:13:23.775914 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 295627us
W20260709 12:13:23.722368 14070 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.625s	user 0.008s	sys 0.003s
I20260709 12:13:23.708556 14120 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.809144 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.736338 14127 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:23.774660 14199 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:13:23.768414 14217 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:13:23.769558 14146 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.471s	user 0.009s	sys 0.004s
I20260709 12:13:23.735988 14148 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:23.713917 14219 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:23.809896 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:23.626111 14071 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:23.801551 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:23.793915 13690 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:13:23.821382 14120 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:23.876940 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 473) took 1767 ms. Trace:
I20260709 12:13:23.895968 11930 rpcz_store.cc:276] 0709 12:13:22.109784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.528850 (+419066us) service_pool.cc:224] Handling call
0709 12:13:22.529402 (+   552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.529792 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.529818 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.531074 (+  1256us) log.cc:844] Serialized 564 byte log entry
0709 12:13:23.104928 (+573854us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:23.104962 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.105020 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:23.876643 (+771623us) raft_consensus.cc:1685] finished
0709 12:13:23.876675 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:23.876862 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":573553,"threads_started":1,"wal-append.queue_time_us":1340021}
I20260709 12:13:23.810637 13993 raft_consensus.cc:993] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
W20260709 12:13:23.690922 11926 rpcz_store.cc:269] 0709 12:13:22.069745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.069878 (+   133us) service_pool.cc:224] Handling call
0709 12:13:23.571465 (+1501587us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b5400690ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:23.583549 (+ 12084us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56972544}
I20260709 12:13:23.813743 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:23.870522 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69048us
W20260709 12:13:23.872298 14214 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:13:23.872797 14206 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:13:23.809836 14229 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:23.871801 14209 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
W20260709 12:13:23.871416 14202 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.273s	user 0.000s	sys 0.000s
W20260709 12:13:23.870949 14208 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
I20260709 12:13:23.872985 14228 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.916051 11329 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:23.870010 14071 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:23.916597 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:23.875850 14120 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:13:23.870177 14219 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:23.811784 14069 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
W20260709 12:13:23.941860 11146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.000s	sys 0.023s
I20260709 12:13:23.913535 13989 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:23.950014 11180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.014s	sys 0.011s
W20260709 12:13:23.976281 11180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.014s	sys 0.011s
W20260709 12:13:23.912735 11328 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.003s
I20260709 12:13:23.925699 14223 raft_consensus.cc:1081] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
W20260709 12:13:23.964534 11254 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.170s	user 0.010s	sys 0.029s
W20260709 12:13:23.980798 11254 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.187s	user 0.010s	sys 0.029s
W20260709 12:13:23.964468 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 314086us
W20260709 12:13:23.968767 11146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.000s	sys 0.023s
I20260709 12:13:23.868721 14146 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:23.866881 14127 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:23.932287 13974 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:23.958614 14049 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.932126 14140 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:23.933563 14067 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.930191 14148 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:23.917760 13965 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:23.977252 11328 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:23.975481 14120 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:23.965416 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.864884 14055 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:23.964442 14219 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:23.974073 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:23.979223 11249 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.545s	user 0.012s	sys 0.000s
W20260709 12:13:24.018561 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.019493 11251 raft_consensus.cc:2368] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9.
I20260709 12:13:24.029873 14121 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.939740 14226 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:13:23.983563 14050 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:13:24.011493 11326 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.024s	sys 0.016s
I20260709 12:13:24.036444 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:24.038069 11326 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.024s	sys 0.016s
I20260709 12:13:24.040246 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:24.042382 11323 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:24.044207 11331 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:24.028725 11249 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:24.036815 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.054072 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.056478 11323 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:13:24.058027 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.061196 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 470) took 1952 ms. Trace:
I20260709 12:13:24.061843 11329 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:24.064091 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:23.943813 14229 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:24.027141 14208 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:13:24.068794 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1754) took 2398 ms. Trace:
I20260709 12:13:24.065686 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.013321 13974 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.067175 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1553) took 2799 ms. Trace:
I20260709 12:13:23.984889 13989 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.069227 11253 rpcz_store.cc:276] 0709 12:13:21.670782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.449263 (+778481us) service_pool.cc:224] Handling call
0709 12:13:22.790289 (+341026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:22.807132 (+ 16843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.068581 (+1261449us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400276850. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:24.068730 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783294208}
I20260709 12:13:24.033535 14128 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:24.070763 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:24.078383 11324 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:24.077991 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.078394 11182 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:13:24.079581 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:24.067503 11329 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:24.070902 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.080248 11329 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:24.080542 11181 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:24.071480 11328 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:24.081658 11328 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:24.071817 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:24.023947 14140 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:24.082504 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1970) took 1906 ms. Trace:
I20260709 12:13:24.072516 11327 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:24.070704 11255 rpcz_store.cc:276] 0709 12:13:21.267787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.065822 (+798035us) service_pool.cc:224] Handling call
0709 12:13:22.066189 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:22.161504 (+ 95315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.161535 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:22.162184 (+   649us) log.cc:844] Serialized 52 byte log entry
0709 12:13:22.685579 (+523395us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:22.685666 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.819469 (+133803us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b540010c054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:22.819538 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.060473 (+1240935us) raft_consensus.cc:1685] finished
0709 12:13:24.060493 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.067116 (+  6623us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90705,"raft.queue_time_us":99991,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":4900,"spinlock_wait_cycles":7161088,"thread_start_us":486349,"threads_started":1,"wal-append.queue_time_us":1348375}
I20260709 12:13:24.061714 11929 rpcz_store.cc:276] 0709 12:13:22.108788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.110497 (+  1709us) service_pool.cc:224] Handling call
0709 12:13:22.110896 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.150220 (+ 39324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.150271 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.170086 (+ 19815us) log.cc:844] Serialized 26 byte log entry
0709 12:13:22.641936 (+471850us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:23.820105 (+1178169us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b18001db660. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:13:24.035232 (+215127us) log.cc:844] Serialized 14 byte log entry
0709 12:13:24.060811 (+ 25579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.060891 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.060928 (+    37us) raft_consensus.cc:1685] finished
0709 12:13:24.061008 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.061137 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75029,"prepare.queue_time_us":74946,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":26294,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":18524,"spinlock_wait_cycles":15846784,"thread_start_us":678335,"threads_started":2,"wal-append.queue_time_us":1528608}
I20260709 12:13:24.073915 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:24.026439 14120 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:23.950443 14228 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.084386 11179 rpcz_store.cc:276] 0709 12:13:22.176223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.188366 (+ 12143us) service_pool.cc:224] Handling call
0709 12:13:22.221069 (+ 32703us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.221389 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.221416 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.221891 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:13:22.702558 (+480667us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:23.087706 (+385148us) log.cc:844] Serialized 14 byte log entry
0709 12:13:23.095570 (+  7864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.095814 (+   244us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.082300 (+986486us) raft_consensus.cc:1685] finished
0709 12:13:24.082321 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.082448 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5548,"prepare.queue_time_us":400133,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3220,"raft.queue_time_us":10415,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":17856,"thread_start_us":480225,"threads_started":1,"wal-append.queue_time_us":1284500}
W20260709 12:13:24.073449 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 479367us
I20260709 12:13:24.030445 14219 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.085546 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:24.097335 14205 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:13:24.086694 14120 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.107689 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:24.108773 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:24.098071 14140 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.111034 14228 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.111893 14140 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:24.098738 14201 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 12:13:24.113871 14204 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:13:24.022631 14067 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.115857 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 471) took 2006 ms. Trace:
I20260709 12:13:24.116828 14067 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.118750 14148 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.122058 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:24.124068 11324 raft_consensus.cc:2410] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:13:24.084218 14241 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:13:24.030654 14210 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:13:24.062794 14121 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.112571 14223 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:24.104925 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:24.064957 14143 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.102255 11329 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:24.086495 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 432885us
I20260709 12:13:24.105962 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:24.026633 14049 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:24.127640 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 446766us
I20260709 12:13:24.082481 11328 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.136731 11324 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:24.146819 11328 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.146991 11329 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:24.148193 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:24.116338 11933 rpcz_store.cc:276] 0709 12:13:22.109119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.109755 (+   636us) service_pool.cc:224] Handling call
0709 12:13:22.162605 (+ 52850us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.162964 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.162995 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.163425 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:13:22.556077 (+392652us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:23.424303 (+868226us) log.cc:844] Serialized 14 byte log entry
0709 12:13:23.427583 (+  3280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.427686 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.115594 (+687908us) raft_consensus.cc:1685] finished
0709 12:13:24.115622 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.115786 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95740,"prepare.queue_time_us":160511,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":159234,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":13114,"thread_start_us":1084030,"threads_started":2,"wal-append.queue_time_us":1536756}
I20260709 12:13:24.151865 11327 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:13:24.153276 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1968) took 1991 ms. Trace:
I20260709 12:13:24.153191 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.153795 11178 rpcz_store.cc:276] 0709 12:13:22.162054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.164325 (+  2271us) service_pool.cc:224] Handling call
0709 12:13:22.184491 (+ 20166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.184932 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:22.224454 (+ 39522us) log.cc:844] Serialized 14 byte log entry
0709 12:13:22.807659 (+583205us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.808884 (+  1225us) log.cc:844] Serialized 564 byte log entry
0709 12:13:22.809065 (+   181us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:22.809112 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.809197 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.153050 (+1343853us) raft_consensus.cc:1685] finished
0709 12:13:24.153068 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.153212 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39507,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":44,"raft.queue_time_us":116,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":561593,"threads_started":1,"wal-append.queue_time_us":1331482}
I20260709 12:13:24.139003 14223 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:24.157680 14223 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:24.158608 11174 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:13:24.154136 11327 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:24.077986 13974 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.160231 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.167202 11327 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:13:24.168350 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.169245 11327 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:24.170215 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.161026 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:24.131245 14159 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.839s	user 0.014s	sys 0.001s
W20260709 12:13:24.164315 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 215787us
I20260709 12:13:24.171582 11324 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:24.172070 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.172648 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.130604 14061 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:24.181944 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:24.184334 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:24.184782 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.185235 11929 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:24.179510 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:24.146111 14231 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:24.194267 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.191802 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1570) took 1337 ms. Trace:
I20260709 12:13:24.152855 14232 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.206409 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.181078 14069 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:13:24.207733 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.205466 11249 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:24.195242 11252 rpcz_store.cc:276] 0709 12:13:22.854839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.522590 (+667751us) service_pool.cc:224] Handling call
0709 12:13:23.563538 (+ 40948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:23.563805 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:23.591194 (+ 27389us) log.cc:844] Serialized 14 byte log entry
0709 12:13:24.191370 (+600176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:24.191421 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:24.191467 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.191533 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.191744 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4066,"prepare.queue_time_us":17748,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"thread_start_us":581766,"threads_started":2,"wal-append.queue_time_us":496299}
I20260709 12:13:24.208825 11328 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:13:24.209553 11328 pending_rounds.cc:85] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 3
I20260709 12:13:24.209857 11328 pending_rounds.cc:107] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:13:24.210160 11328 raft_consensus.cc:2889] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:24.213708 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.162056 14236 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:24.214560 11327 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:24.185928 11324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:59670
I20260709 12:13:24.242452 11324 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:24.243065 11324 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:24.204311 14094 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.750s	user 0.015s	sys 0.000s
I20260709 12:13:24.202395 14225 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.190378 13954 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:13:24.191912 14014 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:13:24.211127 14210 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
W20260709 12:13:24.251513 14237 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:13:24.181787 14159 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.258168 14138 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:13:24.307499 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 101190us
I20260709 12:13:24.260293 14241 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:24.212822 13983 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.217562 14224 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:24.314258 11249 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:24.311684 11324 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.308833 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.315423 11328 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:24.313740 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 129544us
I20260709 12:13:24.319236 11257 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:24.319401 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.296713 14070 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:24.296445 14146 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.205034 14239 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 12:13:24.331674 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:24.316538 14139 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.937s	user 0.014s	sys 0.002s
I20260709 12:13:24.302125 14127 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.336597 11324 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.338639 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:24.211671 14232 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.306877 14138 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:24.281255 14250 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:13:24.353605 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.318358 14069 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:24.279178 14236 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.315302 14224 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.362272 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:24.364827 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.334796 13993 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:24.375520 14231 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:24.335038 14210 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.334666 14249 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:24.356846 14139 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:24.357863 14070 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:24.383386 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 190090us
I20260709 12:13:24.359181 14049 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.371491 14229 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.395568 11256 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:13:24.393615 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:24.378958 14224 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.387651 13993 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:24.386189 13989 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.311146 14225 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.386608 14253 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:24.418355 14253 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:24.399166 14242 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
I20260709 12:13:24.400983 14121 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.419585 14253 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.431481 14269 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:24.432443 14050 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:24.405375 14249 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:24.404119 14049 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:24.432933 14050 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:24.390120 14210 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.406653 13974 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.435994 14121 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:24.449043 14049 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.457947 14121 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:24.427640 14261 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:24.458966 14261 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:24.459227 14121 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.460623 14121 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:24.454015 14253 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.460237 14261 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.441450 14219 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.434922 14069 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.460633 14210 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.467345 14049 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.468688 14236 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.456101 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1969) took 2284 ms. Trace:
I20260709 12:13:24.448853 14223 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.469683 11180 rpcz_store.cc:276] 0709 12:13:22.171620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.173544 (+  1924us) service_pool.cc:224] Handling call
0709 12:13:22.173963 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:23.103385 (+929422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:23.103416 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:23.118452 (+ 15036us) log.cc:844] Serialized 26 byte log entry
0709 12:13:23.976687 (+858235us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:23.976742 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.976814 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.455807 (+478993us) raft_consensus.cc:1685] finished
0709 12:13:24.455835 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.456020 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44225,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":1582083,"threads_started":2,"wal-append.queue_time_us":921167}
I20260709 12:13:24.469198 14219 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.473228 14050 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.438014 14231 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.471406 14223 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:24.479355 14071 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.000s
I20260709 12:13:24.479830 14050 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:24.483824 14141 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.705s	user 0.010s	sys 0.003s
I20260709 12:13:24.484306 14141 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.474768 14232 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:24.486259 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 476) took 2259 ms. Trace:
I20260709 12:13:24.493212 11931 rpcz_store.cc:276] 0709 12:13:22.227366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.227521 (+   155us) service_pool.cc:224] Handling call
0709 12:13:23.575252 (+1347731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:23.575486 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:23.575771 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 12:13:23.769163 (+193392us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:23.769384 (+   221us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:23.769451 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.769538 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.118403 (+348865us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b4800041658. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:24.486057 (+367654us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400690ad0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:24.486197 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20564,"prepare.queue_time_us":416,"prepare.run_cpu_time_us":532,"prepare.run_wall_time_us":1764,"raft.queue_time_us":119641,"raft.run_cpu_time_us":2800,"raft.run_wall_time_us":240913,"spinlock_wait_cycles":3152222976,"thread_start_us":167888,"threads_started":1,"wal-append.queue_time_us":294944}
W20260709 12:13:24.508096 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 145979us
I20260709 12:13:24.492892 14241 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:24.516433 14215 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.934s	user 0.000s	sys 0.009s
I20260709 12:13:24.517220 14141 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.518800 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 472) took 2409 ms. Trace:
W20260709 12:13:24.512746 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.519222 11927 rpcz_store.cc:276] 0709 12:13:22.109449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.383757 (+274308us) service_pool.cc:224] Handling call
0709 12:13:22.384261 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:22.388362 (+  4101us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:22.388408 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:22.389182 (+   774us) log.cc:844] Serialized 52 byte log entry
0709 12:13:23.104152 (+714970us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:23.104188 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.104267 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.515754 (+1411487us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:24.518206 (+  2452us) raft_consensus.cc:1685] finished
0709 12:13:24.518228 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.518725 (+   497us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3507,"raft.queue_time_us":4675,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":14864,"spinlock_wait_cycles":14966144,"thread_start_us":601341,"threads_started":1,"wal-append.queue_time_us":1097824}
I20260709 12:13:24.523639 11180 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:24.526253 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:24.527473 11900 connection.cc:256] RPC call timeout handler was delayed by -1.3141091800000595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
I20260709 12:13:24.529011 11178 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:24.530447 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.477475 14225 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.533569 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:24.535492 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.538630 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.528188 14271 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:13:24.537410 14241 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.534456 14248 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.521836 14224 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.619163 14245 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.622377 14138 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.617882 14246 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:24.611449 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.624910 14141 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 12:13:24.734570 14246 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:24.736613 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.611258 14159 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.610991 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:24.563612 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1571) took 1462 ms (client timeout 1761 ms). Trace:
W20260709 12:13:24.742452 11251 rpcz_store.cc:269] 0709 12:13:23.100987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.315614 (+214627us) service_pool.cc:224] Handling call
0709 12:13:24.563335 (+1247721us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54001cfcd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:24.563528 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563176832}
I20260709 12:13:24.614764 13965 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.744482 13965 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.614377 14228 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:24.746050 14228 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.614601 14140 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.750160 14140 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.616133 11322 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.613288 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1552) took 3410 ms. Trace:
I20260709 12:13:24.611788 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:24.596343 14055 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:24.692453 14239 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:24.774580 11254 rpcz_store.cc:276] 0709 12:13:21.202764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.038476 (+835712us) service_pool.cc:224] Handling call
0709 12:13:22.038827 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:22.041294 (+  2467us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:22.137124 (+ 95830us) log.cc:844] Serialized 14 byte log entry
0709 12:13:23.982888 (+1845764us) log.cc:844] Serialized 14 byte log entry
0709 12:13:23.990798 (+  7910us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:24.013409 (+ 22611us) log.cc:844] Serialized 52 byte log entry
0709 12:13:24.013617 (+   208us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:24.017856 (+  4239us) log.cc:844] Serialized 14 byte log entry
0709 12:13:24.612565 (+594709us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b1800425aa0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:13:24.612823 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.613007 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.613063 (+    56us) raft_consensus.cc:1685] finished
0709 12:13:24.613078 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.613217 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":249084,"prepare.queue_time_us":604848,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":123,"raft.queue_time_us":1257039,"raft.run_cpu_time_us":449,"raft.run_wall_time_us":43357,"spinlock_wait_cycles":5410176,"thread_start_us":1603728,"threads_started":2,"wal-append.queue_time_us":1521158}
I20260709 12:13:24.562371 11330 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:13:24.776795 11330 pending_rounds.cc:85] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 3
I20260709 12:13:24.777047 11330 pending_rounds.cc:107] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:13:24.777338 11330 raft_consensus.cc:2889] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:24.592109 14253 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.618884 13963 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.619475 14251 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.705231 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:24.776073 14239 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:24.779875 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74765.6us
W20260709 12:13:24.684993 14255 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.297s	user 0.003s	sys 0.000s
I20260709 12:13:24.783195 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:24.613978 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.734815 11326 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:24.612661 11323 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:24.779448 14253 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.735308 14138 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:24.785619 14159 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.786201 14159 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.786542 14253 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.787879 13963 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:24.541862 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 153090us
I20260709 12:13:24.617553 14143 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:24.624751 11257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1576) took 1505 ms (client timeout 1527 ms). Trace:
I20260709 12:13:24.792578 14239 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.794092 14239 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:24.794442 14232 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:13:24.794401 11257 rpcz_store.cc:269] 0709 12:13:23.119249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.322045 (+202796us) service_pool.cc:224] Handling call
0709 12:13:24.624482 (+1302437us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b54002a62d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:24.624663 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2246253952}
I20260709 12:13:24.782231 14141 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:24.798969 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1557) took 3448 ms. Trace:
I20260709 12:13:24.799350 11250 rpcz_store.cc:276] 0709 12:13:21.350298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.079605 (+729307us) service_pool.cc:224] Handling call
0709 12:13:22.079939 (+   334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:22.179189 (+ 99250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.179230 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:22.190323 (+ 11093us) log.cc:844] Serialized 52 byte log entry
0709 12:13:22.637950 (+447627us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:24.798182 (+2160232us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b18002778a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf
0709 12:13:24.798334 (+   152us) log.cc:844] Serialized 14 byte log entry
0709 12:13:24.798560 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.798639 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.798692 (+    53us) raft_consensus.cc:1685] finished
0709 12:13:24.798775 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.798913 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":270441,"prepare.queue_time_us":697315,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":6976,"raft.queue_time_us":103260,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2095,"spinlock_wait_cycles":33893760,"thread_start_us":965498,"threads_started":2,"wal-append.queue_time_us":1383014}
I20260709 12:13:24.729192 14245 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:24.776013 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 89979.1us
I20260709 12:13:24.618661 14244 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:24.619108 14225 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:13:24.784096 14251 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.545265 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1755) took 2874 ms. Trace:
I20260709 12:13:24.809927 11256 rpcz_store.cc:276] 0709 12:13:21.671073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.497770 (+826697us) service_pool.cc:224] Handling call
0709 12:13:22.498150 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:24.396273 (+1898123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.544973 (+148700us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54001cedd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:24.545203 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4362856832}
I20260709 12:13:24.793015 14271 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 12:13:24.764200 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 439587us
I20260709 12:13:24.813081 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.749459 14228 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.771075 14070 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:13:24.770617 14261 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:13:24.544339 14120 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.826064 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.806082 14140 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.836726 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:24.844817 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:24.846160 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:24.832321 14014 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:24.808974 14143 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.849341 11329 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:24.851552 14159 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.890858 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.814998 14198 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:13:24.853506 14222 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:24.859073 14245 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.860107 14071 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:24.894546 14071 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:24.894918 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.814473 14244 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.895512 14071 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.896867 14071 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:24.897662 14244 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.897740 14222 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:24.861116 14141 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:24.839303 13809 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.834920 14085 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:13:24.910691 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.885792 14235 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:24.839764 14232 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:13:24.840620 14273 raft_consensus.cc:2764] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:13:24.863108 14269 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.865404 14227 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:24.865592 14138 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.866668 11323 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:24.867522 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:24.930286 11323 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:24.868340 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.868309 14148 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.869294 11327 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:24.938100 11327 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:24.871661 13965 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.873101 14246 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.874728 14120 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:24.878918 14224 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:24.848086 14069 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:24.880268 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:24.950788 11178 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:13:24.880962 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.881096 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.951934 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.881788 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:24.882028 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.865293 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.814913 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:24.777522 14248 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.887818 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:24.956434 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 68695.5us
I20260709 12:13:24.957593 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:24.893500 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:24.959218 11175 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:24.899080 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.907181 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.961117 11927 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:24.901388 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:24.956007 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 141160us
W20260709 12:13:24.962523 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 61210.8us
I20260709 12:13:24.964778 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:24.911374 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.911445 11177 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:24.899873 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45550
I20260709 12:13:24.919093 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.890972 14277 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:24.868287 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.925568 14235 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.932870 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.934504 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:24.939136 11327 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.941563 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:24.940929 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73494.5us
I20260709 12:13:24.943243 11323 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.943796 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:24.947135 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.948709 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:24.949584 14251 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:24.946405 14287 raft_consensus.cc:2955] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:24.952829 11934 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:24.954085 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 88873us
I20260709 12:13:24.843773 14250 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:24.960485 11931 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:24.966988 11933 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:24.968281 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:24.970276 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:24.970322 11328 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:24.970472 11930 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:24.970922 11325 raft_consensus.cc:2410] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:24.971717 11932 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:24.971581 11327 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.944905 14244 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.964856 14239 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:24.990756 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:24.978479 11323 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.969214 13983 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:24.969553 14198 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:24.972993 11928 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:24.989500 11178 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:24.940977 14138 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.973731 14159 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:24.993738 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:25.134290 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 140641us
I20260709 12:13:24.980762 14274 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:13:24.945402 14148 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.972923 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:25.209112 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 236260us
W20260709 12:13:25.209921 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.991979 13983 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:25.032258 14235 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.025918 11325 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:25.056118 14050 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:24.973502 14290 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:25.023195 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:25.025012 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54806.6us
I20260709 12:13:24.994767 14251 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:24.981675 14228 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:13:25.088915 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:25.091707 14284 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:13:25.022084 14069 raft_consensus.cc:2764] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:13:25.216718 14050 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:25.222488 14251 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.227943 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:25.228662 14251 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:25.228246 14050 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:25.229007 11179 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:24.990497 14250 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:25.232828 14250 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:24.938042 14269 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.054348 11926 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:24.979002 14049 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.057s
W20260709 12:13:25.217031 14293 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.116s	user 0.000s	sys 0.002s
I20260709 12:13:24.898795 14222 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:24.982218 14174 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.070s
W20260709 12:13:25.035919 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55483.1us
I20260709 12:13:25.243191 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:25.025334 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:25.244596 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 219508us
I20260709 12:13:25.033551 11322 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:25.245527 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:25.117218 14143 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.126695 14248 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:25.127336 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.251720 11176 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:25.252179 11178 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:25.216830 14290 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:25.113256 14061 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:25.129237 14263 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:25.141908 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:24.941023 14227 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.153290 11323 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:13:25.258435 11323 pending_rounds.cc:85] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 3
I20260709 12:13:25.141665 14159 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:25.261199 14159 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:25.263474 14227 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:25.264012 14239 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
W20260709 12:13:25.264894 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 123078us
I20260709 12:13:25.143682 14146 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:25.265816 14146 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.266209 14146 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:25.267732 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.267818 14140 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:25.268323 14140 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.268935 14140 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:25.254504 14069 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:25.269688 14069 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:25.152356 14271 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.270351 14271 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.270671 14069 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.272657 14271 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.256750 14085 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:13:25.261338 14245 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:13:25.212947 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:25.220849 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.223146 11147 leader_election.cc:400] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9.
I20260709 12:13:25.278369 11147 leader_election.cc:403] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:25.278861 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60089.4us
I20260709 12:13:25.279320 11176 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
W20260709 12:13:25.053789 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.215168 13983 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:25.214159 13965 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:25.240036 14138 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.214308 14148 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.118831 11327 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:13:25.247675 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:25.225508 14228 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.257236 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.258689 11323 pending_rounds.cc:107] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:13:25.293519 11323 raft_consensus.cc:2889] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:25.294589 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:25.297974 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50393.7us
I20260709 12:13:25.299196 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:25.256371 14263 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:25.194919 14055 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.277818 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.219457 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.283138 14120 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:25.288199 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 75316.7us
I20260709 12:13:25.319455 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:25.253934 14241 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:25.308405 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.310158 11322 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:25.310288 11257 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:13:25.310838 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.313292 11249 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:25.323276 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:25.252213 14094 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:13:25.254771 14290 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:25.317694 11928 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:25.328118 11934 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:25.320808 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.324081 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:25.283131 14143 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:25.297415 14228 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:25.295907 14253 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:25.295502 14227 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:25.324667 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.290515 14049 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9.
I20260709 12:13:25.329463 11252 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:25.330806 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2002) took 1631 ms. Trace:
I20260709 12:13:25.338840 11322 rpcz_store.cc:276] 0709 12:13:23.699681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.699839 (+   158us) service_pool.cc:224] Handling call
0709 12:13:23.700236 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:24.615620 (+915384us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4c001e6f00. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec49b8a20 00007f3ec49c9a29 00007f3ec498f644 00007f3ec49b6f2d 00007f3ec49b6b93 00007f3ec66e718b 00007f3ec66e7cb5 00007f3ec66e2f30 00007f3ec66e47de 00007f3ec66eeb49 00007f3ec66ef416
0709 12:13:25.032958 (+417338us) spinlock_profiling.cc:243] Waited 6.86 ms on lock 0x7b5400291420. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6661a1d 00007f3ec66481a4 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66eeb49 00007f3ec66ef416 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419
0709 12:13:25.330533 (+297575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:25.330753 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51681408}
I20260709 12:13:25.331636 11934 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:25.299273 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:25.314718 14277 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:25.304039 14138 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:25.258687 14061 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:25.343732 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.344591 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.377099 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:25.346341 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.320262 14222 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:25.332500 14273 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.348234 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:25.351351 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.328440 13965 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:25.361799 14263 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:25.346786 14295 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:13:25.340809 14253 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:25.325906 14120 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:25.342208 14227 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:25.348145 14143 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.373920 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.375355 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1378) took 3281 ms. Trace:
I20260709 12:13:25.416028 11330 rpcz_store.cc:276] 0709 12:13:22.093857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.093978 (+   121us) service_pool.cc:224] Handling call
0709 12:13:22.137051 (+ 43073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:24.777848 (+2640797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:25.375024 (+597176us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b5400459cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:25.375270 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5576599808}
W20260709 12:13:25.382503 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.364686 14159 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:25.410316 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 62228us
I20260709 12:13:25.322520 14055 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:25.428580 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.429931 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:25.442170 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:25.493641 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 173305us
I20260709 12:13:25.494513 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:25.494822 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:25.393755 14291 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:13:25.542568 11929 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:25.500609 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.554937 11176 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:25.522624 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.565109 11178 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:25.432061 11928 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:25.397863 13965 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.584188 11178 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:25.580999 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.586020 11251 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:25.416496 14227 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:25.603853 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 304722us
I20260709 12:13:25.606736 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:25.496249 14138 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.501884 14263 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:25.502656 14277 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:25.617449 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.541751 11926 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:25.601336 11176 raft_consensus.cc:2955] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:25.502275 14235 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:13:25.620525 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.622004 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:25.630439 11933 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:25.580217 13972 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:25.412211 14273 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:25.632731 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.634712 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.626830 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.635620 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.631580 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.582439 14250 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:25.592201 13965 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:25.623636 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 324434us
I20260709 12:13:25.654717 11934 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:25.655505 11927 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:25.609756 14271 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.665443 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:25.666144 11927 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:25.666360 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:25.643446 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:25.661927 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.667835 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:25.659256 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:25.608734 14071 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:25.672688 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:25.673543 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:25.674350 11927 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:25.655468 11928 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:25.636512 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.676146 11931 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:25.677827 11249 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:13:25.683197 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.686230 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.687727 11253 raft_consensus.cc:2410] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:13:25.688356 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.628647 14235 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.692327 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.695343 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.622876 14269 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:25.632619 14069 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:25.616616 14049 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:13:25.626626 14274 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:13:25.621552 14241 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 12:13:25.626775 14138 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:25.701917 14257 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:25.720422 11935 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:25.700863 14235 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:25.671115 14248 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 12:13:25.715735 14260 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:13:25.633607 14253 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:25.738842 14253 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:25.739475 14299 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:13:25.734722 14055 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.716689 14302 raft_consensus.cc:2764] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:13:25.737530 14014 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:25.676486 14250 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:25.694912 14071 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.690038 14271 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.729501 14269 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:13:25.753031 14303 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 12:13:25.755151 14298 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:13:25.771543 14269 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:25.761200 14277 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:25.797309 14301 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:13:25.841571 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 417) took 1030 ms. Trace:
W20260709 12:13:25.846436 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.848229 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.750586 14014 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:25.862258 14069 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:25.863649 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:25.864948 11935 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:25.778127 14061 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.907943 14055 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:25.864356 14014 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.865926 14143 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:25.946249 14277 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.948662 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:25.950516 14287 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:25.783579 14159 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:25.951504 14287 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:25.950755 14300 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:25.893517 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:25.971940 11251 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:25.782514 14146 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.002s
I20260709 12:13:25.976414 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:25.784591 14269 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:25.842677 11174 rpcz_store.cc:276] 0709 12:13:24.811280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.811844 (+   564us) service_pool.cc:224] Handling call
0709 12:13:24.812209 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:24.812663 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:24.812697 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:24.813240 (+   543us) log.cc:844] Serialized 52 byte log entry
0709 12:13:24.940185 (+126945us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:24.940228 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.940303 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:25.840749 (+900446us) raft_consensus.cc:1685] finished
0709 12:13:25.840840 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:25.841277 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1981,"raft.queue_time_us":1675,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":17905,"thread_start_us":96129,"threads_started":1,"wal-append.queue_time_us":320425}
I20260709 12:13:25.981854 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1418) took 1112 ms. Trace:
I20260709 12:13:25.954774 14061 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:13:25.983199 14138 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:25.969157 14300 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:25.989055 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.992955 13963 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.007905 14226 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:25.838865 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.017041 11331 rpcz_store.cc:276] 0709 12:13:24.868861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.894703 (+ 25842us) service_pool.cc:224] Handling call
0709 12:13:25.981189 (+1086486us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54000c67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:25.981541 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492587904}
I20260709 12:13:25.865092 14069 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.044221 11251 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:26.027616 14300 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:26.027390 14226 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:26.062036 11296 connection.cc:256] RPC call timeout handler was delayed by -2.6513681270000689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 12:13:26.062171 14146 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:26.063784 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.073879 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:26.056248 11180 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:25.970851 14287 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.079658 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:26.082290 14232 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:26.083314 14232 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:26.086010 14070 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.087113 14070 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:26.087669 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 521880us
I20260709 12:13:26.098599 11251 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:26.098747 14317 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:26.099172 14071 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.100200 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:26.100587 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.099795 14317 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:26.101398 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 428592us
I20260709 12:13:26.104063 14317 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.085105 14232 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.107051 14250 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:26.110229 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.110719 14070 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.101552 11251 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:13:26.119136 14251 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.890s	user 0.019s	sys 0.017s
I20260709 12:13:26.119807 14251 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:26.133452 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.129400 14120 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:26.141746 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.143782 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:26.145911 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.146277 14226 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.147873 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.148399 14226 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:26.148947 14244 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.157s	user 0.020s	sys 0.032s
I20260709 12:13:26.149995 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:26.150758 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61216.9us
I20260709 12:13:26.151955 11180 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:26.153751 14318 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:26.154120 11177 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:26.154807 14318 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.155817 11177 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:26.156675 14273 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.157815 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.159682 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:26.154803 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.163018 11296 connection.cc:256] RPC call timeout handler was delayed by -2.6135681270000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 12:13:26.061283 14271 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.151971 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:26.171216 11296 connection.cc:256] RPC call timeout handler was delayed by -2.5037681270000576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 12:13:26.157392 14318 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.181387 11180 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:13:26.178560 11296 connection.cc:256] RPC call timeout handler was delayed by -2.4452681270001904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
I20260709 12:13:26.149504 14244 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.184374 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:26.186405 11296 connection.cc:256] RPC call timeout handler was delayed by -2.4416681270001845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1815s for a response
I20260709 12:13:26.188963 14094 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:26.189584 14094 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:26.190140 14094 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:26.198944 11296 connection.cc:256] RPC call timeout handler was delayed by -2.43266812700017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:13:26.203156 14263 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.014s
I20260709 12:13:26.203469 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:26.203938 14222 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.015s
W20260709 12:13:26.186088 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.561s (SENT) [suppressed 11 similar messages]
W20260709 12:13:26.215701 11296 connection.cc:256] RPC call timeout handler was delayed by -2.43266812700017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:13:26.220757 11296 connection.cc:256] RPC call timeout handler was delayed by -2.3507681270000376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 12:13:26.221307 11296 connection.cc:256] RPC call timeout handler was delayed by -2.3435681270000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
I20260709 12:13:26.203506 14271 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:26.229063 11296 connection.cc:256] RPC call timeout handler was delayed by -2.2958681270001762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
I20260709 12:13:26.230829 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:26.232090 11296 connection.cc:256] RPC call timeout handler was delayed by -2.1599681270001838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 12:13:26.232502 11296 connection.cc:256] RPC call timeout handler was delayed by -2.0762681270000485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
I20260709 12:13:26.236865 11177 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:13:26.240489 11178 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:26.243763 11935 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.224498 14143 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.247537 11180 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:26.239140 11248 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:26.249176 14271 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:26.251201 14227 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.004s
I20260709 12:13:26.251662 14227 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.252460 14271 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.253157 11255 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:26.252916 14300 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:26.256268 13965 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.559s	user 0.016s	sys 0.000s
I20260709 12:13:26.266487 11928 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:26.266366 13965 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.267612 11249 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:26.267727 14143 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:26.268180 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.268448 14143 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:26.256877 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105123us
W20260709 12:13:26.258100 11179 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.024s	user 0.012s	sys 0.018s
I20260709 12:13:26.274865 11179 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.275604 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1986) took 1370 ms. Trace:
I20260709 12:13:26.276010 11179 rpcz_store.cc:276] 0709 12:13:24.904806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.227536 (+322730us) service_pool.cc:224] Handling call
0709 12:13:26.275500 (+1047964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:26.238911 11173 raft_consensus.cc:606] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Received request to transfer leadership
I20260709 12:13:26.276942 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:26.277304 11173 raft_consensus.cc:614] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:26.258266 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:26.258664 14146 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:26.259315 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.238914 14320 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
W20260709 12:13:26.262753 13983 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.947s	user 0.009s	sys 0.011s
W20260709 12:13:26.263187 14140 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.993s	user 0.016s	sys 0.006s
I20260709 12:13:26.278620 11175 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.285858 14148 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.285965 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:26.286875 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 1989) took 1373 ms. Trace:
I20260709 12:13:26.286512 14198 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:26.286587 14148 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.287551 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.279989 11933 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:26.288419 11182 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.284368 13983 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.263664 11252 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:26.283828 11254 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:13:26.264380 11253 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:26.258683 14251 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.269268 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.277576 11179 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:26.289451 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.285460 14049 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:13:26.284710 14140 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.294060 11256 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:26.295171 14269 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:26.296190 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.297335 11177 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:13:26.284564 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.298306 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.298539 11252 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:26.299088 11177 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:26.299328 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:26.299762 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.05291e+06us
I20260709 12:13:26.300585 14248 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:26.300817 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:26.301173 14248 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.302521 14222 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:26.302657 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.302992 14148 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.303299 14222 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.303879 14148 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:26.304399 14148 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.304896 14148 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:26.305224 14248 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:26.307210 14140 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.287317 11175 rpcz_store.cc:276] 0709 12:13:24.913826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.277514 (+363688us) service_pool.cc:224] Handling call
0709 12:13:26.286690 (+1009176us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b54002e3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:26.286821 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2211206528}
I20260709 12:13:26.308475 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.309998 11175 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:26.310878 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56583.6us
I20260709 12:13:26.311316 13983 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:25.904695 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:26.313758 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.313886 14070 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:26.317255 14070 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:26.317909 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.300161 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.324945 11177 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:26.311230 11175 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:26.342556 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:26.253537 14290 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.903s	user 0.012s	sys 0.006s
W20260709 12:13:26.352587 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 83457us
I20260709 12:13:26.362565 11179 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 3. Current term is 5. Ops: []
I20260709 12:13:26.362565 11175 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:26.364418 11182 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:26.368079 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.362635 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 570) took 1043 ms. Trace:
I20260709 12:13:26.324345 14070 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:26.364133 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:26.377344 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.370961 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:26.378167 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 196541us
W20260709 12:13:26.374495 11176 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.590s	user 0.024s	sys 0.002s
I20260709 12:13:26.383534 11331 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:26.371052 11930 rpcz_store.cc:276] 0709 12:13:25.319515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.319645 (+   130us) service_pool.cc:224] Handling call
0709 12:13:25.319959 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:25.320300 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:25.320325 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:25.320892 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:13:26.169579 (+848687us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b18000ad800. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:26.345981 (+176402us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:26.352068 (+  6087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.361956 (+  9888us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:26.362032 (+    76us) raft_consensus.cc:1685] finished
0709 12:13:26.362064 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:26.362416 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4884,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7579,"spinlock_wait_cycles":3076352,"thread_start_us":54899,"threads_started":1,"wal-append.queue_time_us":969251}
W20260709 12:13:26.392244 14253 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.005s
I20260709 12:13:26.392800 14260 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:26.393996 11329 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:26.395401 14251 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:26.396130 14251 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.396731 14251 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:26.407829 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:26.408769 11931 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:26.410863 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:26.414728 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 114018us
I20260709 12:13:26.374819 14263 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.416443 14263 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.419328 14263 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.421330 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 418) took 1609 ms. Trace:
I20260709 12:13:26.422855 14049 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:26.423799 14140 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:26.424335 14049 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:13:26.424651 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 147902us
I20260709 12:13:26.424767 14049 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:26.386036 14287 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.426365 14287 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.376755 14290 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.407308 11928 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:26.407243 14070 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.423900 14143 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.430830 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:26.432642 13989 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:26.433760 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45550 (request call id 81) took 1551 ms. Trace:
I20260709 12:13:26.436185 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:26.443383 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:26.386616 14198 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:26.429376 14287 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.447418 11173 rpcz_store.cc:276] 0709 12:13:24.885686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.885814 (+   128us) service_pool.cc:224] Handling call
0709 12:13:26.433418 (+1547604us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002e3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e6853 00007f3ecb6b460a 00007f3ec4c3a7d4 00007f3ec4c3a762 00007f3ec4c3a691 00007f3ec4c3a60c 00007f3ec4c398c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:26.433600 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2875181312}
I20260709 12:13:26.421888 11181 rpcz_store.cc:276] 0709 12:13:24.811672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.811806 (+   134us) service_pool.cc:224] Handling call
0709 12:13:24.833409 (+ 21603us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:25.031113 (+197704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:25.035373 (+  4260us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:25.906014 (+870641us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b2c00013008. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:13:25.944438 (+ 38424us) log.cc:844] Serialized 52 byte log entry
0709 12:13:26.183899 (+239461us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:26.319783 (+135884us) log.cc:844] Serialized 14 byte log entry
0709 12:13:26.343068 (+ 23285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.420756 (+ 77688us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54002a0154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:26.420820 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:26.420879 (+    59us) raft_consensus.cc:1685] finished
0709 12:13:26.420896 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:26.421119 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43902,"prepare.queue_time_us":199960,"raft.queue_time_us":90803,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":36137,"spinlock_wait_cycles":383596288,"thread_start_us":218404,"threads_started":2,"wal-append.queue_time_us":220108}
I20260709 12:13:26.446192 14232 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.454329 14198 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.467235 11933 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:26.468863 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.453430 13965 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.452582 14271 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.476541 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:26.479030 14271 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.481348 14271 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.486837 14070 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:26.486948 14141 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:13:26.489367 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:26.491132 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55051us
I20260709 12:13:26.492141 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.492379 14148 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.493824 14246 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:26.496726 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71199.8us
I20260709 12:13:26.493880 14061 raft_consensus.cc:2764] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:13:26.497732 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:26.498572 13989 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:26.501327 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.502503 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1577) took 1195 ms. Trace:
W20260709 12:13:26.506050 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.501893 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:26.506270 11256 rpcz_store.cc:276] 0709 12:13:25.307125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.307681 (+   556us) service_pool.cc:224] Handling call
0709 12:13:25.307987 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.294473 (+986486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.502167 (+207694us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:26.502419 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2266618624}
I20260709 12:13:26.473078 14198 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:26.560478 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.470120 14263 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:26.505148 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:26.503314 14055 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.594s	user 0.016s	sys 0.011s
W20260709 12:13:26.598312 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 90074.7us
W20260709 12:13:26.599107 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.505206 14235 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.754s	user 0.008s	sys 0.011s
I20260709 12:13:26.485411 14226 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:13:26.604386 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 112352us
I20260709 12:13:26.612226 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:26.614499 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:26.503583 14228 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.131s	user 0.004s	sys 0.013s
W20260709 12:13:26.620728 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 203204us
I20260709 12:13:26.615486 11329 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:26.619048 11326 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:26.501403 14246 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.624459 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 1998) took 1344 ms. Trace:
I20260709 12:13:26.625370 14140 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:26.626402 14246 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.627429 14277 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:13:26.628113 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 145149us
I20260709 12:13:26.630218 14307 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.630550 14146 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:26.631057 14146 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:26.630923 14307 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.632215 14146 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.634178 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.634436 14257 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:26.634820 14307 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.634872 14257 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:26.624938 11176 rpcz_store.cc:276] 0709 12:13:25.279955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.600445 (+320490us) service_pool.cc:224] Handling call
0709 12:13:25.600894 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:25.601175 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:26.387617 (+786442us) log.cc:844] Serialized 14 byte log entry
0709 12:13:26.591430 (+203813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:26.604307 (+ 12877us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:26.610918 (+  6611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.620333 (+  9415us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:26.624376 (+  4043us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94426,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":177,"thread_start_us":115628,"threads_started":1}
W20260709 12:13:26.654017 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.655438 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.647449 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 169321us
I20260709 12:13:26.647239 11931 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:26.463688 14049 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:26.659610 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45598.3us
I20260709 12:13:26.638780 14253 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.605087 14227 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.613480 14055 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:26.661093 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48959.3us
I20260709 12:13:26.508214 14232 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.638680 14290 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.665058 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:26.672197 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.672791 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.624509 13965 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:26.622131 14235 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.666857 14140 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.694679 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.675652 14146 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:26.687732 14269 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:13:26.731388 14269 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:26.734264 14269 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:26.736023 14141 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:26.736542 14141 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.736886 14141 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:26.738550 14322 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.739102 14148 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.739212 14322 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.739876 14148 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:26.740317 14148 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:26.742115 14322 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.743278 14235 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.659005 14094 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.748189 14094 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:26.753957 14227 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:26.754534 14227 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.754909 14227 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:26.758042 13965 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:26.759297 13965 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.705250 13809 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:13:26.659032 14228 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.766098 11250 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:26.706295 13974 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:26.766786 13974 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:26.767187 13974 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:26.685959 14253 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.771878 14253 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:26.708588 14336 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:13:26.711126 14232 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:26.712723 14049 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:26.714617 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:26.715765 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 214039us
I20260709 12:13:26.716086 14271 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:26.718713 14251 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.720887 14061 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:26.726711 14014 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:26.676581 14300 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:26.728996 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49825.4us
I20260709 12:13:26.741295 14148 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.797009 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:26.798420 14061 consensus_queue.cc:1243] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:26.774963 14228 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.805235 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:26.816857 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.818465 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 154437us
W20260709 12:13:26.819653 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105213us
I20260709 12:13:26.807447 14322 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.821110 11251 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:26.822877 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.811611 14348 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.833210 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1769) took 1172 ms. Trace:
I20260709 12:13:26.834131 11251 rpcz_store.cc:276] 0709 12:13:25.660297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.157349 (+497052us) service_pool.cc:224] Handling call
0709 12:13:26.820907 (+663558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.821584 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.832957 (+ 11373us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400068450. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:26.833140 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520816512}
I20260709 12:13:26.784096 14248 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.835492 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:26.829798 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1578) took 1517 ms (client timeout 1607 ms). Trace:
W20260709 12:13:26.836549 11250 rpcz_store.cc:269] 0709 12:13:25.307536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.308170 (+   634us) service_pool.cc:224] Handling call
0709 12:13:26.816400 (+1508230us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400068454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:26.824430 (+  8030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3320548096}
I20260709 12:13:26.846098 11331 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:26.850898 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:26.855274 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:26.838693 14300 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:26.858079 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.833343 14070 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:26.844188 14253 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:13:26.872669 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.877106 11324 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:26.883929 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.884644 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.865478 14300 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.892992 14070 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:26.893572 14300 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:26.932304 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46984.1us
I20260709 12:13:26.910696 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.909602 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.934844 11330 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:13:26.906934 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:26.910211 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.878234 14253 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:26.911312 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.938143 11331 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
W20260709 12:13:26.939745 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 217598us
I20260709 12:13:26.945839 11176 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:26.943405 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.913065 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.944235 11329 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.913354 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.949178 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.939929 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.912472 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.954981 11323 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 12:13:26.958393 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:26.889969 14322 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.957918 14235 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:26.960819 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.975283 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.975370 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:26.958092 11324 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.969821 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:26.958026 11327 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:26.962196 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:26.978562 11250 raft_consensus.cc:2368] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
W20260709 12:13:26.979158 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72405.7us
I20260709 12:13:26.965472 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:26.980167 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 141618us
I20260709 12:13:26.980329 11931 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:26.893182 14246 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:26.978264 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:26.974864 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:26.970316 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.988636 11173 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:26.901491 14198 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:26.963680 14222 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:26.983927 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:26.984177 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:26.981187 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:26.953974 14070 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:26.985283 11325 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:26.982393 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.998238 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:26.997978 14235 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.000726 14235 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.001914 14148 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:27.002152 11251 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:27.002919 11176 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:27.010252 11329 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:27.011860 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:27.014917 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.020087 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:27.020195 11173 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
W20260709 12:13:27.022085 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 357080us
I20260709 12:13:27.023708 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:27.012650 14345 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:13:27.026456 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.029517 11901 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:27.014438 14055 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:27.034796 11176 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:13:27.036652 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:27.038157 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.053325 11325 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:27.053151 11256 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:27.055382 11173 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:27.008956 14300 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.002175 14253 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.063814 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:27.059983 14355 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:13:27.094722 11176 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.096118 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.096992 11328 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:27.097227 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.098387 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.098670 14146 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.098912 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.099543 11323 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:27.067411 14050 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:13:27.100263 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.070269 11329 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:27.066901 11926 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:27.076823 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:27.066108 14349 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:13:27.036966 14348 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:13:27.078989 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.081557 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:27.082116 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.082439 11248 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:13:27.082582 11252 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:27.083071 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.019748 14246 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.083667 14257 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.084571 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.112843 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.085438 11933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:13:27.116585 11933 raft_consensus.cc:606] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:27.116640 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 96666.4us
I20260709 12:13:27.117013 11933 raft_consensus.cc:614] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:27.088820 11174 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:13:27.089248 14269 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.090636 14198 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.091809 14094 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:27.092119 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.120251 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.092060 14251 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:27.122123 14061 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.058225 14055 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:27.131631 14061 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.132244 14055 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.130786 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.098507 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.135183 11326 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.099254 14146 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.136931 14013 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:27.106019 14300 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:27.138083 14146 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.122392 14235 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:13:27.138530 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:27.074064 14248 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:27.078470 13989 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:27.139242 11926 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.104817 11250 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:27.139755 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.140601 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.141043 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:27.141806 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.142601 11250 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:13:27.142447 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.143378 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:27.143858 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.144691 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.139106 14013 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:27.129364 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.146282 14013 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:27.053114 14322 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:27.113866 11249 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:27.148486 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.113376 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.107367 11257 raft_consensus.cc:2376] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8. Re-sending same reply.
I20260709 12:13:27.115981 11251 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:27.149606 14352 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.150334 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:27.034890 14227 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.152863 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:27.153409 14227 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.154295 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:27.155197 14307 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:27.117185 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:27.129243 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.130198 14269 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.130775 14246 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:27.101819 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.139063 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:27.146687 11173 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:27.154474 14251 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:27.157168 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:27.158051 11248 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:27.123318 11256 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:27.145452 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.105381 11176 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:27.163545 11257 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:27.170243 11253 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:27.174487 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:27.176290 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.164880 14228 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:27.178582 14246 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:27.179004 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.179953 11181 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:27.180559 14253 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:27.179073 14246 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.181784 14246 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:27.181995 14061 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.183398 14358 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:27.183900 14358 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.184274 14358 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:27.185866 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95208.7us
I20260709 12:13:27.188710 11326 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:27.191470 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.192389 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.192957 11173 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:27.151593 14013 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.194788 11250 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:27.196767 11180 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:27.171451 14222 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.203115 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:27.206115 11181 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.178764 14228 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:27.173540 14140 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:27.207929 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:27.218899 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:27.221792 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.223244 11324 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:27.223819 11324 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:27.193641 14307 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:27.211392 14359 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:13:27.263078 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:27.263957 11178 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:27.265856 11256 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:27.274114 11179 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
W20260709 12:13:27.267407 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.214241 14228 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:27.277078 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74344.8us
I20260709 12:13:27.150359 14352 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.214723 14140 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:27.275439 11252 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:27.277119 11324 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.263077 14307 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.295291 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:27.278996 13974 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.511s	user 0.019s	sys 0.000s
I20260709 12:13:27.268603 14227 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:27.321733 11926 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:27.292737 14235 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:27.307978 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.309505 11179 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:27.324532 14362 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.325117 14362 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.310096 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.326316 11322 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.326359 14061 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:27.326946 14061 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.310309 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:27.310876 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.329921 11329 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:27.311069 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:27.330181 14061 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:27.311611 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.330899 11325 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:27.318056 11176 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:27.314376 14141 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.002s
I20260709 12:13:27.318405 11326 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
W20260709 12:13:27.311326 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 124971us
I20260709 12:13:27.330916 11173 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:27.331795 14141 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:27.325523 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.333892 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.318809 14269 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.334852 14141 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.335562 14290 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:27.336786 11255 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:27.334400 11181 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:27.337884 14322 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:13:27.352738 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.353434 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:27.214565 14222 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.352797 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.357925 11256 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:27.359692 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:27.290967 14013 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:27.297154 14307 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:27.323307 14014 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.000s
I20260709 12:13:27.374995 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:27.375253 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.320947 14227 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.378877 11324 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:27.393080 11253 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:27.394137 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.349962 14257 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:27.351065 13974 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.369887 14235 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:27.397192 14227 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:27.429157 14362 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.430195 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:27.431841 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 78542.9us
W20260709 12:13:27.433738 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 299160us
I20260709 12:13:27.436815 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:27.431567 11249 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:27.438592 11324 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:13:27.438997 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:27.440001 13989 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:27.456768 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 182149us
I20260709 12:13:27.472126 11324 pending_rounds.cc:85] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 5
I20260709 12:13:27.477347 11182 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:27.449000 11329 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:27.380879 14055 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:27.480393 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.396294 14300 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:27.479424 11329 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:27.497488 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 434776us
I20260709 12:13:27.411819 14222 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:27.507158 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:27.474844 11324 pending_rounds.cc:107] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:13:27.511942 11324 raft_consensus.cc:2889] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:27.508035 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:27.517469 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:27.520247 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 180789us
W20260709 12:13:27.521281 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 83579.3us
I20260709 12:13:27.523151 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:27.550137 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.572491 11326 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:27.397939 13965 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 12:13:27.496960 14055 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:27.575400 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55821.8us
W20260709 12:13:27.581314 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 142407us
I20260709 12:13:27.430935 14235 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:27.595407 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78050.2us
I20260709 12:13:27.675076 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.676059 11324 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:27.597820 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:27.681149 11928 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:27.682003 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.682514 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.584647 14362 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.704658 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:27.704409 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:27.586520 11329 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.576974 14014 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:27.768613 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:27.778285 11935 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:27.775463 11932 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:27.573814 14222 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.184s
W20260709 12:13:27.712791 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 129998us
I20260709 12:13:27.518677 14271 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:27.589010 13989 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:27.527168 14071 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:27.678033 14070 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:27.588275 14055 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:27.546258 14322 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:27.575737 14300 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:27.782547 11324 pending_rounds.cc:85] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:13:27.839251 11324 pending_rounds.cc:107] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:27.839648 11324 raft_consensus.cc:2889] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:27.693019 14198 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.710263 14050 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:27.759202 14232 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:27.805687 11180 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:27.862460 11180 raft_consensus.cc:606] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:27.862957 11180 raft_consensus.cc:614] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:27.785400 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:27.572309 14140 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.780462 14348 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:27.859563 11329 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:27.803774 14271 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.834419 14071 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.840519 14070 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.810745 13974 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:27.807139 13989 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:27.871572 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.876048 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:27.842319 14322 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:27.884562 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:27.804147 14362 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:27.874323 14246 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.692s	user 0.009s	sys 0.004s
I20260709 12:13:27.902791 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:27.894733 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:27.705149 14379 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.176s
I20260709 12:13:27.925608 11930 raft_consensus.cc:2368] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
W20260709 12:13:27.894997 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 373863us
I20260709 12:13:27.877782 14271 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:27.874111 14251 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 12:13:27.891405 14049 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.110s	user 0.012s	sys 0.002s
W20260709 12:13:27.919668 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.810000 13972 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.456s
W20260709 12:13:27.918824 14365 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.437s	user 0.001s	sys 0.000s
I20260709 12:13:27.914870 11326 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:27.933203 11932 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.930859 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.890820 14070 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.871294 14232 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.938187 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:27.888469 14380 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.860044 14198 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:27.957340 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72884.9us
I20260709 12:13:27.886770 14071 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:27.958343 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.967330 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:27.911513 13989 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:27.867002 14370 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:13:27.936002 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.977591 11253 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:27.979038 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.960763 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.917292 14290 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:27.964838 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.894840 14348 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.994658 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:27.962287 14226 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:27.996162 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 308428us
I20260709 12:13:27.981837 11078 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 5 to 8, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 8 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:27.984140 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:27.989038 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 254036us
I20260709 12:13:27.993211 14290 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:27.993816 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:27.998756 11250 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:27.987067 11252 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:27.982470 14141 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:27.929345 14246 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:27.888792 14050 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.003196 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.007688 11325 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:13:28.007197 14141 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.011885 14368 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:13:28.012264 14375 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 12:13:28.013077 14377 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 12:13:28.014396 14383 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:13:28.016863 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.007416 14322 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.008524 14381 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:13:28.024888 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.024026 11325 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:28.035231 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.033869 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.037021 11325 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:28.039409 14061 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:28.040024 14061 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.040470 14061 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:28.009765 14290 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.015159 14348 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.066264 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.067159 11326 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:28.003537 14226 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.083707 11928 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:13:28.085117 11900 leader_election.cc:336] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.753s (SENT)
I20260709 12:13:27.974967 14380 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.090365 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.977594 14300 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.074050 14369 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
W20260709 12:13:28.094760 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.072280 14322 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:28.127579 11900 leader_election.cc:336] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.814s (SENT)
I20260709 12:13:28.158720 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.159917 11325 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:28.165268 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:28.166491 11325 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:28.174830 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:28.179366 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 146591us
W20260709 12:13:28.186434 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.194144 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:28.196108 14376 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:13:28.197882 14405 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.198660 14405 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.202031 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 82354.4us
W20260709 12:13:28.202924 11900 leader_election.cc:336] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.988s (SENT)
I20260709 12:13:28.202725 14070 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.203382 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.194136 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:28.204463 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 220440us
I20260709 12:13:28.210402 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.218035 14198 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.219652 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:28.207219 14371 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.532s	user 0.003s	sys 0.002s
I20260709 12:13:28.229642 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.230418 11253 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:28.215822 14393 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:13:28.206557 14362 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.245985 11934 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:28.248301 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.253718 11251 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:28.212692 14403 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:28.221911 14222 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:28.267755 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.229909 14232 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:28.279019 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.285873 11250 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:28.265486 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.282555 14397 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.282598 14412 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.221292 14405 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:28.252852 14253 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.072s	user 0.003s	sys 0.011s
I20260709 12:13:28.220635 14406 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:28.245589 14246 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.238698 14400 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:28.288944 14050 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.295799 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.273526 14222 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.275820 13809 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:28.309187 11931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:13:28.309904 11931 raft_consensus.cc:606] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:13:28.310285 11931 raft_consensus.cc:614] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:28.305754 14228 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.012s	user 0.013s	sys 0.000s
I20260709 12:13:28.291466 14232 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:28.295032 14406 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.327035 11932 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:28.306011 14013 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.930s	user 0.013s	sys 0.000s
I20260709 12:13:28.292301 14403 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:28.302801 14362 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:28.335702 11329 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 12:13:28.311398 14222 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:28.373927 14408 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:28.374996 14379 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.300141 14397 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.375909 14379 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:28.376255 14379 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:28.378613 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1601) took 1298 ms. Trace:
I20260709 12:13:28.380318 14094 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:28.380867 14094 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:28.382539 11178 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:28.383431 14055 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 12:13:28.383881 14055 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.384306 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:28.350112 14390 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:13:28.350423 14374 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.386328 14374 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.387854 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 602545us
I20260709 12:13:28.388707 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:28.339195 14228 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.351506 14410 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:13:28.304906 14412 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.353528 14352 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.402432 14061 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.404628 14061 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.404458 14222 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.343410 14406 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:28.408104 11323 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:28.408888 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1421) took 1041 ms. Trace:
I20260709 12:13:28.409914 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.354312 14070 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:28.416466 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:28.417111 11928 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:28.354851 14380 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.358467 11329 pending_rounds.cc:85] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 3
W20260709 12:13:28.361177 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.368388 14271 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.370290 14307 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.941s	user 0.011s	sys 0.002s
W20260709 12:13:28.371158 14227 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.946s	user 0.013s	sys 0.000s
I20260709 12:13:28.321764 14198 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:28.358088 14362 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.379045 11254 rpcz_store.cc:276] 0709 12:13:27.079808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.079931 (+   123us) service_pool.cc:224] Handling call
0709 12:13:27.080307 (+   376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:27.080797 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:27.172018 (+ 91221us) log.cc:844] Serialized 14 byte log entry
0709 12:13:27.397166 (+225148us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:27.399099 (+  1933us) log.cc:844] Serialized 52 byte log entry
0709 12:13:27.399297 (+   198us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:27.399348 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:27.399449 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:28.067239 (+667790us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:28.256535 (+189296us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b4800042b58. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:28.378362 (+121827us) raft_consensus.cc:1685] finished
0709 12:13:28.378387 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:28.378526 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14568,"prepare.queue_time_us":104077,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3527,"raft.queue_time_us":17954,"raft.run_cpu_time_us":8977,"raft.run_wall_time_us":875945,"spinlock_wait_cycles":156028160,"thread_start_us":274558,"threads_started":2,"wal-append.queue_time_us":250267}
W20260709 12:13:28.347620 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51911.1us
I20260709 12:13:28.344993 14232 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.377184 14379 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.388509 13989 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.398787 14094 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.410722 11181 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:28.411067 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:28.412509 11325 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:28.414232 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.414714 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.415773 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.416054 14412 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.416121 14228 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.417486 11933 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:28.409354 11322 rpcz_store.cc:276] 0709 12:13:27.367790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.367953 (+   163us) service_pool.cc:224] Handling call
0709 12:13:27.398401 (+ 30448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:27.838565 (+440164us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:28.084362 (+245797us) log.cc:844] Serialized 14 byte log entry
0709 12:13:28.237214 (+152852us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:28.273253 (+ 36039us) log.cc:844] Serialized 26 byte log entry
0709 12:13:28.273411 (+   158us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:28.273446 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:28.273516 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:28.408628 (+135112us) raft_consensus.cc:1685] finished
0709 12:13:28.408660 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:28.408816 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47426,"prepare.queue_time_us":328843,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":13814,"raft.queue_time_us":399685,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":49439,"thread_start_us":776353,"threads_started":3,"wal-append.queue_time_us":224296}
I20260709 12:13:28.504276 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:28.505740 11177 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:28.506558 11934 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:28.417587 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:28.508014 11931 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 5: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:28.508859 14352 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:28.490935 11329 pending_rounds.cc:107] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:13:28.511549 11329 raft_consensus.cc:2889] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:28.418030 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.484411 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.512444 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.431703 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:28.430284 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.492790 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.463719 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.434069 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.495738 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:28.498096 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.500494 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.514639 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.515354 13972 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:13:28.515558 14382 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.505s	user 0.002s	sys 0.000s
I20260709 12:13:28.515842 13972 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:28.516273 13972 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.516516 14344 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.517478 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.517294 14344 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.517961 14389 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.518533 14389 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.520082 14344 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.521083 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 436) took 1202 ms. Trace:
I20260709 12:13:28.521463 11179 rpcz_store.cc:276] 0709 12:13:27.319046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.323856 (+  4810us) service_pool.cc:224] Handling call
0709 12:13:27.324189 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:27.339602 (+ 15413us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:27.339644 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:27.340974 (+  1330us) log.cc:844] Serialized 26 byte log entry
0709 12:13:27.777399 (+436425us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:27.777441 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:27.777523 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:28.389855 (+612332us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:28.520811 (+130956us) raft_consensus.cc:1685] finished
0709 12:13:28.520831 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:28.521025 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6404,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":123,"spinlock_wait_cycles":16246528,"thread_start_us":435940,"threads_started":1,"wal-append.queue_time_us":576891}
I20260709 12:13:28.523429 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.524088 13965 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.524611 13965 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.524966 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1606) took 1443 ms. Trace:
I20260709 12:13:28.526800 11179 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:28.525444 11257 rpcz_store.cc:276] 0709 12:13:27.081365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.313189 (+231824us) service_pool.cc:224] Handling call
0709 12:13:27.313483 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:27.319575 (+  6092us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:27.319892 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 12:13:28.283106 (+963214us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b1800213460. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:13:28.524571 (+241465us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:28.524628 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:28.524684 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:28.524754 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:28.524897 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144692,"prepare.queue_time_us":30140,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":92802,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":7842176,"thread_start_us":148322,"threads_started":1}
W20260709 12:13:28.443210 14358 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.258s	user 0.012s	sys 0.004s
I20260709 12:13:28.516706 14412 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:28.528394 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:28.498495 14410 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:13:28.388582 14374 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.533746 11181 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:28.534907 11926 raft_consensus.cc:2368] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9.
I20260709 12:13:28.509007 14387 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:28.540800 14344 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:28.432003 14013 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.542068 14232 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:28.542184 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1422) took 1173 ms. Trace:
I20260709 12:13:28.505975 14222 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.430780 14392 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.499366 14362 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:28.498098 14227 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.544799 14271 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.590162 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 376) took 1499 ms. Trace:
I20260709 12:13:28.590796 14399 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:28.586431 14380 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.591336 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.591715 11256 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:28.559500 14246 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:28.591831 14380 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.592337 14246 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:28.592769 14246 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:28.593467 14140 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.551067 11255 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:28.470428 14071 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.565487 14257 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:28.553177 14412 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:28.566931 13989 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.567148 14141 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.550132 14379 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.541321 11173 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:28.574994 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1607) took 1493 ms. Trace:
I20260709 12:13:28.550644 14352 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:28.572616 11329 rpcz_store.cc:276] 0709 12:13:27.368956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.369239 (+   283us) service_pool.cc:224] Handling call
0709 12:13:27.396738 (+ 27499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:28.541906 (+1145168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:28.542109 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:28.605017 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.576979 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:28.605865 11257 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:28.576265 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 373368us
I20260709 12:13:28.605962 11325 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:28.576746 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 165758us
I20260709 12:13:28.588495 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.588589 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:28.589783 14227 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.574542 14397 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.560205 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.607710 11932 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:28.589293 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:28.595214 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:28.587019 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74632.4us
I20260709 12:13:28.608893 11250 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:28.609145 11251 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:28.595275 14400 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:28.590651 11249 rpcz_store.cc:276] 0709 12:13:27.090817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.148222 (+ 57405us) service_pool.cc:224] Handling call
0709 12:13:28.589967 (+1441745us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540010b654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:28.590085 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649564928}
I20260709 12:13:28.502596 14307 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.571033 14344 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.603615 11248 rpcz_store.cc:276] 0709 12:13:27.082388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.314940 (+232552us) service_pool.cc:224] Handling call
0709 12:13:27.315376 (+   436us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:27.315879 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:27.315912 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:13:27.316670 (+   758us) log.cc:844] Serialized 78 byte log entry
0709 12:13:27.715637 (+398967us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:27.810227 (+ 94590us) log.cc:844] Serialized 14 byte log entry
0709 12:13:28.387310 (+577083us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b18002133a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:13:28.510727 (+123417us) log.cc:844] Serialized 14 byte log entry
0709 12:13:28.569858 (+ 59131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:28.569933 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:28.574773 (+  4840us) raft_consensus.cc:1685] finished
0709 12:13:28.574794 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:28.574939 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119566,"prepare.queue_time_us":117645,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":4970,"raft.queue_time_us":745726,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":40286,"spinlock_wait_cycles":8204032,"thread_start_us":521424,"threads_started":2,"wal-append.queue_time_us":669464}
I20260709 12:13:28.594447 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:28.599787 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.600075 11253 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:28.600654 11180 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:28.593907 14392 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.614856 11328 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:28.572074 14094 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.596359 14399 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:28.597918 14257 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:28.610736 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:28.606880 11324 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:28.608887 11331 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:28.561257 14358 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.615470 11927 raft_consensus.cc:2368] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:28.574661 14013 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.622975 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.637807 11248 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:28.620864 14398 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:28.595664 11296 connection.cc:256] RPC call timeout handler was delayed by -2.3853533540000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:13:28.635130 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.509038 14385 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:28.601338 14412 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:28.630376 14386 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.624080 14055 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.640363 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.650553 11935 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:28.611243 13965 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.616074 14344 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.638968 11249 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:28.646270 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.645749 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:28.651311 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3774986159999116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 12:13:28.653048 11325 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:28.576546 14403 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.653484 14386 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.655153 11222 leader_election.cc:400] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:28.659956 11222 leader_election.cc:403] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:28.660084 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.661051 14427 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.661228 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:28.662941 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.664182 11222 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:28.664407 13965 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:28.665225 13965 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:13:28.658200 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2883986159997676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:13:28.665843 13965 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:28.666035 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2883986159997676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:13:28.666493 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.667237 11173 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:28.666872 13965 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.667732 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:28.668414 11932 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
W20260709 12:13:28.666486 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2685986159997356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 12:13:28.671674 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1686986159998014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
W20260709 12:13:28.672055 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0984986159999153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 12:13:28.670073 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:28.672789 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0849986159998934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 12:13:28.673100 11248 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:28.673174 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0849986159998934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 12:13:28.674651 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.675108 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:28.676746 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.683303 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.684145 11182 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:28.642910 14228 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:28.688244 14386 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.678290 14146 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:28.690424 11932 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:28.691852 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.693071 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.648236 14381 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:28.693907 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.697365 14381 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.697821 14055 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:28.698442 14055 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:28.699513 14381 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:28.699385 14055 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:28.701740 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:28.702154 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.702052 14399 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.688304 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.703832 11930 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:28.689356 14146 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.659868 13974 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:28.705905 14257 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.693593 14412 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.706882 14146 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.695703 14392 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.707911 14406 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.697196 14307 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.696215 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.708952 11179 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:28.659338 14061 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:28.709702 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.709991 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.710573 11179 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:28.711030 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.711894 14408 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.712635 14412 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.712379 14408 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.713164 14322 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:28.713965 14322 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:28.714586 14408 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:28.716886 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.695506 14013 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.643092 14385 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:28.645818 14398 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.722291 14322 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:28.722662 14385 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.673715 14352 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.724620 14385 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.723682 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.725700 14322 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:13:28.717873 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72242.4us
I20260709 12:13:28.727360 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:28.644352 14380 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:28.648715 14389 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.719107 11932 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:13:28.706998 14257 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.726745 14228 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:28.737705 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:28.738119 14228 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:28.738606 13989 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:28.739892 13989 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:28.719635 14408 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:28.741693 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:28.742187 14406 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.694217 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.743324 11257 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:28.730201 14061 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.731397 11928 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:28.734120 11250 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:28.718102 11222 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:28.747401 11222 leader_election.cc:403] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:13:28.655120 14071 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:28.745709 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:28.746057 11182 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:28.750814 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.751456 11174 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:28.752215 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.753126 11182 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:28.686874 11251 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:28.747784 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 138712us
I20260709 12:13:28.721362 11179 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
W20260709 12:13:28.748831 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51326.9us
I20260709 12:13:28.757927 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.759003 11251 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:28.761298 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.762259 11250 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:28.788092 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:28.750248 14061 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.723750 14398 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.748688 14386 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:28.797243 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:28.766957 14389 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:28.805245 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.806453 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:28.807061 11174 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:28.808041 11928 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:28.786336 14391 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:13:28.808933 11257 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:28.809814 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:28.788885 13989 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:13:28.786882 13972 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.817407 14071 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.818068 14071 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.821481 13972 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.821943 11254 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:28.826714 14246 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.815544 14403 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:13:28.828977 14362 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.850003 11181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:28.850718 11181 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:28.850833 14362 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.851091 11181 raft_consensus.cc:614] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:28.849156 14246 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.858390 11174 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:28.833827 14380 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.834484 14381 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:28.836233 14427 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:28.869978 14269 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:28.836421 11254 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:13:28.840116 11173 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:28.840567 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.841174 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.841331 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.872203 11931 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:28.844058 14358 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.844245 14422 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.846268 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.847121 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.858521 11256 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [2.2-5.3]
I20260709 12:13:28.832448 14408 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:28.859669 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:28.861340 11127 auto_rebalancer.cc:893] Removing replace marker failed: Replica 016411114c6d42039bd04025378dfc98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:13:28.861792 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.862228 11181 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:13:28.862546 14378 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:28.865994 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.869665 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.873760 14269 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.950891 11256 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 9. Ops: []
I20260709 12:13:28.951923 14226 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.952334 14269 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:28.952996 14094 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:28.874977 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:28.954006 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 79150.7us
I20260709 12:13:28.953330 11078 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 3 to 5, now has a pending config: VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193), VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194), VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195), VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 5 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:28.956700 14094 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:28.958091 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 111040us
I20260709 12:13:28.950167 14422 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:28.959960 14380 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:28.960390 14380 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:28.960738 14380 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:28.880249 11076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 237) took 16384 ms (client timeout 15000 ms). Trace:
W20260709 12:13:28.823925 14420 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:13:28.942391 11929 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:28.975131 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:28.976038 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 106486us
I20260709 12:13:28.976989 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:28.873337 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:28.950011 14381 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:13:28.950649 11127 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:13:28.951980 14408 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:28.875145 11932 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:28.959239 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.876372 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:28.985004 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.863651 14071 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.957468 11078 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 reported cstate change: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:28.981837 11173 raft_consensus.cc:606] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.848142 14228 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:28.985149 11931 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:28.876251 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:28.987967 11173 raft_consensus.cc:614] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:28.966068 14378 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:28.983250 11127 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:13:28.965049 14422 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:28.979426 14094 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:29.003146 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:29.004699 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.983083 14381 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:29.007565 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:28.998000 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:29.100595 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.011044 14422 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:13:29.110761 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 505) took 1255 ms. Trace:
W20260709 12:13:29.147729 14440 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:13:29.085603 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.409307 11935 rpcz_store.cc:276] 0709 12:13:27.855359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.868833 (+ 13474us) service_pool.cc:224] Handling call
0709 12:13:29.110526 (+1241693us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54004c2554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:29.110686 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1708249856}
I20260709 12:13:29.410782 11329 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:29.411604 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 404070us
W20260709 12:13:29.120435 14437 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:13:29.177904 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:29.413228 14362 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:29.413515 11248 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:29.095805 11931 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:29.009990 14398 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:29.414062 14362 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:29.414700 14398 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.415405 14422 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:29.415807 14422 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:29.416785 14398 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:29.416608 14422 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.129990 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.108180 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.410532 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:29.418020 11323 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:29.139817 14251 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:29.419003 14251 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:29.078965 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.419670 11330 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:29.420039 14251 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.047573 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:29.096411 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1619) took 1154 ms. Trace:
I20260709 12:13:29.158811 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:29.422016 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 374506us
I20260709 12:13:29.422362 11251 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:29.423761 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.424978 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:29.077791 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.105547 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.103901 14094 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.103931 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.003938 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:29.373399 14380 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.011384 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:29.139155 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 342022us
I20260709 12:13:29.432797 11330 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:29.432971 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:29.046739 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:29.434007 11930 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:29.130293 14228 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:29.058199 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 508) took 1110 ms. Trace:
W20260709 12:13:29.039386 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 80312.8us
I20260709 12:13:29.101691 11331 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:29.097883 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:29.135797 14378 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.152750 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:29.039491 11078 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:29.181999 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:28.982302 14344 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.283185 11256 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:29.445226 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:29.150146 14381 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:29.139606 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 329919us
I20260709 12:13:29.326442 14061 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.172482 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:29.441802 14352 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:29.449826 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 445505us
I20260709 12:13:29.450345 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:29.451712 11325 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:29.452709 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:29.441939 14427 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:29.280822 14389 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:29.003120 14226 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.074869 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:29.477521 11933 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
W20260709 12:13:29.118377 14436 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:13:29.328830 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.272810 11254 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:29.174528 14399 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.233446 11252 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:13:28.961856 11076 rpcz_store.cc:269] 0709 12:13:12.495733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.495948 (+   215us) service_pool.cc:224] Handling call
0709 12:13:28.879946 (+16383998us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:23.927367 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:23.928086 (+   719us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:23.928130 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:13:23.928150 (+    20us) tablet.cc:662] Decoding operations
0709 12:13:24.061593 (+133443us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:24.063753 (+  2160us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:24.068620 (+  4867us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:13:24.131764 (+ 63144us) tablet.cc:701] Row locks acquired
0709 12:13:24.131789 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:13:24.162249 (+ 30460us) write_op.cc:270] Start()
0709 12:13:24.191770 (+ 29521us) write_op.cc:276] Timestamp: P: 1783599204158647 usec, L: 0
0709 12:13:24.191811 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:13:24.354395 (+162584us) log.cc:844] Serialized 7235 byte log entry
0709 12:13:24.592850 (+238455us) op_driver.cc:464] REPLICATION: finished
0709 12:13:24.787366 (+194516us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e7e28 00007f3eca122581 00007f3eca121d58 00007f3eca1269f1 00007f3eca1269a9 00007f3eca126939 00007f3eca126901 00007f3eca125b2d 0000000000388634 0000000000388479
0709 12:13:24.792755 (+  5389us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b18002ca3a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:24.863438 (+ 70683us) write_op.cc:301] APPLY: starting
0709 12:13:24.863548 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:24.863924 (+   376us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:24.863947 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:24.871226 (+  7279us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:24.871484 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 12:13:24.871529 (+    45us) write_op.cc:312] APPLY: finished
0709 12:13:24.871886 (+   357us) log.cc:844] Serialized 125 byte log entry
0709 12:13:24.872060 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:24.872218 (+   158us) write_op.cc:454] Released schema lock
0709 12:13:24.872271 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:24.872691 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:24.872720 (+    29us) write_op.cc:454] Released schema lock
0709 12:13:25.363693 (+490973us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18002ca3a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff557 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
Metrics: {"mutex_wait_us":2650595,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":250367,"apply.run_cpu_time_us":4131,"apply.run_wall_time_us":23519,"num_ops":18,"prepare.queue_time_us":1551248,"prepare.run_cpu_time_us":32397,"prepare.run_wall_time_us":871262,"raft.queue_time_us":87114,"raft.run_cpu_time_us":22789,"raft.run_wall_time_us":192678,"replication_time_us":389338,"spinlock_wait_cycles":130783744,"thread_start_us":1581340,"threads_started":4,"wal-append.queue_time_us":145715,"wal-append.run_cpu_time_us":10345,"wal-append.run_wall_time_us":1477629}]]}
I20260709 12:13:29.006368 14290 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:29.043042 14198 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:29.057761 14403 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.138957 14055 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:29.417882 11326 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:28.984552 14408 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:29.010059 14014 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:29.419109 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:29.426805 11322 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:29.179900 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:29.426973 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 423101us
I20260709 12:13:28.992580 14269 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:29.421882 11255 rpcz_store.cc:276] 0709 12:13:27.943115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.943443 (+   328us) service_pool.cc:224] Handling call
0709 12:13:27.943841 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:28.594046 (+650205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:29.096119 (+502073us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b5400276854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:29.096328 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1396056832}
I20260709 12:13:29.427745 14398 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:29.427985 14422 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:29.427747 14094 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:29.540680 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.427767 14071 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:29.431468 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:29.435058 11248 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:29.437211 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:29.439059 11251 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:29.443229 11250 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:29.102306 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.437829 11927 rpcz_store.cc:276] 0709 12:13:27.937520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.937782 (+   262us) service_pool.cc:224] Handling call
0709 12:13:29.047729 (+1109947us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54006c5054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:29.047866 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481903872}
I20260709 12:13:29.158903 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:29.447444 11253 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:29.450613 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:29.354007 14385 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.629s	user 0.008s	sys 0.005s
I20260709 12:13:29.460870 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:29.490165 11174 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:29.541606 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 114060us
I20260709 12:13:29.495967 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:29.497968 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 85137.2us
I20260709 12:13:29.543772 11256 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:29.543963 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:29.545306 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:29.516077 11176 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:13:29.546343 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 109212us
W20260709 12:13:29.535888 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 90760.1us
I20260709 12:13:29.548368 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:29.527006 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:29.428843 14372 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:29.493932 14290 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:29.542500 11257 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:29.494860 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:29.551321 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:29.488081 14226 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:29.563637 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68915.9us
I20260709 12:13:29.564625 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:29.565615 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:29.568518 11078 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 2 to 7, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 7 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:29.579105 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:29.550369 14071 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:29.585371 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:29.589032 11322 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:29.589753 11078 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:29.515642 14442 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:29.591542 14448 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.552531 14372 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:29.552665 14352 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:29.583683 13972 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:29.560776 14290 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:29.584131 11175 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:29.556023 14389 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:29.597334 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:29.598377 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54367.5us
I20260709 12:13:29.503611 14399 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.567122 14055 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:29.567322 14198 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:29.559435 14398 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:29.602083 11078 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:13:29.538892 14014 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:29.617966 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 54659.1us
I20260709 12:13:29.598930 13972 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:29.567948 14447 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:29.620550 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 170003us
I20260709 12:13:29.583629 14307 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:29.592458 14385 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:29.596827 14228 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:13:29.671818 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46005.5us
I20260709 12:13:29.592777 14071 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:29.558993 14403 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.597141 14269 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:29.595551 14235 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:29.600248 14372 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:29.607731 14386 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:29.714025 11249 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
W20260709 12:13:29.715545 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 100541us
I20260709 12:13:29.715062 11078 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 3, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 3 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:29.615540 14055 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:29.597385 14443 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:29.677186 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:29.619633 14251 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:29.670816 13972 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:29.678017 14385 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:29.747031 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 117979us
I20260709 12:13:29.675164 14379 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.750773 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2031) took 1079 ms. Trace:
I20260709 12:13:29.598603 14448 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.749644 11328 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:29.751217 11174 rpcz_store.cc:276] 0709 12:13:28.671140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.859402 (+188262us) service_pool.cc:224] Handling call
0709 12:13:29.750541 (+891139us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54000da054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:29.750711 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1256221056}
I20260709 12:13:29.764293 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:29.717175 14307 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:29.722788 14055 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:29.769886 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1804) took 1346 ms. Trace:
I20260709 12:13:29.770470 11175 rpcz_store.cc:276] 0709 12:13:28.408348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.411290 (+  2942us) service_pool.cc:224] Handling call
0709 12:13:29.097750 (+686460us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7eee 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:29.754725 (+656975us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002323d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:29.754841 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334533120}
I20260709 12:13:29.726711 14269 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:29.727329 14441 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:29.778096 11174 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:13:29.614261 14442 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.778170 14443 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.740463 14380 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:29.786713 11078 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 6 to 7, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 7 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:29.740036 14061 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:29.675418 13989 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:13:29.752159 14385 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:29.614408 14408 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.751109 14450 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:29.797427 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:29.785430 14447 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:29.791285 14441 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:29.816360 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 139088us
I20260709 12:13:29.794739 14352 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:29.768006 14379 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.812180 14389 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:29.808699 14380 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:29.814003 14449 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:29.797848 14443 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:29.830544 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 234504us
I20260709 12:13:29.815481 14408 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:29.841818 11078 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 5 to 6, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 6 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:29.843569 11078 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 3 to 6, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 6 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:29.849206 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.849923 11326 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:29.851085 11330 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:13:29.852290 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.825958 14094 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:29.860951 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:29.857694 11078 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:13:29.851020 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 305837us
I20260709 12:13:29.865339 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.867491 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.867658 11326 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:29.868360 11329 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:29.869830 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.870493 11322 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:29.871603 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.872255 11325 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:29.872444 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.873265 11329 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:29.871603 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.823484 14246 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:13:29.877445 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:29.877550 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.839807 14226 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:29.881563 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.901096 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:29.902168 11323 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:29.895046 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:29.892274 11325 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:29.903136 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:29.903741 11328 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:29.845917 14251 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:29.901523 11331 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:29.904654 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:29.884289 11296 leader_election.cc:400] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9.
I20260709 12:13:29.905942 11330 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:29.907238 11296 leader_election.cc:403] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:13:29.901700 14447 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:13:29.908267 14457 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:29.908710 14447 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:29.909088 14447 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:29.909314 14443 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:29.905332 14094 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:29.850447 14307 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.917811 14061 raft_consensus.cc:2764] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:29.918177 14399 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:29.918774 14399 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:29.921959 14455 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:13:29.922801 14352 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.925778 14389 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.818943 14398 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.929771 14198 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:29.930263 14198 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:29.929970 14447 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:29.931479 14447 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:29.930719 14198 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:29.934984 11329 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:29.937131 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 218499us
I20260709 12:13:29.938740 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:29.882012 14014 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:29.946436 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 62271.9us
I20260709 12:13:29.944873 11934 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:29.942210 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 415324us
I20260709 12:13:29.954563 11329 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:13:29.966002 11329 pending_rounds.cc:85] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 4
I20260709 12:13:29.966391 11329 pending_rounds.cc:107] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:13:29.869860 14257 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:13:29.970060 11935 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:13:29.972743 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:29.972827 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:29.978020 11175 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:29.973232 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:29.966853 11329 raft_consensus.cc:2889] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:29.974848 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:29.975546 11253 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:29.978181 11180 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 5. Current term is 6. Ops: [2.2-5.3]
I20260709 12:13:29.974718 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:29.982741 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:29.849685 14269 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:29.983696 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:29.984349 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:29.984539 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 154220us
I20260709 12:13:29.984807 11175 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 12:13:29.935212 14403 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:29.893427 14380 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
W20260709 12:13:29.992925 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.994216 11250 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 12:13:29.994724 11182 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:29.995028 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:29.997750 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:29.993747 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.004016 11322 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:30.004490 11255 raft_consensus.cc:2410] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:13:30.005607 11257 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:30.005863 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.006644 11174 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:29.964263 14449 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:30.012200 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.013736 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:30.009172 11330 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:29.961885 14094 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:30.007175 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49691.5us
I20260709 12:13:29.979014 14386 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:29.954058 14451 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9.
I20260709 12:13:29.940984 14457 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:13:30.018512 11329 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:30.019796 11178 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:30.021842 11251 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:29.979578 14362 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:13:30.011451 14449 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:30.022375 14358 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.054957 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:30.032958 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:30.057606 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:30.082615 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:30.124838 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:30.062314 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 81589.3us
I20260709 12:13:30.019804 14246 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:13:30.055128 14449 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:30.129652 14454 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 12:13:30.128918 14358 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.060729 14467 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:30.197850 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 181032us
I20260709 12:13:30.191074 14464 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:30.191352 14469 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:30.057950 13965 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:30.055913 14235 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:30.192636 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 135084us
I20260709 12:13:30.213505 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:30.195923 14379 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:30.223075 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:30.226230 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 171338us
I20260709 12:13:30.032305 14251 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.236896 14450 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:30.225912 14307 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:30.232115 14358 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.212248 14235 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:30.243834 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:30.248811 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.269825 14358 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:30.227294 14386 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.232115 14071 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.274623 14055 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.266393 14226 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:30.244340 14459 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:13:30.205713 14362 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:13:30.279557 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.365126 11180 raft_consensus.cc:2368] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
I20260709 12:13:30.365944 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.366595 11180 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:30.241503 14469 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:30.245339 14307 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:30.357115 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66213.4us
I20260709 12:13:30.250802 14235 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:30.372361 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.373759 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:30.387339 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 188293us
W20260709 12:13:30.394937 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.375118 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.397110 11180 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:30.408762 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.410106 11180 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:30.411281 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.458209 11180 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
W20260709 12:13:30.363234 14422 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.813s	user 0.012s	sys 0.002s
I20260709 12:13:30.483016 11252 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:30.374684 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.483969 11250 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:30.484333 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:30.485230 11181 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:30.485364 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.486356 11250 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:30.240893 14473 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 12:13:30.215749 14464 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.514859 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.489434 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:30.517187 11180 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:30.246428 14465 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.128s
I20260709 12:13:30.358332 14246 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:30.246680 14398 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 12:13:30.501442 11257 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:30.487497 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.508589 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:30.550644 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:30.518918 11900 leader_election.cc:400] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
I20260709 12:13:30.552299 11900 leader_election.cc:403] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:30.552577 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.549145 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:30.560612 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.563642 11252 raft_consensus.cc:2368] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7.
I20260709 12:13:30.380218 14307 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:30.583153 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64455.1us
I20260709 12:13:30.582269 11181 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
W20260709 12:13:30.585657 11328 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.001s
I20260709 12:13:30.362340 14300 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.193s
I20260709 12:13:30.586376 11328 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:30.595315 11180 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:30.596925 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:30.599443 11180 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:30.603259 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:30.567082 13989 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.611279 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.611956 11251 raft_consensus.cc:2376] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8. Re-sending same reply.
I20260709 12:13:30.608090 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:30.613159 11181 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:30.610203 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.577562 14379 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:30.567670 14469 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:30.623086 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:30.598584 14422 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:13:30.631091 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:30.637126 11181 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:30.638298 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:30.638377 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 249510us
I20260709 12:13:30.580520 14246 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.651913 14379 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:30.638872 14470 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:13:30.638921 14472 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:13:30.670188 14071 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.639112 14460 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.671311 14460 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:13:30.674033 14471 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.435s	user 0.001s	sys 0.000s
I20260709 12:13:30.675642 14469 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.678539 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1635) took 1522 ms. Trace:
I20260709 12:13:30.678763 14398 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:30.679165 14398 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:30.678942 11249 rpcz_store.cc:276] 0709 12:13:29.156687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.159001 (+  2314us) service_pool.cc:224] Handling call
0709 12:13:30.678369 (+1519368us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:30.678468 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431052672}
I20260709 12:13:30.680109 14398 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.681314 14398 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:30.659866 11251 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:30.661598 14379 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:30.662664 14055 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:30.663859 13972 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.912s	user 0.010s	sys 0.001s
I20260709 12:13:30.639196 14474 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.408s
W20260709 12:13:30.665882 14198 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.710s	user 0.007s	sys 0.004s
I20260709 12:13:30.667181 14464 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.637915 14479 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:30.625332 14480 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:30.682515 14484 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:13:30.684360 14226 consensus_queue.cc:1175] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 3 status { last_received { term: 3 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 3 index: 2 } }
I20260709 12:13:30.684994 14389 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:30.659081 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:30.731560 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.004s	sys 0.007s
W20260709 12:13:30.714558 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.625708 13989 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:30.728665 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 105066us
W20260709 12:13:30.732355 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73375.9us
W20260709 12:13:30.692191 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:30.737121 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.004s	sys 0.007s
I20260709 12:13:30.733115 11182 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:30.739365 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:30.740648 11182 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:13:30.742125 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:30.742719 11328 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:30.744872 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:30.744976 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:30.745613 11330 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:30.747808 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:30.752736 11328 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
W20260709 12:13:30.737613 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 533452us
I20260709 12:13:30.756383 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.756853 11180 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:13:30.759966 11180 pending_rounds.cc:85] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 4
I20260709 12:13:30.760318 11180 pending_rounds.cc:107] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:13:30.760048 11179 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:30.760842 11180 raft_consensus.cc:2889] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:30.757899 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72475us
I20260709 12:13:30.769443 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:30.757325 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:30.771443 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 187248us
I20260709 12:13:30.728158 14484 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:30.772523 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:30.773985 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:30.756408 14385 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.951s	user 0.012s	sys 0.000s
I20260709 12:13:30.774250 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:30.725405 14198 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.775022 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:30.773452 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.731688 14389 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:30.741950 14228 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.005s	sys 0.010s
I20260709 12:13:30.737932 14379 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.788205 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:30.720551 13972 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.766695 14488 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:30.772352 14460 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.794056 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:30.795027 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:30.787823 14071 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.794096 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:30.789840 14228 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.005s	sys 0.010s
I20260709 12:13:30.799233 11181 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:30.785856 14465 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:30.728294 14479 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.807693 11255 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:30.730160 14480 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.792994 14457 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:30.799502 14467 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:30.792191 14488 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:30.788738 14385 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:30.821295 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:30.802239 14269 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:13:30.807565 14447 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.876s	user 0.014s	sys 0.000s
I20260709 12:13:30.842099 11178 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:30.787627 14481 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:13:30.805982 14228 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:30.791862 14490 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:30.796607 14379 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.813062 14494 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:30.821432 14457 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:30.802374 14391 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:13:30.805713 14071 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.853133 11256 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:30.824179 14450 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:30.855597 14492 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:30.877789 14492 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:30.878814 14492 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.856364 14452 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.857018 14451 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:30.880118 14451 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:30.880977 14451 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.882195 14451 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:30.882870 14372 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.156s	user 0.013s	sys 0.000s
I20260709 12:13:30.841933 14465 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:30.824107 14469 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:30.876941 14071 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.863394 14344 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.847097 14467 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:30.896224 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:30.914508 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.850867 14490 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:30.862088 14447 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:30.834676 14464 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:13:30.923434 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:30.922921 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 153561us
I20260709 12:13:30.924145 11324 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:30.842715 14497 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:30.854285 14494 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:30.866109 14457 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:30.922573 14490 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:30.883371 14372 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.850129 14389 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:30.851893 14484 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:30.903738 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59109.3us
W20260709 12:13:30.984321 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 189372us
I20260709 12:13:30.897243 14246 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.922645 14447 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.124428 14447 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.124905 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:31.125679 11180 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:30.898334 14481 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:31.130204 14481 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:31.123385 11178 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:13:30.927963 14290 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.300s	user 0.013s	sys 0.001s
I20260709 12:13:31.050359 14448 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:31.049333 14235 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.532s	user 0.004s	sys 0.007s
W20260709 12:13:31.029224 14500 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:13:30.927801 14493 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.044926 14442 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:31.051802 13972 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:31.051119 14385 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.056128 14422 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
I20260709 12:13:31.004920 14498 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.087325 14494 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.138688 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:31.144994 14389 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:30.956821 14488 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:31.147706 14449 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.948s	user 0.013s	sys 0.000s
I20260709 12:13:31.166231 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:30.961199 14198 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.150746 14498 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.152495 14509 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.050985 14479 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:31.153194 14372 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:30.927088 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:30.928910 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 171680us
I20260709 12:13:31.053885 14480 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:13:31.185824 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:31.159257 11178 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:31.187326 14502 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:13:31.004844 13989 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:31.159482 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:31.139499 14460 consensus_queue.cc:1175] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "786851d4227e4dd89b75ccdea0b1f03b" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
I20260709 12:13:31.049831 14465 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.171518 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:31.057152 14379 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:31.170060 14385 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:31.212491 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 285500us
I20260709 12:13:31.003211 14228 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.163995 11180 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
W20260709 12:13:31.216352 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44918.4us
I20260709 12:13:31.174197 11179 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:30.880129 14452 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.176570 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:30.936424 14469 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:31.188717 11182 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:31.167070 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:31.160090 14497 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.189963 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:31.226589 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:31.227908 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:31.228932 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:31.175101 14449 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:31.206286 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.142073 14235 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:31.209811 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.059974 14307 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.223140 11174 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:31.233222 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:31.223418 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:31.229000 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52594us
I20260709 12:13:31.138228 14290 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.240334 14469 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.241391 14381 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:31.243161 14235 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.244182 11326 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:31.231076 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.209977 14486 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:31.238050 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:31.248059 11927 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:13:31.239517 11935 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:31.239992 14510 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:13:31.223455 14502 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:31.251530 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:31.253749 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:31.255808 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:31.261000 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:31.271700 11179 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:31.273469 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:31.288028 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:31.288597 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 65293.5us
I20260709 12:13:31.236474 14014 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:31.289202 11248 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.277665 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:31.289904 11933 raft_consensus.cc:2368] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8.
I20260709 12:13:31.292908 11252 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:31.293128 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:31.292908 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:31.293947 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:31.234902 14473 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:31.236218 14488 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.224336 14512 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:13:31.276506 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.244096 14344 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:31.050225 14467 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.155905 14493 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.247624 14386 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:31.319409 11182 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:31.250270 13972 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:31.289739 14372 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:31.184864 14509 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.263209 14502 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:31.341225 14502 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.346761 14307 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.348467 14452 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:31.349907 14290 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:31.350046 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:31.236100 14497 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.351487 14502 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:31.337296 11148 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 2.402s (SENT) [suppressed 8 similar messages]
I20260709 12:13:31.352800 13972 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.235033 14481 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.354399 14481 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:31.355669 13972 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.306114 14473 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.357600 14502 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:31.358387 14522 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:13:31.279363 14071 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:31.361912 14071 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.362459 14071 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:31.306977 14391 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:13:31.369752 14386 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.373018 14386 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.357908 14473 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.351889 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:31.337935 11248 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:31.384310 11251 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:31.304296 14014 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:31.303404 14246 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.308511 14498 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.313916 14512 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.389982 14505 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:31.390651 14519 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:31.322723 14269 raft_consensus.cc:2764] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:13:31.353389 14497 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.325299 14344 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:31.397395 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:31.400234 14506 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:31.395103 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:31.405264 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 138410us
I20260709 12:13:31.592974 11251 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:31.530948 11934 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:31.393419 14014 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:31.604429 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0745286279998254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 12:13:31.606091 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0251866549999704s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 12:13:31.609370 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 676321us
W20260709 12:13:31.610690 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:31.611150 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:31.611490 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.612627 11147 leader_election.cc:400] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8.
I20260709 12:13:31.613037 11147 leader_election.cc:403] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:31.612783 14380 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:31.613763 14380 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:31.615635 14380 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:31.616650 14526 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.616843 14448 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:31.617424 14448 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.617942 14474 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.618705 14474 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.619870 14448 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:31.620729 14448 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:31.621089 14448 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:31.621865 14442 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:31.622325 14442 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:31.623303 14442 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.625545 14448 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.627123 14448 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:31.626381 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:31.634956 14526 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8.
I20260709 12:13:31.635828 14526 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:31.636375 14526 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:31.409320 14071 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.637332 14526 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.651238 14071 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:31.653360 11256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1794) took 1681 ms (client timeout 2024 ms). Trace:
W20260709 12:13:31.654095 11256 rpcz_store.cc:269] 0709 12:13:29.972294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.972490 (+   196us) service_pool.cc:224] Handling call
0709 12:13:31.653015 (+1680525us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b5400069354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:31.653267 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2012299520}
I20260709 12:13:31.655678 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:31.656718 11256 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:31.658931 14461 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:31.659546 14461 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:31.667971 14441 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:31.671043 14499 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.750s	user 0.020s	sys 0.000s
I20260709 12:13:31.601527 11900 leader_election.cc:400] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7.
I20260709 12:13:31.708329 11900 leader_election.cc:403] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:13:31.397805 14505 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.397367 14512 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:31.411294 14451 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.004s
I20260709 12:13:31.398468 14519 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:31.406123 14509 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.531320 14422 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:31.537201 14510 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:31.602044 14464 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.602305 14498 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.746238 11248 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:31.660640 14461 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.406530 14449 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.531185 14372 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:31.712211 14486 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:31.885895 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:31.804677 14372 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:31.744076 14505 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:31.748571 14519 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:31.866673 11252 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:31.864178 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 514) took 1920 ms. Trace:
W20260709 12:13:31.853734 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.889946 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:31.941983 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.891569 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:31.876829 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 383) took 1503 ms. Trace:
I20260709 12:13:31.907414 11250 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: [6.8-8.9]
W20260709 12:13:31.738912 14447 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.614s	user 0.012s	sys 0.000s
I20260709 12:13:32.026620 11176 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:31.891285 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:31.795881 14457 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.000s
I20260709 12:13:31.724265 14474 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:31.854898 14541 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:13:31.724987 14506 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.025733 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:32.143898 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:32.559871 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:32.560523 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:32.103335 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69549us
I20260709 12:13:31.848742 14461 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:32.563196 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 390) took 2043 ms. Trace:
W20260709 12:13:32.362902 14467 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.002s	sys 0.015s
I20260709 12:13:32.563627 11254 rpcz_store.cc:276] 0709 12:13:30.519975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.613905 (+ 93930us) service_pool.cc:224] Handling call
0709 12:13:32.562949 (+1949044us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b5400086bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:32.563117 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2212064000}
W20260709 12:13:32.564304 14467 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.003s	sys 0.016s
W20260709 12:13:31.859987 14494 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.001s
I20260709 12:13:32.564716 14469 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.262193 14537 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:32.564980 14494 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.565296 14469 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.565876 14537 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.566337 11248 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:32.214851 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 613434us
I20260709 12:13:32.567570 14469 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:32.568439 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:32.539417 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:32.026080 11927 rpcz_store.cc:276] 0709 12:13:29.943398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.943585 (+   187us) service_pool.cc:224] Handling call
0709 12:13:29.980133 (+ 36548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:31.262704 (+1282571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.863865 (+601161us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540059a6d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:31.864084 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2906648832}
I20260709 12:13:32.302956 11933 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:32.147513 11330 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:13:32.466099 14450 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.540s	user 0.017s	sys 0.018s
I20260709 12:13:32.414364 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:31.861457 14504 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:32.571367 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:32.131089 14476 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.510s	user 0.000s	sys 0.009s
I20260709 12:13:32.072945 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:32.148738 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:31.884210 14443 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.215441 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:32.052708 14449 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.404023 11180 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:32.128114 14451 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.579456 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 395) took 1189 ms. Trace:
I20260709 12:13:32.148262 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:32.621928 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1653) took 1344 ms. Trace:
I20260709 12:13:32.221469 14513 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7.
I20260709 12:13:32.134616 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:32.327756 14536 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.027851 11255 rpcz_store.cc:276] 0709 12:13:30.374001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.374126 (+   125us) service_pool.cc:224] Handling call
0709 12:13:31.876525 (+1502399us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54000f87d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:31.876722 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584078592}
I20260709 12:13:32.579982 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:32.582299 11929 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:32.244604 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:31.883966 14491 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:31.854414 14378 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:31.900146 14488 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.568s	user 0.006s	sys 0.001s
I20260709 12:13:31.882483 14493 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:31.918334 13989 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:32.139245 14490 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.018s	user 0.012s	sys 0.000s
W20260709 12:13:32.135044 14246 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.737s	user 0.003s	sys 0.007s
W20260709 12:13:32.098186 14257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.088s	user 0.023s	sys 0.051s
W20260709 12:13:32.460947 14535 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:13:31.918334 14398 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.044307 14498 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.149158 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:32.558329 14456 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.221854 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:32.671463 14246 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:32.244872 14389 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.070s	user 0.012s	sys 0.000s
I20260709 12:13:32.243553 14480 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.794320 14464 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:32.618716 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:32.614241 11331 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:32.725608 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:32.123674 14386 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:32.016885 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 405151us
I20260709 12:13:32.621733 11250 rpcz_store.cc:276] 0709 12:13:31.390342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.906579 (+516237us) service_pool.cc:224] Handling call
0709 12:13:31.907048 (+   469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:32.579216 (+672168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.579406 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:31.753784 14422 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.625923 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2070) took 2754 ms. Trace:
I20260709 12:13:32.574219 14441 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:32.606495 14473 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.210s	user 0.011s	sys 0.001s
I20260709 12:13:32.243335 14479 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.580505 14537 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.582005 14467 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:32.638615 11330 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:32.637256 11182 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 12:13:32.627022 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 554188us
I20260709 12:13:32.622313 11251 rpcz_store.cc:276] 0709 12:13:31.277851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.385068 (+107217us) service_pool.cc:224] Handling call
0709 12:13:31.389603 (+  4535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:31.601819 (+212216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.621592 (+1019773us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b5400109354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:32.621868 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57383808}
I20260709 12:13:32.625460 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:32.752466 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 127089us
I20260709 12:13:32.703439 14398 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.614771 14307 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.606387 14504 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:32.599072 14508 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:32.621361 13972 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.723129 14456 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.625690 14492 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.643535 14509 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:32.784281 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:32.611281 11328 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:32.737691 14386 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.626274 14531 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:32.605801 14447 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.599069 14457 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.616895 14449 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.706099 14502 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:32.671597 13989 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.221225 14486 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:32.671651 14257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.664s	user 0.047s	sys 0.116s
I20260709 12:13:32.818202 14398 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:32.819202 14504 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.820554 14504 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:32.824405 14447 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.825567 11331 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:32.829357 11926 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:32.711113 14442 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:32.653358 14061 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:32.582718 14450 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.662s	user 0.018s	sys 0.019s
I20260709 12:13:32.717932 11323 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 5.
I20260709 12:13:32.646270 14491 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.636754 14451 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.625924 14443 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:32.732138 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:32.734619 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:32.714445 14385 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.648115 14536 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.702234 14372 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:32.729835 11326 rpcz_store.cc:276] 0709 12:13:29.871677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.877222 (+  5545us) service_pool.cc:224] Handling call
0709 12:13:29.877525 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:31.244652 (+1367127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.625680 (+1381028us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000c5dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:32.625858 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3142582656}
W20260709 12:13:32.758153 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 146763us
I20260709 12:13:32.748605 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:34.809443 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 2.06075e+06us
I20260709 12:13:34.812780 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:32.787376 11174 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:34.816425 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2069) took 2717 ms. Trace:
I20260709 12:13:34.816768 11174 rpcz_store.cc:276] 0709 12:13:32.099098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.100102 (+  1004us) service_pool.cc:224] Handling call
0709 12:13:32.216283 (+116181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:34.815529 (+2599246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.816200 (+   671us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54001f36d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:34.816364 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1312433536}
I20260709 12:13:34.817830 14486 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:34.818300 14486 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:34.818650 14486 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:32.678356 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1658) took 1352 ms. Trace:
I20260709 12:13:34.824249 11248 rpcz_store.cc:276] 0709 12:13:31.326230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.848015 (+521785us) service_pool.cc:224] Handling call
0709 12:13:31.848399 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.566739 (+718340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.678002 (+111263us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54001dd3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:32.678253 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1650684800}
I20260709 12:13:32.760718 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:34.834725 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 399) took 2074 ms (client timeout 1859 ms). Trace:
W20260709 12:13:34.835107 11250 rpcz_store.cc:269] 0709 12:13:32.760210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.760347 (+   137us) service_pool.cc:224] Handling call
0709 12:13:34.834644 (+2074297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:32.771952 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:32.858778 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 124272us
I20260709 12:13:34.853279 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:32.786429 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:34.854732 11328 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:32.786644 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:32.780285 11330 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
W20260709 12:13:34.865662 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2092) took 2613 ms (client timeout 1528 ms). Trace:
W20260709 12:13:34.866024 11330 rpcz_store.cc:269] 0709 12:13:32.252062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.579754 (+327692us) service_pool.cc:224] Handling call
0709 12:13:34.865531 (+2285777us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:34.867156 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2095) took 2568 ms (client timeout 1832 ms). Trace:
W20260709 12:13:34.867532 11330 rpcz_store.cc:269] 0709 12:13:32.298504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.866970 (+2568466us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:34.867103 (+   133us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:34.868237 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2089) took 2617 ms (client timeout 1933 ms). Trace:
W20260709 12:13:34.868506 11330 rpcz_store.cc:269] 0709 12:13:32.251218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.868099 (+2616881us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:34.868192 (+    93us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:34.869196 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1463) took 2542 ms (client timeout 1991 ms). Trace:
I20260709 12:13:32.801949 11934 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:32.768610 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2075) took 2760 ms. Trace:
W20260709 12:13:34.872751 11934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 520) took 2847 ms (client timeout 2156 ms). Trace:
W20260709 12:13:34.873358 11934 rpcz_store.cc:269] 0709 12:13:32.025456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.025576 (+   120us) service_pool.cc:224] Handling call
0709 12:13:34.872559 (+2846983us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540053a7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:34.872684 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401042176}
I20260709 12:13:34.872624 11322 rpcz_store.cc:276] 0709 12:13:30.007829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.009095 (+  1266us) service_pool.cc:224] Handling call
0709 12:13:30.020731 (+ 11636us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:30.025988 (+  5257us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:30.202988 (+177000us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.606074 (+1403086us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b180031c3c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:13:32.151934 (+545860us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:32.189296 (+ 37362us) log.cc:844] Serialized 26 byte log entry
0709 12:13:32.189574 (+   278us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:32.648750 (+459176us) log.cc:844] Serialized 14 byte log entry
0709 12:13:32.649017 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.649115 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:32.768281 (+119166us) raft_consensus.cc:1685] finished
0709 12:13:32.768312 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:32.768515 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":300868,"prepare.queue_time_us":905673,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":110,"raft.queue_time_us":344015,"raft.run_cpu_time_us":564,"raft.run_wall_time_us":57299,"spinlock_wait_cycles":3662592,"thread_start_us":1019764,"threads_started":3,"wal-append.queue_time_us":1906754}
I20260709 12:13:32.860414 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:34.511394 14564 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:34.874919 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:34.875721 14564 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:34.869503 11330 rpcz_store.cc:269] 0709 12:13:32.327171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.869049 (+2541878us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:34.869148 (+    99us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:34.877043 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 541) took 2155 ms (client timeout 1859 ms). Trace:
W20260709 12:13:34.877364 11330 rpcz_store.cc:269] 0709 12:13:32.722154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.876884 (+2154730us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:34.876986 (+   102us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:34.878129 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 535) took 2260 ms (client timeout 2183 ms). Trace:
I20260709 12:13:34.878165 14564 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:32.846853 14061 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:32.765743 14246 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:34.875211 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 542) took 2144 ms (client timeout 1808 ms). Trace:
I20260709 12:13:32.857910 14385 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.884042 14385 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.671231 14490 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:34.887693 14490 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:33.349532 14491 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:34.889683 14491 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.890156 14491 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.892200 14491 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.783583 13972 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.894870 13972 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:32.931470 14014 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.206s	user 0.012s	sys 0.000s
I20260709 12:13:34.896886 14014 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.899049 14014 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.919998 14257 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:34.901074 14257 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:34.903337 14257 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:32.773473 14307 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:34.905941 14307 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:31.743050 14344 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:33.361563 14378 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.004s
I20260709 12:13:32.798960 14422 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:34.917299 11327 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:32.743129 14441 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:34.923800 14441 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:33.352248 14442 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.522s	user 0.004s	sys 0.004s
I20260709 12:13:33.324412 14443 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.907449 14447 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:34.927405 14447 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:33.325424 14449 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:32.838061 14450 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:34.931860 14450 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:33.322589 14451 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.793982 14456 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:32.835104 14457 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:33.361207 14461 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.798s	user 0.011s	sys 0.000s
I20260709 12:13:32.784879 14467 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:34.938297 14467 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:34.938748 14467 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:32.743422 14473 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.746843 14479 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.944468 14479 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.736171 14480 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.947377 14480 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.787218 14492 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:33.050386 14498 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:33.366794 14504 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
W20260709 12:13:32.919327 14505 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.815s	user 0.000s	sys 0.011s
I20260709 12:13:34.952184 14505 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.782570 14508 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.955899 14508 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.807049 14531 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:31.799260 14510 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:34.960038 14510 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:33.348364 14519 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.136s	user 0.005s	sys 0.005s
I20260709 12:13:32.834420 14513 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.962270 14519 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.963011 14513 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.964622 14519 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:34.965322 11932 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:34.965811 14513 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:32.747318 14532 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.446s	user 0.000s	sys 0.001s
W20260709 12:13:33.357542 14512 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.571s	user 0.011s	sys 0.001s
W20260709 12:13:32.793072 14533 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:13:34.977172 14492 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:34.977317 14498 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:32.856832 14535 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:34.978534 14505 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:32.847295 14537 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.968816 14447 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:34.980163 14537 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:34.981799 11148 connection.cc:256] RPC call timeout handler was delayed by -2.6938303540000561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
I20260709 12:13:34.982425 14461 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.982906 14461 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:32.917407 14543 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:13:32.798362 14542 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:13:32.793877 14547 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:13:32.701309 14548 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:13:32.935469 14552 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.585202 13989 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:34.590114 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 529) took 2029 ms. Trace:
I20260709 12:13:34.591364 14427 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.595546 14300 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:34.599910 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:34.606798 14386 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:34.610821 11325 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:34.619726 14094 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:34.625737 11179 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.817s	user 0.027s	sys 0.047s
W20260709 12:13:34.633416 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.871s	user 0.045s	sys 0.038s
I20260709 12:13:34.646550 14567 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.855475 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:34.660396 14536 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:34.684165 11178 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:34.691097 14484 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:34.699785 11935 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.934s	user 0.006s	sys 0.012s
I20260709 12:13:34.703208 14506 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:34.704584 11928 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.949s	user 0.012s	sys 0.007s
W20260709 12:13:34.715456 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:34.726150 11222 connection.cc:256] RPC call timeout handler was delayed by -2.1649240750000445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 12:13:34.726826 14568 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:34.745855 11901 connection.cc:256] RPC call timeout handler was delayed by -1.0948299720000705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 12:13:34.749364 14389 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:13:34.756389 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:34.761894 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:34.763322 11926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 527) took 2294 ms (client timeout 1580 ms). Trace:
W20260709 12:13:34.773595 14546 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 2.014s	user 0.002s	sys 0.009s
I20260709 12:13:34.776194 14055 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.788039 14460 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:34.790181 11933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:13:34.798884 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:34.799248 11331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2088) took 2548 ms (client timeout 1580 ms). Trace:
I20260709 12:13:34.800374 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:34.800881 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 396) took 3410 ms. Trace:
I20260709 12:13:34.803000 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2068) took 2749 ms. Trace:
I20260709 12:13:34.826233 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:34.827278 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:34.829180 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:34.838683 11223 connection.cc:256] RPC call timeout handler was delayed by -1.878439655999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 12:13:34.844055 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:34.848387 14486 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.849184 14464 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:34.850304 11257 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:34.859984 14403 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:34.861303 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:34.862989 14494 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:34.863937 11328 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:34.870647 11079 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:13:34.878539 11330 rpcz_store.cc:269] 0709 12:13:32.617861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.877972 (+2260111us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:34.878070 (+    98us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:34.875463 11323 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:34.881341 11322 rpcz_store.cc:269] 0709 12:13:32.731287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.875047 (+2143760us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:34.875154 (+   107us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:34.983027 14531 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:35.056205 11179 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.247s	user 0.029s	sys 0.048s
I20260709 12:13:34.881244 14246 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.056324 14531 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.056072 11326 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:35.056666 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.056977 14531 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:35.057845 11147 connection.cc:256] RPC call timeout handler was delayed by -2.2580847429999267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:13:35.058220 11147 connection.cc:256] RPC call timeout handler was delayed by -2.1474824179999814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
I20260709 12:13:35.058348 14246 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:34.888151 14061 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:32.716149 14228 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:34.899624 11176 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:35.122357 11927 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:13:35.124053 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.124727 11176 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:34.518460 14565 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.125967 14565 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.128348 14565 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:35.122360 14228 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:34.926156 11324 raft_consensus.cc:2410] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:13:35.129809 11324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1455) took 2983 ms (client timeout 1938 ms). Trace:
I20260709 12:13:35.129499 14228 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:35.130213 11324 rpcz_store.cc:269] 0709 12:13:32.146183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.148881 (+  2698us) service_pool.cc:224] Handling call
0709 12:13:35.129567 (+2980686us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b54000c6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:35.129734 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5108587392}
W20260709 12:13:35.131299 11324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 543) took 2285 ms (client timeout 2112 ms). Trace:
W20260709 12:13:35.131642 11324 rpcz_store.cc:269] 0709 12:13:32.845843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.131105 (+2285262us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:35.131232 (+   127us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:35.132377 11324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1468) took 2560 ms (client timeout 2402 ms). Trace:
W20260709 12:13:35.132686 11324 rpcz_store.cc:269] 0709 12:13:32.571999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.132225 (+2560226us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:35.132325 (+   100us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:34.937126 11180 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:34.961077 11329 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:34.961232 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1654) took 3669 ms. Trace:
W20260709 12:13:34.970045 11932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 607) took 3693 ms (client timeout 1887 ms). Trace:
I20260709 12:13:34.976447 14512 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:34.977838 14498 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.055994 11175 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:35.058786 11147 connection.cc:256] RPC call timeout handler was delayed by -1.8738824179999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
I20260709 12:13:35.059065 14531 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.059103 14061 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.001824 14443 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:34.984359 14473 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.060142 14094 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.060186 14567 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.134855 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2072) took 2655 ms. Trace:
I20260709 12:13:35.061197 14508 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.984747 14492 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:35.001047 11327 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:35.143823 14506 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:35.144137 14508 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:35.144958 11327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1450) took 6053 ms (client timeout 2402 ms). Trace:
I20260709 12:13:35.145193 14506 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:35.062480 14505 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.062701 14491 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.141589 14443 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.146725 14491 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.062731 11325 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:35.135872 11329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1453) took 2990 ms (client timeout 3002 ms). Trace:
W20260709 12:13:35.148097 11329 rpcz_store.cc:269] 0709 12:13:32.145462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.147904 (+  2442us) service_pool.cc:224] Handling call
0709 12:13:35.135657 (+2987753us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:35.135805 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5378354048}
I20260709 12:13:35.148620 14443 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.148286 11325 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.061458 14460 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.061024 11929 rpcz_store.cc:276] 0709 12:13:32.560850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.649723 (+ 88873us) service_pool.cc:224] Handling call
0709 12:13:32.681264 (+ 31541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.767453 (+ 86189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:32.767502 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:32.767562 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:32.767607 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.589804 (+1822197us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b54003b2754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:34.589856 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.590028 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197463808}
I20260709 12:13:35.061762 14300 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:13:35.065744 14486 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.066385 14494 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:35.136181 11932 rpcz_store.cc:269] 0709 12:13:31.277184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.277368 (+   184us) service_pool.cc:224] Handling call
0709 12:13:34.969862 (+3692494us) spinlock_profiling.cc:243] Waited 3.67 s on lock 0x7b5400690ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:34.969965 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8439792512}
I20260709 12:13:35.144163 14492 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.141217 14061 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.151435 11932 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:13:35.151603 14460 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.152238 14492 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.065948 14568 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.067107 14403 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:35.067307 11933 raft_consensus.cc:606] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:35.141983 14473 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:13:35.153228 14494 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.153903 14508 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:35.143867 11180 rpcz_store.cc:276] 0709 12:13:32.479414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.617355 (+137941us) service_pool.cc:224] Handling call
0709 12:13:32.624754 (+  7399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.133811 (+2509057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.134558 (+   747us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b54002b8254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:35.134779 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5317051136}
I20260709 12:13:35.154646 14505 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.155323 14494 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.155548 11251 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
W20260709 12:13:35.156994 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 398) took 2397 ms (client timeout 2577 ms). Trace:
W20260709 12:13:35.069586 11328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1461) took 2807 ms (client timeout 1583 ms). Trace:
W20260709 12:13:35.157354 11251 rpcz_store.cc:269] 0709 12:13:32.759888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.760009 (+   121us) service_pool.cc:224] Handling call
0709 12:13:35.156425 (+2396416us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54000884d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:35.156918 (+   493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682860928}
W20260709 12:13:35.157785 11328 rpcz_store.cc:269] 0709 12:13:32.262619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.786190 (+523571us) service_pool.cc:224] Handling call
0709 12:13:35.069503 (+2283313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.121618 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 454) took 3958 ms. Trace:
I20260709 12:13:35.065860 14467 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.152796 11325 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.158402 11180 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:35.158687 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.159538 11328 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 12:13:35.061484 11223 connection.cc:256] RPC call timeout handler was delayed by -1.8032444510001824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
I20260709 12:13:35.142426 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 528) took 2655 ms. Trace:
I20260709 12:13:35.059368 11177 rpcz_store.cc:276] 0709 12:13:32.053210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.053491 (+   281us) service_pool.cc:224] Handling call
0709 12:13:32.053873 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:32.721877 (+668004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:32.721939 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:32.722441 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:13:34.727513 (+2005072us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b1800227c20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:34.746575 (+ 19062us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:34.746628 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.802659 (+ 56031us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54003d22d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:34.802715 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:34.802764 (+    49us) raft_consensus.cc:1685] finished
0709 12:13:34.802778 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.802923 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2036,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":8802,"spinlock_wait_cycles":1538160768,"thread_start_us":18847,"threads_started":1,"wal-append.queue_time_us":2004063}
I20260709 12:13:35.146283 14445 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.140686 11175 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:35.160719 14445 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.061810 11253 rpcz_store.cc:276] 0709 12:13:31.390637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.052287 (+661650us) service_pool.cc:224] Handling call
0709 12:13:32.053433 (+  1146us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:32.102608 (+ 49175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:32.102659 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:32.623107 (+520448us) log.cc:844] Serialized 52 byte log entry
0709 12:13:32.843336 (+220229us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:32.843395 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.843498 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:34.800603 (+1957105us) raft_consensus.cc:1685] finished
0709 12:13:34.800626 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.800789 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48322,"raft.queue_time_us":79212,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":131,"thread_start_us":161317,"threads_started":1,"wal-append.queue_time_us":532463,"wal-append.run_cpu_time_us":1911,"wal-append.run_wall_time_us":1001922}
I20260709 12:13:35.074433 11254 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:35.162147 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.162773 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.163340 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.163693 14445 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:35.164121 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:35.060844 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 199590us
W20260709 12:13:35.145469 11327 rpcz_store.cc:269] 0709 12:13:29.091074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.100196 (+  9122us) service_pool.cc:224] Handling call
0709 12:13:35.144784 (+6044588us) spinlock_profiling.cc:243] Waited 5.51 s on lock 0x7b5400322dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:35.144890 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12667199872}
I20260709 12:13:35.071859 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.160269 11927 rpcz_store.cc:276] 0709 12:13:32.487227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.618374 (+131147us) service_pool.cc:224] Handling call
0709 12:13:35.122075 (+2503701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.122811 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.142219 (+ 19408us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b54006c5050. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:35.142343 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5702707072}
I20260709 12:13:35.165872 11250 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:35.072811 11248 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:35.166311 11325 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
W20260709 12:13:35.082530 11222 connection.cc:256] RPC call timeout handler was delayed by -2.1015366810001979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 12:13:35.149106 14491 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:35.166826 11222 connection.cc:256] RPC call timeout handler was delayed by -1.9030240750000758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
W20260709 12:13:35.078217 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 544) took 2232 ms (client timeout 2013 ms). Trace:
W20260709 12:13:35.082166 11935 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.317s	user 0.007s	sys 0.013s
I20260709 12:13:35.167492 14505 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:35.167584 11330 rpcz_store.cc:269] 0709 12:13:32.846108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.077970 (+2231862us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:35.078137 (+   167us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:35.083468 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 536) took 2459 ms (client timeout 2238 ms). Trace:
W20260709 12:13:35.065768 11928 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.313s	user 0.012s	sys 0.009s
W20260709 12:13:35.085868 11901 connection.cc:256] RPC call timeout handler was delayed by -1.0489299719999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 12:13:35.168462 11322 rpcz_store.cc:269] 0709 12:13:32.623868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.083288 (+2459420us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:35.083412 (+   124us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:35.168207 11178 rpcz_store.cc:276] 0709 12:13:31.163072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.205108 (+ 42036us) service_pool.cc:224] Handling call
0709 12:13:31.205467 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.082918 (+3877451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.121398 (+ 38480us) spinlock_profiling.cc:243] Waited 3.48 s on lock 0x7b54002b7d54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:35.121566 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7999666432}
W20260709 12:13:35.160157 11223 connection.cc:256] RPC call timeout handler was delayed by -1.7915444510001635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18320000000000003s for a response
W20260709 12:13:35.056334 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.294s	user 0.046s	sys 0.038s
I20260709 12:13:35.169839 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.170550 11322 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:35.171036 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:35.171800 14519 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.172528 14519 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.173221 14464 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.174889 14460 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:35.092370 14577 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.093029 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2066) took 3073 ms. Trace:
I20260709 12:13:35.093583 11257 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
W20260709 12:13:35.181461 11257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 400) took 2421 ms (client timeout 1808 ms). Trace:
W20260709 12:13:35.182078 11257 rpcz_store.cc:269] 0709 12:13:32.760486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.771665 (+ 11179us) service_pool.cc:224] Handling call
0709 12:13:35.181370 (+2409705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.093951 14552 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.183480 14577 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.181241 11179 rpcz_store.cc:276] 0709 12:13:32.019197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.019508 (+   311us) service_pool.cc:224] Handling call
0709 12:13:32.124804 (+105296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:32.572443 (+447639us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:32.766447 (+194004us) log.cc:844] Serialized 14 byte log entry
0709 12:13:35.066082 (+2299635us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:35.066742 (+   660us) log.cc:844] Serialized 52 byte log entry
0709 12:13:35.092192 (+ 25450us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:35.092239 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.092774 (+   535us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:35.092824 (+    50us) raft_consensus.cc:1685] finished
0709 12:13:35.092835 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.092964 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":446728,"prepare.queue_time_us":146438,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3527,"raft.queue_time_us":448695,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":189,"thread_start_us":2475585,"threads_started":3,"wal-append.queue_time_us":586168,"wal-append.run_cpu_time_us":1606,"wal-append.run_wall_time_us":1002364}
I20260709 12:13:34.983659 14537 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.097599 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2062) took 3209 ms. Trace:
I20260709 12:13:35.186298 14577 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.142987 14512 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.140144 14498 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.186676 11173 rpcz_store.cc:276] 0709 12:13:31.888165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.888322 (+   157us) service_pool.cc:224] Handling call
0709 12:13:31.888765 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:32.024482 (+135717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:32.024531 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:32.102331 (+ 77800us) log.cc:844] Serialized 26 byte log entry
0709 12:13:32.620245 (+517914us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:32.620285 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.620352 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:35.097254 (+2476902us) raft_consensus.cc:1685] finished
0709 12:13:35.097286 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.097514 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29354,"raft.queue_time_us":29548,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":32858,"thread_start_us":491910,"threads_started":1,"wal-append.queue_time_us":573974}
I20260709 12:13:35.149398 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.188627 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.189340 11329 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:13:35.089267 11326 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:35.192811 11326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2096) took 2894 ms (client timeout 1819 ms). Trace:
W20260709 12:13:35.193214 11326 rpcz_store.cc:269] 0709 12:13:32.298816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.855134 (+556318us) service_pool.cc:224] Handling call
0709 12:13:35.192720 (+2337586us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.194229 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.195174 11326 raft_consensus.cc:2410] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:13:35.196154 11326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1470) took 2623 ms (client timeout 3334 ms). Trace:
W20260709 12:13:35.196549 11326 rpcz_store.cc:269] 0709 12:13:32.572587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.193948 (+2621361us) service_pool.cc:224] Handling call
0709 12:13:35.196102 (+  2154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.109438 14539 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.198060 14539 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.201267 14539 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:35.202431 11329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 532) took 2628 ms (client timeout 2640 ms). Trace:
I20260709 12:13:35.115458 11182 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:35.197515 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.168735 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.210891 11326 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:35.121600 11931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 523) took 2877 ms (client timeout 1871 ms). Trace:
I20260709 12:13:35.145519 14567 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:35.143436 14094 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:35.211872 11931 rpcz_store.cc:269] 0709 12:13:32.244227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.244351 (+   124us) service_pool.cc:224] Handling call
0709 12:13:35.121355 (+2877004us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b54006c5054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:35.121514 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5590719872}
I20260709 12:13:35.136034 11256 rpcz_store.cc:276] 0709 12:13:31.292004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.701486 (+409482us) service_pool.cc:224] Handling call
0709 12:13:31.740880 (+ 39394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:34.958820 (+3217940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:34.958852 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:34.958897 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:34.958937 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.960959 (+  2022us) spinlock_profiling.cc:243] Waited 3.22 s on lock 0x7b54000f7dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:34.961005 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.961168 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7400046848}
I20260709 12:13:35.059854 14427 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.218111 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:35.222369 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.153198 11933 raft_consensus.cc:614] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:35.155040 14300 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.068807 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.222992 11182 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:35.149158 14443 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:35.068869 14389 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:35.161198 11177 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 12:13:35.073403 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.158418 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.223698 14568 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.160169 14467 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.165159 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:35.146072 14506 consensus_queue.cc:1243] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:13:35.114385 11181 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:35.075320 11249 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:35.230149 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 65087.4us
I20260709 12:13:35.075989 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.230805 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.165545 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.231536 11327 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:35.231577 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.231889 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:35.232074 11327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2097) took 2933 ms (client timeout 3050 ms). Trace:
W20260709 12:13:35.059870 11331 rpcz_store.cc:269] 0709 12:13:32.250895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.725267 (+474372us) service_pool.cc:224] Handling call
0709 12:13:34.799043 (+2073776us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b5400131fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:34.799179 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227386624}
W20260709 12:13:35.232472 11327 rpcz_store.cc:269] 0709 12:13:32.299109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.165269 (+2866160us) service_pool.cc:224] Handling call
0709 12:13:35.232026 (+ 66757us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.055994 14055 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.233049 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:35.167222 11222 connection.cc:256] RPC call timeout handler was delayed by -1.7197430430001077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18320000000000003s for a response
W20260709 12:13:35.233804 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5349240750001627s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22799999999999998s for a response
I20260709 12:13:35.234323 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:35.234789 14055 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.234889 11934 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:35.235129 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:35.168394 11325 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:13:35.235846 11325 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:13:35.236112 11325 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:35.236744 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2074) took 5249 ms. Trace:
I20260709 12:13:35.236910 14300 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:35.237126 11325 rpcz_store.cc:276] 0709 12:13:29.987510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.989739 (+  2229us) service_pool.cc:224] Handling call
0709 12:13:29.990122 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:34.610699 (+4620577us) spinlock_profiling.cc:243] Waited 2.82 s on lock 0x7b54004f0720. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6663dc0 00007f3ec66ee1c0 00007f3ec66ef231 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c
0709 12:13:35.236506 (+625807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.236689 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6474572672}
I20260709 12:13:35.237885 14228 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:35.233367 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.238296 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.238804 11327 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
W20260709 12:13:35.239015 14408 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:13:35.225797 14422 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:13:35.239705 14422 consensus_queue.cc:1243] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:13:35.240268 14422 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:35.240712 14422 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:35.175673 14519 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.226704 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:35.242394 11178 raft_consensus.cc:2368] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:35.060233 11930 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:35.228186 14389 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.228524 11255 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:35.244477 14389 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:35.245136 14403 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:35.236753 14494 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:13:35.246114 14403 raft_consensus.cc:2764] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:35.246248 14494 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:35.246766 14494 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:35.233099 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.246335 11079 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:35.247655 11331 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:35.247653 14494 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.248822 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.249032 14494 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:35.248456 11079 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 7, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.250382 11933 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:35.250639 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.178262 14457 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:35.251418 14457 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:13:35.159921 11328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 540) took 2459 ms (client timeout 2577 ms). Trace:
I20260709 12:13:35.161266 11175 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:35.176035 14456 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:35.252578 11931 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:35.252842 11175 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:35.252300 11328 rpcz_store.cc:269] 0709 12:13:32.700191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.158472 (+2458281us) service_pool.cc:224] Handling call
0709 12:13:35.159869 (+  1397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:34.884850 14576 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:35.245121 11222 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.871s (SENT)
I20260709 12:13:35.254148 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.253986 14576 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.254758 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
W20260709 12:13:35.255216 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.255582 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.256340 11328 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:35.256558 14576 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.256891 14567 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:35.257346 14567 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.257576 11175 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:35.225232 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:35.258641 11173 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:35.251785 14457 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:35.259913 14457 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.261094 14457 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.205901 14464 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.262739 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.262970 11323 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
W20260709 12:13:35.263554 11323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1462) took 2955 ms (client timeout 1788 ms). Trace:
I20260709 12:13:35.263676 14464 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:35.263902 11323 rpcz_store.cc:269] 0709 12:13:32.307905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.836526 (+528621us) service_pool.cc:224] Handling call
0709 12:13:35.263490 (+2426964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.264760 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.265354 11929 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.238886 11325 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.266163 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.266881 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.267392 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.267809 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:35.268108 11929 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:35.268368 14567 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.268715 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.269171 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.152720 14061 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.207326 14512 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:35.269301 11930 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:35.269843 14512 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:35.270506 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.270620 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.270707 14512 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.271287 11930 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:35.271565 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.272185 14512 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.273193 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.226390 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.117779 14461 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.213235 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.227540 14443 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:35.275291 11326 raft_consensus.cc:2376] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7. Re-sending same reply.
I20260709 12:13:35.275929 14443 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.277113 14443 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:35.133602 11324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2105) took 2351 ms (client timeout 2246 ms). Trace:
I20260709 12:13:35.220714 14582 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:13:35.209874 11329 rpcz_store.cc:269] 0709 12:13:32.573665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.149128 (+2575463us) service_pool.cc:224] Handling call
0709 12:13:35.202361 (+ 53233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.223369 11174 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
W20260709 12:13:35.277805 11324 rpcz_store.cc:269] 0709 12:13:32.781915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.133407 (+2351492us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:35.133538 (+   131us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:35.068236 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.278823 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.279201 14491 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:35.279520 11324 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 3.
I20260709 12:13:35.280429 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.280910 14491 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.281163 11324 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:35.281354 14491 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:35.283345 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.284217 14537 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:35.284580 14537 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.284940 14537 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.225191 14427 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.286262 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.286935 11249 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
W20260709 12:13:34.994067 11148 connection.cc:256] RPC call timeout handler was delayed by -2.359930353999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
I20260709 12:13:35.250174 11079 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 7, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 7 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:13:35.084828 11926 rpcz_store.cc:269] 0709 12:13:32.469046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.469234 (+   188us) service_pool.cc:224] Handling call
0709 12:13:34.763034 (+2293800us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400617954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:34.763218 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573313792}
W20260709 12:13:35.287647 11148 connection.cc:256] RPC call timeout handler was delayed by -2.3212303540001358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:13:35.288014 11148 connection.cc:256] RPC call timeout handler was delayed by -2.3190071850001459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
I20260709 12:13:35.288173 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:35.288352 11148 connection.cc:256] RPC call timeout handler was delayed by -2.3041303540001081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
I20260709 12:13:35.282505 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.177192 14513 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:13:35.288793 11148 connection.cc:256] RPC call timeout handler was delayed by -2.134507185000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 12:13:35.289160 11148 connection.cc:256] RPC call timeout handler was delayed by -1.8646560950001003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 12:13:35.289335 11931 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:13:35.289609 11148 connection.cc:256] RPC call timeout handler was delayed by -1.4023303540000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3002s for a response
I20260709 12:13:35.289273 14513 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:35.289954 11148 connection.cc:256] RPC call timeout handler was delayed by -1.3242149229999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24020000000000002s for a response
I20260709 12:13:35.290871 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.291380 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:35.268388 11927 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:35.239653 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.292013 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.292253 11327 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:35.293141 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.293910 11327 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:35.210978 11330 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.278165 14582 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:35.295035 14582 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:35.295105 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2094) took 2997 ms (client timeout 3087 ms). Trace:
W20260709 12:13:35.295521 11330 rpcz_store.cc:269] 0709 12:13:32.298200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.168489 (+2870289us) service_pool.cc:224] Handling call
0709 12:13:35.295051 (+126562us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:35.296344 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.274477 11179 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.288851 11926 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.274457 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:35.249405 11331 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.272240 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.288358 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.298821 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.299234 11249 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:35.299607 11927 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:35.300434 14061 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:35.300806 14061 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:35.301159 14061 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:35.186298 14552 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.302155 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.303231 11327 raft_consensus.cc:2410] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:35.303943 11934 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:35.282575 14408 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.331466 14408 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.334985 14408 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.257437 14422 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.336330 14422 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:35.285233 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.337957 11252 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.342713 14567 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.284600 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.291043 14513 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.267474 11325 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:35.343989 11255 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.344954 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.345172 11325 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:35.295921 14582 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.345767 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1457) took 3199 ms. Trace:
I20260709 12:13:35.346326 14582 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.346120 11325 rpcz_store.cc:276] 0709 12:13:32.146831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.344740 (+3197909us) service_pool.cc:224] Handling call
0709 12:13:35.344995 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.345537 (+   542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.345716 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.348318 14537 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.220839 14505 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:35.350463 14537 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:35.296402 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.351485 11330 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:35.351941 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1466) took 2886 ms. Trace:
I20260709 12:13:35.352281 11330 rpcz_store.cc:276] 0709 12:13:32.476945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.296172 (+2819227us) service_pool.cc:224] Handling call
0709 12:13:35.351873 (+ 55701us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.353695 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.353899 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1460) took 3157 ms. Trace:
I20260709 12:13:35.354444 11182 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.354218 11330 rpcz_store.cc:276] 0709 12:13:32.196131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.353078 (+3156947us) service_pool.cc:224] Handling call
0709 12:13:35.353405 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.353580 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.353605 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.353668 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:35.353699 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.353755 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.353851 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.355739 11330 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:35.355791 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.356330 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2090) took 3104 ms. Trace:
I20260709 12:13:35.356642 11330 rpcz_store.cc:276] 0709 12:13:32.251507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.355192 (+3103685us) service_pool.cc:224] Handling call
0709 12:13:35.355447 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.356114 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.356284 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.281690 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:35.358628 11933 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.360925 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.361552 11182 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 4.
I20260709 12:13:35.363986 11324 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:35.364598 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1459) took 3217 ms. Trace:
I20260709 12:13:35.364878 11324 rpcz_store.cc:276] 0709 12:13:32.147475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.351257 (+3203782us) service_pool.cc:224] Handling call
0709 12:13:35.351567 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.364355 (+ 12788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.364544 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.257200 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.304211 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.282096 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.366101 11324 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:35.366446 11329 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.367651 11328 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:35.367980 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2098) took 3068 ms. Trace:
I20260709 12:13:35.368363 11329 rpcz_store.cc:276] 0709 12:13:32.299388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.367173 (+3067785us) service_pool.cc:224] Handling call
0709 12:13:35.367514 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.367681 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.367717 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.367756 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:35.367791 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.367836 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.367932 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.369014 11328 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:35.225443 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.369997 11176 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.370301 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.370000 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:35.371155 11173 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.371358 11181 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.276255 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.372226 11329 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 12:13:35.306737 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.372874 11256 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.373726 11329 pending_rounds.cc:85] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 1
I20260709 12:13:35.373988 11329 pending_rounds.cc:107] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:13:35.374198 11329 raft_consensus.cc:2889] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:35.374475 11329 pending_rounds.cc:107] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:35.374670 11329 raft_consensus.cc:2889] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:35.375267 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2100) took 3075 ms. Trace:
I20260709 12:13:35.375548 11329 rpcz_store.cc:276] 0709 12:13:32.300013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.369227 (+3069214us) service_pool.cc:224] Handling call
0709 12:13:35.369462 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.375069 (+  5607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.375222 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.376732 11329 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:35.377184 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2101) took 3076 ms. Trace:
I20260709 12:13:35.305405 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.377895 11251 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:35.378325 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:35.379858 11900 connection.cc:256] RPC call timeout handler was delayed by -1.9819052029999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3295s for a response
I20260709 12:13:35.380623 14548 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.381109 14548 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.308061 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.382020 11248 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.383320 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.383607 14548 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.384162 11181 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:13:35.385042 11256 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:35.308670 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.308624 14061 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.307422 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.369580 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2099) took 3070 ms. Trace:
I20260709 12:13:35.386632 11257 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:35.386843 11328 rpcz_store.cc:276] 0709 12:13:32.299665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.368467 (+3068802us) service_pool.cc:224] Handling call
0709 12:13:35.368729 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.369362 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.369520 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.387740 14061 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:35.220022 11322 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:35.388147 11328 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:35.388474 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 534) took 2790 ms (client timeout 2833 ms). Trace:
I20260709 12:13:35.310199 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:35.388801 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1464) took 2992 ms. Trace:
I20260709 12:13:35.309401 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.311592 11331 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:35.311842 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50727.2us
I20260709 12:13:35.264806 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.314481 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.315725 14450 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.316533 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 524) took 3053 ms. Trace:
I20260709 12:13:35.321981 14508 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:35.323331 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.323761 11175 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:13:35.308774 14491 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.366565 11327 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:35.370435 11179 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:35.441293 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:35.372306 11326 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:13:35.306062 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.373513 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:35.306723 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 126671us
I20260709 12:13:35.417734 11935 rpcz_store.cc:276] 0709 12:13:32.263321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.263518 (+   197us) service_pool.cc:224] Handling call
0709 12:13:32.400506 (+136988us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:32.490918 (+ 90412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:32.490966 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:32.562711 (+ 71745us) log.cc:844] Serialized 52 byte log entry
0709 12:13:32.757895 (+195184us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:35.168037 (+2410142us) log.cc:844] Serialized 14 byte log entry
0709 12:13:35.315606 (+147569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.316085 (+   479us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:35.316133 (+    48us) raft_consensus.cc:1685] finished
0709 12:13:35.316152 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.316364 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":567838,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":146533,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":162,"thread_start_us":2742896,"threads_started":3,"wal-append.queue_time_us":536415,"wal-append.run_cpu_time_us":2080,"wal-append.run_wall_time_us":1361574}
I20260709 12:13:35.447547 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.389292 11328 rpcz_store.cc:276] 0709 12:13:32.396717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.387734 (+2991017us) service_pool.cc:224] Handling call
0709 12:13:35.387991 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.388586 (+   595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.388755 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.454249 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.454584 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.455401 11328 raft_consensus.cc:2376] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8. Re-sending same reply.
I20260709 12:13:35.456694 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.393136 11929 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:35.409117 11323 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:13:35.395998 11252 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.409886 11930 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:35.389554 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79450.5us
I20260709 12:13:35.410887 11176 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:35.388823 11079 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:13:35.459451 11148 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:13:35.464574 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:35.462373 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 139113us
I20260709 12:13:35.414867 11251 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.406006 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.245229 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.466487 11180 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:35.374678 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.206524 11295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2102) took 2906 ms. Trace:
I20260709 12:13:35.467519 11173 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.389551 14403 raft_consensus.cc:2764] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:13:35.468194 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.469772 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.457594 11328 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:35.366690 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2093) took 3069 ms. Trace:
I20260709 12:13:35.377442 11329 rpcz_store.cc:276] 0709 12:13:32.300300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.376228 (+3075928us) service_pool.cc:224] Handling call
0709 12:13:35.376462 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.376992 (+   530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.377142 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.389819 14061 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:35.417389 14445 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.410676 11253 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.418754 14548 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.472549 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.419902 11257 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:35.472798 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.437794 11175 pending_rounds.cc:85] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 2
I20260709 12:13:35.473798 11175 pending_rounds.cc:107] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:35.474097 11175 raft_consensus.cc:2889] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:35.467299 11295 rpcz_store.cc:276] 0709 12:13:32.300600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.206462 (+2905862us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:35.474282 11148 leader_election.cc:336] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:13:35.476070 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.471216 11324 rpcz_store.cc:276] 0709 12:13:32.297752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.365555 (+3067803us) service_pool.cc:224] Handling call
0709 12:13:35.365915 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.366455 (+   540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.366639 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.486593 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.487560 11326 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:35.500406 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.501991 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.453935 14389 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:35.428905 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.401424 11933 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:35.442359 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:35.462466 11331 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:35.506675 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1465) took 3056 ms. Trace:
I20260709 12:13:35.507123 11331 rpcz_store.cc:276] 0709 12:13:32.450560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.410450 (+2959890us) service_pool.cc:224] Handling call
0709 12:13:35.462135 (+ 51685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.506422 (+ 44287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.506621 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.461282 11250 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:35.388834 11322 rpcz_store.cc:269] 0709 12:13:32.598095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.169546 (+2571451us) service_pool.cc:224] Handling call
0709 12:13:35.388395 (+218849us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.507999 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.508695 11179 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:35.508922 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.509688 11322 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
W20260709 12:13:35.509954 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67683.1us
W20260709 12:13:35.511053 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 199457us
I20260709 12:13:35.405720 14567 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:35.419816 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.501458 11079 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 4, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 4 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.489357 11253 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.490309 11257 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:13:35.470847 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.511575 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.505491 11177 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:35.515884 11249 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
W20260709 12:13:35.499732 14486 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:13:35.444000 14512 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:13:35.444624 14494 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:13:35.528496 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.523826 11256 raft_consensus.cc:2368] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:35.521740 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:35.529150 11331 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:35.521880 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:35.526808 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:35.526885 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58798.4us
I20260709 12:13:35.413806 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:35.390810 11078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 258) took 16312 ms (client timeout 15000 ms). Trace:
I20260709 12:13:35.526455 11079 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 7, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.499421 14344 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:35.530767 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.532537 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.537276 11253 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:35.440503 14450 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.534839 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.538930 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.539173 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.471639 14061 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:35.539916 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 126171us
I20260709 12:13:35.540845 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:35.507647 14389 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.490985 14461 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:35.478421 14537 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:35.574297 11295 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39342 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:35.566243 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 530) took 2993 ms. Trace:
I20260709 12:13:35.490275 14445 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:35.593997 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53215.6us
I20260709 12:13:35.510563 14543 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:35.594554 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.595017 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:35.605408 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.605649 11179 raft_consensus.cc:2410] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:13:35.582574 11079 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 7, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 7 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:35.581825 14061 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:35.606433 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.607170 11182 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:13:35.607939 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002596e0 after lost signal to thread 10014
I20260709 12:13:35.607788 11079 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:35.609076 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002d47c0 after lost signal to thread 11113
I20260709 12:13:35.563038 14055 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 12:13:35.610422 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7440 after lost signal to thread 11206
W20260709 12:13:35.610730 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08003063a0 after lost signal to thread 11207
I20260709 12:13:35.595964 14537 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.611470 14537 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:35.611776 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002df4a0 after lost signal to thread 11281
I20260709 12:13:35.611672 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:35.612155 11956 debug-util.cc:398] Leaking SignalData structure 0x7b0800329840 after lost signal to thread 11282
I20260709 12:13:35.612774 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:35.525903 14427 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:35.613893 14427 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.614266 14427 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:35.528987 13989 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
W20260709 12:13:35.618387 14589 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:13:35.582388 14457 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:35.619292 14457 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.619655 14457 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:35.619865 14461 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:35.620558 14461 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:35.620698 14094 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:35.621261 14094 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:35.582005 14344 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:35.621901 11327 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:35.622388 14094 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.624060 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1467) took 3115 ms. Trace:
I20260709 12:13:35.588397 11323 rpcz_store.cc:276] 0709 12:13:32.573161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.565167 (+2992006us) service_pool.cc:224] Handling call
0709 12:13:35.565554 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.565806 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.565844 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.565901 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:35.565945 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.566013 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.566166 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.589416 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:35.624521 11327 rpcz_store.cc:276] 0709 12:13:32.508797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.441089 (+2932292us) service_pool.cc:224] Handling call
0709 12:13:35.441468 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.622380 (+180912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.623716 (+  1336us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54001c3ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:35.623977 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414392192}
I20260709 12:13:35.626830 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 533) took 3052 ms. Trace:
I20260709 12:13:35.627646 11327 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:35.627285 11323 rpcz_store.cc:276] 0709 12:13:32.574103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.626045 (+3051942us) service_pool.cc:224] Handling call
0709 12:13:35.626339 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.626500 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.626525 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.626568 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:35.626601 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.626651 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.626773 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.590207 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.628593 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2103) took 3045 ms. Trace:
I20260709 12:13:35.595352 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.590507 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.629884 11178 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.630362 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.631304 11326 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:35.592048 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:35.632241 11181 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:35.594169 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.500023 14548 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.602110 14498 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:35.541397 14539 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:35.582108 14386 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:13:35.610304 11079 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 3 to 6, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 6 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.626449 11935 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.587427 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 525) took 3261 ms. Trace:
W20260709 12:13:35.635093 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002f2d00 after lost signal to thread 11355
I20260709 12:13:35.551156 11328 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:35.629268 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 537) took 3000 ms. Trace:
I20260709 12:13:35.636523 11927 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:13:35.643244 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.643565 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.644241 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.644878 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.645517 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:35.531607 11078 rpcz_store.cc:269] 0709 12:13:19.122760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.139517 (+ 16757us) service_pool.cc:224] Handling call
0709 12:13:35.390741 (+16251224us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:30.790120 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:31.030205 (+240085us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:31.043041 (+ 12836us) write_op.cc:435] Acquired schema lock
0709 12:13:31.043070 (+    29us) tablet.cc:662] Decoding operations
0709 12:13:31.044480 (+  1410us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:31.044610 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:31.044642 (+    32us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:13:31.147054 (+102412us) tablet.cc:701] Row locks acquired
0709 12:13:31.147113 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:13:31.147354 (+   241us) write_op.cc:270] Start()
0709 12:13:31.147520 (+   166us) write_op.cc:276] Timestamp: P: 1783599211147274 usec, L: 0
0709 12:13:31.147554 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:13:31.148386 (+   832us) log.cc:844] Serialized 5211 byte log entry
0709 12:13:31.632995 (+484609us) op_driver.cc:464] REPLICATION: finished
0709 12:13:31.647817 (+ 14822us) write_op.cc:301] APPLY: starting
0709 12:13:31.647931 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:31.648327 (+   396us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:31.648490 (+   163us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:31.664795 (+ 16305us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:31.665337 (+   542us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 12:13:31.665416 (+    79us) write_op.cc:312] APPLY: finished
0709 12:13:31.677775 (+ 12359us) log.cc:844] Serialized 89 byte log entry
0709 12:13:31.678904 (+  1129us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:31.679119 (+   215us) write_op.cc:454] Released schema lock
0709 12:13:31.679215 (+    96us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:31.745518 (+ 66303us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:31.749042 (+  3524us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1528641,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":14456,"apply.run_cpu_time_us":8881,"apply.run_wall_time_us":55356,"num_ops":12,"prepare.queue_time_us":120004,"prepare.run_cpu_time_us":14963,"prepare.run_wall_time_us":407265,"raft.queue_time_us":413365,"raft.run_cpu_time_us":1334,"raft.run_wall_time_us":19788,"replication_time_us":482717,"thread_start_us":419051,"threads_started":4,"wal-append.queue_time_us":31261,"wal-append.run_cpu_time_us":14555,"wal-append.run_wall_time_us":1498995}]]}
I20260709 12:13:35.674307 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.673336 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.675886 11932 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.677392 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1469) took 3105 ms. Trace:
I20260709 12:13:35.677934 11328 rpcz_store.cc:276] 0709 12:13:32.572304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.550542 (+2978238us) service_pool.cc:224] Handling call
0709 12:13:35.550908 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.677080 (+126172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.677318 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.679011 11323 rpcz_store.cc:276] 0709 12:13:32.629193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.628574 (+2999381us) service_pool.cc:224] Handling call
0709 12:13:35.628857 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.629002 (+   145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.629023 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.629060 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:35.629088 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.629134 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.629223 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:35.681488 11956 debug-util.cc:398] Leaking SignalData structure 0x7b080013a1a0 after lost signal to thread 11959
W20260709 12:13:35.681919 11956 debug-util.cc:398] Leaking SignalData structure 0x7b0800304240 after lost signal to thread 11960
I20260709 12:13:35.582922 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:35.684134 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 101298us
I20260709 12:13:35.685302 11252 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.629066 11327 rpcz_store.cc:276] 0709 12:13:32.583291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.626934 (+3043643us) service_pool.cc:224] Handling call
0709 12:13:35.627256 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.628300 (+  1044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.628530 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.688061 11331 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:35.688064 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.688827 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 538) took 3041 ms. Trace:
I20260709 12:13:35.689677 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2091) took 3437 ms. Trace:
I20260709 12:13:35.689894 11331 rpcz_store.cc:276] 0709 12:13:32.647631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.633495 (+2985864us) service_pool.cc:224] Handling call
0709 12:13:35.687770 (+ 54275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.688589 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.688776 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.685721 11928 rpcz_store.cc:276] 0709 12:13:32.325724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.325906 (+   182us) service_pool.cc:224] Handling call
0709 12:13:32.326314 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.740323 (+414009us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:13:35.169748 (+2429425us) log.cc:844] Serialized 14 byte log entry
0709 12:13:35.439152 (+269404us) log.cc:844] Serialized 14 byte log entry
0709 12:13:35.440142 (+   990us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.440203 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:35.527404 (+ 87201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.587017 (+ 59613us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5400555354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:35.587098 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.587334 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":413767,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":139246,"raft.queue_time_us":15006,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":5862,"spinlock_wait_cycles":951379712,"thread_start_us":2684984,"threads_started":3,"wal-append.queue_time_us":145830}
I20260709 12:13:35.583632 11173 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:35.703056 11900 leader_election.cc:400] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:35.703586 11900 leader_election.cc:403] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 12:13:35.706324 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.706910 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.707090 11250 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:13:35.590655 14389 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:35.711972 11323 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:35.712342 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.712649 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2106) took 2930 ms. Trace:
I20260709 12:13:35.713274 11328 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:35.714272 11253 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:35.714577 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.715408 11932 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:35.717265 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 539) took 3041 ms. Trace:
I20260709 12:13:35.717794 11326 rpcz_store.cc:276] 0709 12:13:32.675838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.641849 (+2966011us) service_pool.cc:224] Handling call
0709 12:13:35.716665 (+ 74816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.716863 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.716900 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.716960 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:35.717008 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.717077 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.717197 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.720544 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.707095 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.709079 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.713022 11323 rpcz_store.cc:276] 0709 12:13:32.782186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.679894 (+2897708us) service_pool.cc:224] Handling call
0709 12:13:35.711792 (+ 31898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.712416 (+   624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.712596 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.722110 11249 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 4: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:35.711303 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:35.723084 11933 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:35.723874 11079 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.727453 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2104) took 2946 ms. Trace:
I20260709 12:13:35.727998 11328 rpcz_store.cc:276] 0709 12:13:32.781436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.678939 (+2897503us) service_pool.cc:224] Handling call
0709 12:13:35.712874 (+ 33935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.713995 (+  1121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.727384 (+ 13389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.729262 11256 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:35.524345 14576 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.165s
I20260709 12:13:35.723232 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:35.730528 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.734009 11079 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 1 to 5, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 5 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.727510 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.642710 14228 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:35.587884 11295 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39342 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2109)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1471)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2115)
I20260709 12:13:35.651051 14539 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:35.738655 11296 connection.cc:256] RPC call timeout handler was delayed by -3.8168379530000038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
I20260709 12:13:35.740482 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.709404 14422 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:35.739283 14228 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.743094 14408 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.743237 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.749150 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.672987 14246 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:13:35.751196 11933 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.730131 14467 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:35.707407 14498 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:35.750674 11251 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.742846 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:35.753034 11079 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:35.759104 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.762677 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.764068 11253 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:35.745090 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:35.744071 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.742347 14539 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:35.739796 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 218011us
I20260709 12:13:35.744374 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.733938 14567 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.781198 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:35.759687 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.751940 14228 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:35.622426 14344 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.643967 14461 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.758067 14422 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:35.798703 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.788568 11928 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:35.751727 14508 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:35.786048 14519 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:35.789611 14300 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:35.789642 14494 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:13:35.690055 11330 rpcz_store.cc:276] 0709 12:13:32.251790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.357357 (+3105567us) service_pool.cc:224] Handling call
0709 12:13:35.385716 (+ 28359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.385968 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:35.522714 (+136746us) log.cc:844] Serialized 14 byte log entry
0709 12:13:35.688973 (+166259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.689057 (+    84us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:35.689099 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.689477 (+   378us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.689601 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"thread_start_us":302236,"threads_started":2}
I20260709 12:13:35.812618 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.795161 11079 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:35.743824 14513 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:35.755004 14408 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.767325 14491 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:35.806421 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.810468 11323 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:35.825301 11253 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.825599 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2110) took 3036 ms. Trace:
I20260709 12:13:35.826048 11323 rpcz_store.cc:276] 0709 12:13:32.788756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.809859 (+3021103us) service_pool.cc:224] Handling call
0709 12:13:35.810154 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.825306 (+ 15152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.825539 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.827124 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.813208 11330 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:35.828603 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.818378 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.813928 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:35.802167 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.831184 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.827737 11323 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:35.833117 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.834213 11326 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:35.803917 14464 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.806874 14519 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.835618 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.831702 11079 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 4, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 4 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.836478 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:35.837085 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.837793 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.838002 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.808092 11251 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.814993 14300 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.790772 14567 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.844211 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:35.846629 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.859697 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:35.808813 14543 raft_consensus.cc:2764] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:35.866848 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.867771 11323 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:13:35.817159 14513 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.861285 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.846220 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:35.844468 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:35.875805 11933 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
W20260709 12:13:35.875860 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 109287us
I20260709 12:13:35.860379 14300 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:35.871417 11330 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:35.844200 14519 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:35.841087 11079 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 7, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:35.859935 14505 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:35.883672 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.839689 14582 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:35.888586 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:35.889444 11251 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:35.846200 14491 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:35.888624 11079 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:35.889921 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:35.878824 14513 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:35.894743 11079 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 4 to 7, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 7 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:35.898066 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:35.901141 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:35.831971 14596 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
W20260709 12:13:35.908196 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77856.2us
I20260709 12:13:35.905165 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.858484 14464 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.889272 14582 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.918017 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:35.910466 11326 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 12:13:35.911428 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 168662us
W20260709 12:13:35.920642 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.915956 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:35.887751 14505 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:35.892714 14552 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:35.923609 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:35.981901 11323 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:35.982560 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.983368 11326 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:35.896540 14486 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:35.868779 14498 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:35.960882 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2107) took 3173 ms. Trace:
I20260709 12:13:35.890941 14386 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:13:35.896888 14491 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:35.967389 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 89121.5us
I20260709 12:13:35.907196 14460 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:35.902637 14598 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.904853 14494 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:35.922031 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:36.033385 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 111438us
I20260709 12:13:35.972240 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:36.035066 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:35.977947 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 131826us
I20260709 12:13:36.035684 11323 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:36.036170 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:35.883530 14427 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:35.925998 11079 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 7, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.958503 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:36.037278 11251 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.987426 11327 rpcz_store.cc:276] 0709 12:13:32.785822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.686135 (+2900313us) service_pool.cc:224] Handling call
0709 12:13:35.686438 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.686643 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:35.868078 (+181435us) log.cc:844] Serialized 14 byte log entry
0709 12:13:35.959008 (+ 90930us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:35.959094 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:35.959165 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.959265 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.959435 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2929,"prepare.queue_time_us":285471,"raft.queue_time_us":1082,"raft.run_wall_time_us":8733,"spinlock_wait_cycles":29166464,"thread_start_us":227509,"threads_started":2}
I20260709 12:13:36.038587 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.039158 11327 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10. Re-sending same reply.
I20260709 12:13:36.040045 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.040565 11327 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:36.034133 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:36.041035 14604 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:35.925017 14582 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:36.041502 14604 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.041989 14604 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:36.042606 14594 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:13:36.045499 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.039167 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.046926 11326 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:35.997454 11935 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:35.899315 14599 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 12:13:35.884516 14422 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.041414 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.972826 14548 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:35.989284 14491 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:35.987571 14486 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:36.108884 11327 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 5: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:35.981155 14552 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:13:36.119201 14408 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:36.104346 14592 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.306s	user 0.001s	sys 0.001s
I20260709 12:13:36.120007 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:36.122879 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 221828us
I20260709 12:13:36.123355 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:36.182267 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 148216us
I20260709 12:13:36.188282 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:35.998412 14460 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:36.152719 11257 raft_consensus.cc:2368] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9.
I20260709 12:13:36.191226 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.206526 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:36.219492 11323 raft_consensus.cc:2410] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:36.222537 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.209836 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:36.233181 11323 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 3.
W20260709 12:13:36.161196 14547 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:13:36.218068 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:36.033669 14505 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.162106 14494 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:36.138664 14486 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:36.129266 14450 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.216584 14460 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.246425 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:36.249128 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:36.313146 11327 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:36.259306 14601 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.277s	user 0.000s	sys 0.000s
I20260709 12:13:36.038661 14598 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.238518 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.128698 14539 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:36.259300 14600 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:13:36.255353 14494 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.431640 14494 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:36.432032 14539 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.253954 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2108) took 3467 ms. Trace:
I20260709 12:13:36.181267 14228 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.210929 14389 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:36.314962 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 403187us
I20260709 12:13:36.216316 14408 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:36.316926 11326 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.320353 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 529) took 3747 ms. Trace:
I20260709 12:13:36.438882 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.270437 13989 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:36.308389 14593 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.406s	user 0.001s	sys 0.000s
I20260709 12:13:36.302881 14618 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:13:36.340443 14613 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:13:36.339454 14427 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:36.353109 14457 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.733s	user 0.004s	sys 0.004s
I20260709 12:13:36.356338 14300 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.359596 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 625) took 1046 ms. Trace:
W20260709 12:13:36.359855 14061 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.753s	user 0.008s	sys 0.000s
W20260709 12:13:36.363267 14537 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.751s	user 0.009s	sys 0.000s
I20260709 12:13:36.363620 14621 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:36.320443 14386 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:36.365860 14464 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.451681 14621 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:36.452795 14621 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.453708 14537 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:36.368125 14610 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:13:36.371117 14598 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.372879 14543 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:36.335249 14344 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:36.381757 14567 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:36.384397 14513 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.458453 14061 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.460258 14513 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:36.389695 14605 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:13:36.400199 14519 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.005s
I20260709 12:13:36.378868 14604 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.404548 14491 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.440790 14408 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:36.378978 14582 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.407835 14486 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.418197 14460 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:36.423125 14055 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.075s	sys 0.089s
I20260709 12:13:36.443532 14614 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:36.446465 11927 rpcz_store.cc:276] 0709 12:13:35.313451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.314115 (+   664us) service_pool.cc:224] Handling call
0709 12:13:35.314429 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.680053 (+365624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.359305 (+679252us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54005afd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:36.359507 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721106304}
I20260709 12:13:36.457058 14543 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.475760 14486 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.472406 14582 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.471382 14445 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:36.457021 14537 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:36.463987 14505 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.465776 14519 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.467993 14604 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.499111 14543 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.503738 14519 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.539177 11174 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: [7.5-9.6]
W20260709 12:13:36.539579 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 493950us
I20260709 12:13:36.541271 11175 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:36.543803 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.547955 11177 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
W20260709 12:13:36.318109 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 108772us
I20260709 12:13:36.360383 11256 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:36.571573 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 348264us
I20260709 12:13:36.573029 14300 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.461668 14457 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.575732 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.443353 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:36.578541 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 410) took 1071 ms. Trace:
W20260709 12:13:36.578752 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 135421us
I20260709 12:13:36.580133 11901 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:36.583475 11935 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:36.439081 11322 rpcz_store.cc:276] 0709 12:13:32.572890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.562950 (+2990060us) service_pool.cc:224] Handling call
0709 12:13:35.563351 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.563612 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:35.771526 (+207914us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.319846 (+548320us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.320004 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:36.320054 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:36.320104 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.320174 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.320295 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13220,"prepare.queue_time_us":342358,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":6378,"raft.queue_time_us":165253,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":2761,"thread_start_us":755033,"threads_started":3,"wal-append.queue_time_us":482893}
I20260709 12:13:36.590158 11329 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:36.591298 14539 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:36.591921 14539 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:36.592422 14539 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:36.592329 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.371738 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 531) took 3798 ms. Trace:
I20260709 12:13:36.398245 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.594092 11324 rpcz_store.cc:276] 0709 12:13:32.573407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.577559 (+3004152us) service_pool.cc:224] Handling call
0709 12:13:35.639759 (+ 62200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.640232 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.640275 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.640851 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 12:13:35.826410 (+185559us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:35.826469 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.826573 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:36.371504 (+544931us) raft_consensus.cc:1685] finished
0709 12:13:36.371524 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.371673 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1311,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":39065,"thread_start_us":96243,"threads_started":1,"wal-append.queue_time_us":198299}
I20260709 12:13:36.395107 11328 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:36.596585 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:36.433185 11331 rpcz_store.cc:276] 0709 12:13:32.786060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.690794 (+2904734us) service_pool.cc:224] Handling call
0709 12:13:35.691091 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.691332 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:35.871689 (+180357us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.253504 (+381815us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:36.253585 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:36.253664 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.253742 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.253881 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38814,"prepare.queue_time_us":295339,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":6407,"raft.queue_time_us":206540,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"spinlock_wait_cycles":1352448,"thread_start_us":509028,"threads_started":3,"wal-append.queue_time_us":387199}
I20260709 12:13:36.416533 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.598423 11249 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:36.598742 11330 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:36.599567 14486 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:36.600517 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:36.465875 14389 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.470247 14491 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.469998 14408 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.602198 14582 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:36.603353 14491 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:36.603768 14582 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:36.604261 14582 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:36.461597 14061 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.500092 14505 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.463301 14228 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.607402 14228 raft_consensus.cc:2764] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:13:36.597190 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2109) took 3808 ms. Trace:
I20260709 12:13:36.571672 14577 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:36.423877 11323 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:36.607901 11328 rpcz_store.cc:276] 0709 12:13:32.788471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.394292 (+3605821us) service_pool.cc:224] Handling call
0709 12:13:36.394710 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:36.596449 (+201739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.596976 (+   527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:36.609409 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.609711 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.610702 11323 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:36.611719 11930 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 9. Ops: [7.7-7.7]
I20260709 12:13:36.611944 11328 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:36.612766 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2115) took 1410 ms. Trace:
I20260709 12:13:36.587761 14422 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:13:36.589735 11220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.007s	sys 0.010s
I20260709 12:13:36.617807 11328 rpcz_store.cc:276] 0709 12:13:35.202433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.610792 (+1408359us) service_pool.cc:224] Handling call
0709 12:13:36.611360 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:36.612447 (+  1087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.612686 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:36.618427 11220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.007s	sys 0.010s
I20260709 12:13:36.597173 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.601569 11330 raft_consensus.cc:2376] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6. Re-sending same reply.
I20260709 12:13:36.595739 11327 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:36.598596 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.602231 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1683) took 1294 ms. Trace:
I20260709 12:13:36.585891 14614 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:36.579272 11255 rpcz_store.cc:276] 0709 12:13:35.507021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.507165 (+   144us) service_pool.cc:224] Handling call
0709 12:13:35.507494 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.525341 (+ 17847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.525372 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.562344 (+ 36972us) log.cc:844] Serialized 26 byte log entry
0709 12:13:35.820497 (+258153us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:35.820553 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.820631 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:36.577781 (+757150us) raft_consensus.cc:1685] finished
0709 12:13:36.577834 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.578221 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17489,"raft.queue_time_us":35640,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":26774,"thread_start_us":146160,"threads_started":1,"wal-append.queue_time_us":280757}
I20260709 12:13:36.614359 11323 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:36.615854 14445 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:36.625162 14614 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:36.627027 14614 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.622687 11254 rpcz_store.cc:276] 0709 12:13:35.307975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.308086 (+   111us) service_pool.cc:224] Handling call
0709 12:13:35.308400 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.314003 (+  5603us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:35.440057 (+126054us) log.cc:844] Serialized 14 byte log entry
0709 12:13:35.761467 (+321410us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.777059 (+ 15592us) log.cc:844] Serialized 26 byte log entry
0709 12:13:35.777231 (+   172us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:35.777278 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.777370 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:36.586246 (+808876us) raft_consensus.cc:1685] finished
0709 12:13:36.586301 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.601991 (+ 15690us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12047,"prepare.queue_time_us":214662,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":6027,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":18719,"thread_start_us":398034,"threads_started":2,"wal-append.queue_time_us":413064}
I20260709 12:13:36.630053 11331 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:36.557402 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:36.484601 14055 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.076s	sys 0.091s
I20260709 12:13:36.594933 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.585266 14460 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.632458 11322 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:36.639258 14408 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.653990 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.642537 14467 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:36.656176 14408 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:36.643090 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48448.2us
I20260709 12:13:36.650424 14094 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:13:36.651932 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 94753.2us
W20260709 12:13:36.642964 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.013s	sys 0.018s
W20260709 12:13:36.661561 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.013s	sys 0.018s
I20260709 12:13:36.662592 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.664157 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:36.666944 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:36.668596 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.12879e+06us
I20260709 12:13:36.666608 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:36.671811 11901 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:36.674763 14552 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:36.675045 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:36.675436 14552 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.676486 14552 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.676782 14445 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:36.677222 14450 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:36.679503 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1458) took 4532 ms. Trace:
I20260709 12:13:36.679819 14450 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.661011 14582 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.680433 14450 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:36.683893 14582 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.680048 11325 rpcz_store.cc:276] 0709 12:13:32.147122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.347081 (+3199959us) service_pool.cc:224] Handling call
0709 12:13:35.347428 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.508112 (+160684us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:35.572149 (+ 64037us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b1800282820. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:13:35.888086 (+315937us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.423942 (+535856us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.433740 (+  9798us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:36.437097 (+  3357us) log.cc:844] Serialized 26 byte log entry
0709 12:13:36.437387 (+   290us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:36.437815 (+   428us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.441944 (+  4129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.647211 (+205267us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b1800282820. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:36.676274 (+ 29063us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b1800282820. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419
0709 12:13:36.677519 (+  1245us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001c43d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:36.677571 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:36.678920 (+  1349us) raft_consensus.cc:1685] finished
0709 12:13:36.678946 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.679273 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25895,"prepare.queue_time_us":379195,"prepare.run_cpu_time_us":157,"prepare.run_wall_time_us":161,"raft.queue_time_us":43675,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":12575,"spinlock_wait_cycles":355875456,"thread_start_us":795337,"threads_started":2,"wal-append.queue_time_us":517524}
W20260709 12:13:36.678297 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:36.692958 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 593) took 1159 ms. Trace:
I20260709 12:13:36.693647 11930 rpcz_store.cc:276] 0709 12:13:35.533190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.533340 (+   150us) service_pool.cc:224] Handling call
0709 12:13:35.580879 (+ 47539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:36.612299 (+1031420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.692408 (+ 80109us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400673ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:36.692840 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2313320192}
I20260709 12:13:36.696283 14494 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.698257 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2124) took 1248 ms. Trace:
I20260709 12:13:36.698784 11329 rpcz_store.cc:276] 0709 12:13:35.450314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.500067 (+ 49753us) service_pool.cc:224] Handling call
0709 12:13:36.697975 (+1197908us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001c5cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:36.698149 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2451024128}
I20260709 12:13:36.701988 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:36.702240 14577 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:36.702997 14577 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:36.690008 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59561.8us
I20260709 12:13:36.705859 14445 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:36.707844 11255 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:13:36.708765 14445 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:36.708961 11257 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:13:36.709288 14445 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:36.706216 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:36.710758 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:36.711637 11254 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:13:36.711908 11253 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:36.713131 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:36.710732 14445 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:36.713881 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43482.2us
I20260709 12:13:36.714895 14445 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:36.714942 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.720992 11174 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:36.723356 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.724267 11174 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:36.707864 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.726675 14621 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:36.727967 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.737784 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:36.738116 14621 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:36.739331 14621 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:36.730700 14568 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:36.747057 14568 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:36.747054 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.747803 14568 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:36.748845 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.749195 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.749702 11173 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:36.724213 14494 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.756945 14386 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:36.758036 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:36.758775 14457 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.759367 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:36.763017 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47004.3us
W20260709 12:13:36.760110 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 85311.2us
I20260709 12:13:36.766101 11253 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:36.767726 11935 raft_consensus.cc:2376] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9. Re-sending same reply.
I20260709 12:13:36.742408 14450 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.780817 14450 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:36.781741 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.781514 14386 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.783134 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.784108 11173 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:36.784626 14386 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.794435 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:36.795670 11254 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:36.797861 11174 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:36.800595 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.813990 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.800395 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:36.859189 11178 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:36.859647 11930 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:13:36.859496 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.862231 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.863056 11173 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.868080 14568 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.870283 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.870770 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.871646 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.872180 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.872933 11174 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:36.873476 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:36.874244 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.875528 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.876816 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.877579 11329 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:13:36.880857 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 130338us
I20260709 12:13:36.881815 11181 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:36.883288 11177 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:36.883963 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 86349.8us
I20260709 12:13:36.886394 11176 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.886552 14443 raft_consensus.cc:2764] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:36.886847 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:36.887674 11178 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:36.887969 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 146999us
I20260709 12:13:36.888522 11180 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:36.888854 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:36.889652 14599 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:36.890786 14408 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:36.891250 14408 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:36.885262 14450 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:36.891798 14408 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.892194 14450 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.891165 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:36.898290 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:36.899176 11331 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:36.900298 11174 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:36.904024 14450 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:36.907455 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:36.908859 11930 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:36.909765 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:36.911662 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:36.913946 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:36.926080 11176 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:36.930261 11173 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:36.885655 14539 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.954669 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2105) took 1083 ms. Trace:
I20260709 12:13:36.953529 14386 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:36.962059 14443 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:36.963590 14443 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:36.963968 14443 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:36.967172 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:36.961005 11326 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.005s
I20260709 12:13:36.969594 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:36.969976 14644 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:13:36.970772 14543 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:36.971233 14543 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.965013 14443 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.930255 14408 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.975407 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:36.976737 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 104052us
I20260709 12:13:36.978472 11173 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:36.980650 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:36.967245 14386 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:36.982412 14539 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:36.983350 11176 raft_consensus.cc:2376] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8. Re-sending same reply.
I20260709 12:13:36.983132 14386 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:36.979202 11326 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:36.986364 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 556) took 1209 ms. Trace:
I20260709 12:13:36.986950 14443 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:36.988677 14408 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:36.976230 14543 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:36.991328 11296 leader_election.cc:400] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9.
I20260709 12:13:36.991757 11296 leader_election.cc:403] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:36.997030 14628 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:13:36.997721 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:36.999497 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:37.000735 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:37.000988 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:37.002307 11177 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:37.003134 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:37.004313 11178 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:37.004796 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:37.004954 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:37.008795 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:37.010092 11176 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:37.010248 14408 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:37.010947 14408 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.011539 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:37.002439 11182 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:37.008816 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.015117 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:37.015944 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:37.017119 11182 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:36.986872 11326 rpcz_store.cc:276] 0709 12:13:35.776645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.182277 (+405632us) service_pool.cc:224] Handling call
0709 12:13:36.986239 (+803962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:37.017869 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:37.015799 14094 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:37.015892 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:37.023224 14094 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:37.023545 11173 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:37.023867 14094 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:37.026381 14486 raft_consensus.cc:2764] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:13:37.027211 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:37.028117 11176 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:37.035059 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:37.036231 11933 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:37.039122 11927 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:13:37.002324 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:37.041353 11174 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.041559 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:37.042833 11927 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:13:37.044533 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:37.045817 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:37.046545 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:37.048238 14450 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:37.048619 14457 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:37.049182 14450 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:37.050107 14614 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:37.050540 14614 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:37.050860 14645 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.165s	user 0.004s	sys 0.000s
I20260709 12:13:37.050405 14450 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.051755 14614 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.018589 14539 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:37.053722 14539 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:37.054203 14539 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:37.015328 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.058121 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:37.018265 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:37.019392 14552 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:36.977568 14445 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:37.059718 11253 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:37.059836 14445 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:37.060905 11933 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:13:37.061101 14445 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.034847 14577 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.058316 14568 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:37.065299 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92994.5us
I20260709 12:13:37.073674 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.076130 11174 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:37.072750 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:37.077561 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:37.081137 14577 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:37.081188 14408 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:37.083663 14300 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:37.077270 14621 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:37.084224 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:37.085057 14621 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.085997 14300 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:37.086511 14460 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:37.087668 14621 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:37.088516 14464 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:37.088967 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:37.090189 11932 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:37.090042 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:37.090698 14460 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:37.089216 14614 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:37.091079 14460 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.955120 11175 rpcz_store.cc:276] 0709 12:13:35.871148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.542335 (+671187us) service_pool.cc:224] Handling call
0709 12:13:36.542868 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:36.543526 (+   658us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:36.735904 (+192378us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.928216 (+192312us) log.cc:844] Serialized 14 byte log entry
0709 12:13:36.952119 (+ 23903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:36.952178 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:36.954428 (+  2250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.954494 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.954614 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7191,"prepare.queue_time_us":381332,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":9364,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3999,"spinlock_wait_cycles":41216,"thread_start_us":307406,"threads_started":2}
I20260709 12:13:37.107522 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.130970 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.131943 11330 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:37.134024 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:37.134514 11254 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:37.134639 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:37.080319 14552 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:37.136112 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.114291 14460 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.128733 14628 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9.
I20260709 12:13:37.138824 11325 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:37.137370 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.141805 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:37.143281 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:37.143427 14457 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:37.144542 14443 raft_consensus.cc:2764] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:37.145915 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:37.146880 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:37.147358 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69912.5us
I20260709 12:13:37.148460 14443 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:37.148803 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 121105us
I20260709 12:13:37.148957 14443 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:37.149446 14443 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:37.149694 14608 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:37.149938 14548 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:37.150099 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:37.150987 14608 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:37.151067 14582 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:37.151542 14608 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:37.151769 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:37.152274 11327 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:13:37.152715 11327 pending_rounds.cc:85] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:13:37.153112 11327 pending_rounds.cc:107] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:13:37.153455 11327 raft_consensus.cc:2889] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:37.152762 14608 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.153945 14582 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:37.145990 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:37.154568 14608 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:37.154798 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.155498 11329 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.160365 14443 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.160967 11325 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:37.153208 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:37.146633 14543 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.091243 14389 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:37.168166 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:37.168967 14582 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:37.176030 14582 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:37.176203 14543 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:37.176584 14582 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:37.169447 14611 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:37.170811 14246 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:37.182516 14246 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:37.184064 14486 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:37.183890 14246 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.184674 14486 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:37.185151 14486 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.165442 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.185993 11326 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.185905 14443 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:37.186012 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71310.3us
I20260709 12:13:37.188596 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:37.174214 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:37.088967 14464 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:37.186218 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78831.8us
I20260709 12:13:37.192577 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:37.185415 14246 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.202435 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:37.189946 14568 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:37.209308 11330 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:37.206784 14443 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:13:37.211946 14464 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.216631 11326 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:37.175127 14389 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:37.229732 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.216814 14543 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.213598 14464 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:37.268327 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74095us
I20260709 12:13:37.239737 14628 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:37.251717 14467 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:37.249388 14577 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:37.296373 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.217326 14460 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.239569 14548 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:13:37.265339 14621 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.259788 14539 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.304607 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:37.303597 14386 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:13:37.314220 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 124635us
I20260709 12:13:37.311092 14408 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:37.304922 14467 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:37.298309 14543 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.361399 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:37.323146 14300 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:13:37.370842 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 141284us
I20260709 12:13:37.327600 14552 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:37.324887 14445 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.314855 14628 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:37.385371 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 188668us
W20260709 12:13:37.400781 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.992s (SENT) [suppressed 26 similar messages]
I20260709 12:13:37.401283 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.402650 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:37.403900 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 229788us
I20260709 12:13:37.404037 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.404799 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:37.419088 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 81907us
I20260709 12:13:37.376940 14300 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:37.430327 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:37.437955 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44515.8us
I20260709 12:13:37.364892 14621 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.420599 14408 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:37.449419 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 256933us
I20260709 12:13:37.432657 14300 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.437649 14599 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:37.464823 14664 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:37.468428 14577 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:37.470525 14655 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:13:37.472124 14552 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:37.478456 14659 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:13:37.478997 14467 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.479851 14443 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:37.480595 14582 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.481546 14389 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.482378 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:37.450347 14094 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.488736 14246 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:37.437554 14386 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:37.489913 14055 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:37.490665 14543 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:37.491405 14460 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:37.551112 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.493666 11174 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:37.494410 14654 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:13:37.494822 14628 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:37.457116 14598 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:13:37.496402 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.497059 14665 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:37.500027 14621 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:37.457994 14651 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:13:37.453907 14445 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.501219 14408 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:37.502537 11329 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:37.502738 14672 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:37.504536 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:37.505378 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:37.506294 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.506780 11927 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:37.507668 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:37.510677 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 109899us
I20260709 12:13:37.519028 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.526175 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.528239 11179 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:13:37.564824 14228 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:37.515498 14577 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.543907 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.572541 14389 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:37.589515 11935 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:13:37.524684 14552 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.551615 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.519716 14599 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:37.554548 11180 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:37.492422 14539 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:37.593070 11930 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:37.564949 11327 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.570113 14386 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.534035 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.599656 11326 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:37.564271 14443 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:37.605768 14650 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.117s	user 0.003s	sys 0.000s
I20260709 12:13:37.606528 14443 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.584268 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:37.545378 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.612255 11323 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:37.581245 14598 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:37.615468 14598 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.618659 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.619719 11326 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:37.620918 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.558909 14486 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:37.621794 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 217861us
I20260709 12:13:37.622301 14582 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:37.624152 11325 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:37.624555 14094 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:37.625736 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.626186 11324 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:37.626631 11325 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:37.628194 11324 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:13:37.629623 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 147354us
I20260709 12:13:37.632000 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:37.595667 14445 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:37.595538 14228 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:37.635321 14599 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.635013 14228 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.582684 11330 raft_consensus.cc:2376] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10. Re-sending same reply.
I20260709 12:13:37.547111 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.578557 14408 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:37.595975 11322 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.650014 11326 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:37.654191 11177 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:13:37.659439 14486 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:37.662447 11180 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:37.664927 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.672986 11935 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:37.674861 11331 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:37.646178 14408 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:37.677171 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 135134us
I20260709 12:13:37.664310 14548 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.692178 11325 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:37.698594 11324 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:13:37.698719 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76007us
I20260709 12:13:37.642192 14300 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.707734 11326 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:37.676143 14055 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:37.709558 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:37.679955 14494 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:37.706306 14548 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:37.727185 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:37.745807 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 113989us
I20260709 12:13:37.746174 11325 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:13:37.778823 11326 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:37.775090 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:37.735486 14246 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:37.736260 14055 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:37.726912 14614 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:13:37.793566 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.796031 11177 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:37.797593 11322 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:37.838215 11900 leader_election.cc:336] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.043s (SENT)
I20260709 12:13:37.839241 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.840340 11322 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:37.843001 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68037.1us
I20260709 12:13:37.790143 14621 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:37.858448 14548 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:37.862490 14519 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 12:13:37.863369 14670 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 12:13:37.900012 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:37.901003 11330 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:37.897472 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:37.899645 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 190216us
I20260709 12:13:37.903626 11323 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
W20260709 12:13:37.866166 14464 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.007s
I20260709 12:13:37.906083 11327 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:37.861155 14577 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:37.913108 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:37.915057 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:37.916795 14678 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:13:37.916473 14055 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:37.911109 14548 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 12:13:37.918038 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 101463us
I20260709 12:13:37.918849 14671 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:13:37.914561 14389 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:13:37.932549 11322 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:37.933735 14464 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:37.924578 14621 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:37.940424 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.946666 11180 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
W20260709 12:13:37.948707 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 518490us
I20260709 12:13:37.952726 14614 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.951398 14464 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:37.963075 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 65787.7us
I20260709 12:13:37.964628 11330 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:37.954555 14300 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:37.955426 14494 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:37.974131 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:37.977460 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.972075 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:37.980217 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:37.961278 14389 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:13:38.012960 11176 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:13:38.015338 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 118965us
W20260709 12:13:38.018496 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.018522 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:38.017678 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:38.021231 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50549.1us
I20260709 12:13:38.022209 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:37.990057 14494 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:37.967882 14614 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.018472 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:38.000844 14621 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:38.011158 14300 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:37.984594 14599 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:13:38.028162 14494 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.039266 14673 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:38.010576 14464 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.052285 14386 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:38.052803 14386 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:38.057039 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.032495 14445 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:38.063580 14690 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:38.057832 14665 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:13:38.058014 14582 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:13:38.058566 14494 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:38.065249 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:38.065191 14460 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:38.077428 14690 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:38.078003 14582 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:38.078275 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:38.078563 14582 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:38.078619 14690 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.079452 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:38.066354 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44287.9us
I20260709 12:13:38.070549 14577 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:38.070463 14386 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.074617 14665 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:38.081349 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:38.079663 14582 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.084630 14677 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:38.085462 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.085688 14443 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:38.085193 14539 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:38.086994 14445 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:38.088027 14228 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:38.088447 14228 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:38.089494 14676 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:38.091094 14668 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.091709 14676 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:38.092487 14676 consensus_queue.cc:1243] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:38.092751 14300 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:38.052362 14464 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:38.093791 14300 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.080837 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:38.091660 14668 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:38.099855 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 118297us
I20260709 12:13:38.057592 14513 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:13:38.084699 14665 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:38.105321 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.093817 14614 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:38.096897 14677 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:38.112389 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.105845 14464 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:13:38.124723 14445 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:38.125618 14450 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:38.126139 14450 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:38.126171 14246 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:38.126732 14246 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:38.127297 14450 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.127708 14246 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.128831 14450 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:38.129114 14246 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:38.130044 14665 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.112440 14598 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:38.146958 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:38.101229 14552 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.117923 14582 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:38.108139 14673 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
W20260709 12:13:38.150784 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 76743us
I20260709 12:13:38.156625 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:38.089534 14228 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.148417 14676 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:13:38.170851 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 90088.2us
I20260709 12:13:38.171927 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:38.175475 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68782.9us
I20260709 12:13:38.152742 14621 raft_consensus.cc:2764] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:38.127297 14408 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.184741 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:38.186206 14539 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:38.191340 14539 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:38.201695 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70624.3us
I20260709 12:13:38.166342 14386 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:38.127702 14699 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:13:38.184423 14665 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:38.201529 14621 raft_consensus.cc:2764] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:13:38.202580 14443 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:38.234634 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.239696 11331 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:13:38.244565 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 85537.4us
I20260709 12:13:38.205008 14676 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:38.241184 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53352.2us
I20260709 12:13:38.250473 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:38.257077 14228 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:38.257498 14464 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.258253 14464 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.260867 14464 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:38.265749 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 109257us
I20260709 12:13:38.281687 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:38.294572 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 176264us
W20260709 12:13:38.307191 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 197957us
I20260709 12:13:38.309715 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.262004 14702 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.281986 14539 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:38.327301 14702 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.329926 14707 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.330502 14568 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:38.204788 14677 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.284118 14703 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:38.351388 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:38.334302 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:38.333825 14568 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:38.369676 11173 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:38.370417 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:38.372035 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62546.9us
I20260709 12:13:38.371803 14568 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:38.373200 14682 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:13:38.373473 14568 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:38.374701 14706 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:13:38.377895 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 96341.2us
I20260709 12:13:38.347506 14621 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:38.372994 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.383185 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:38.384608 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52803.6us
I20260709 12:13:38.389864 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:38.390084 14445 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:38.390630 11928 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:38.389667 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 135747us
I20260709 12:13:38.394129 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:38.395948 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.399791 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:38.400547 11935 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:38.408283 11933 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:38.389039 14513 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.409336 14300 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.396615 14389 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:38.418679 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.361989 14443 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:38.469159 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 191763us
I20260709 12:13:38.380847 14494 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.357201 14707 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.392745 14548 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:13:38.407843 14408 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:38.490370 11173 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:38.533133 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 114562us
I20260709 12:13:38.534075 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:38.536381 14460 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:38.536863 14460 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:38.538049 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 66385.3us
I20260709 12:13:38.465863 14539 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:38.539991 14577 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:38.467962 14389 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:38.412216 14716 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:38.378669 14676 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.548834 14460 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.549371 14539 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.466524 14621 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.551367 14621 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:38.488897 14717 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:13:38.491463 14599 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:38.556432 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.547473 14389 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:38.581374 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:38.470420 14513 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.547153 14445 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:13:38.591818 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.545830 14386 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:13:38.483304 14300 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.580806 14703 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:38.588687 14408 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:38.606226 14568 raft_consensus.cc:2764] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:13:38.616835 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53701.9us
I20260709 12:13:38.585614 14599 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:38.631676 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 97687.4us
I20260709 12:13:38.637378 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:38.635313 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:38.639452 14494 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:38.639479 14702 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:38.642975 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:38.647701 14700 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:38.648232 14700 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:13:38.649854 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.650830 14703 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:38.654851 14726 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:13:38.655169 14494 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.655787 14494 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:38.656903 14718 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:13:38.657472 14513 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:38.657954 14494 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:38.631605 14699 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.659688 14699 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:38.661437 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 288511us
W20260709 12:13:38.668115 14582 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 12:13:38.630638 14445 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:38.669973 14445 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.670333 14445 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:38.673086 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:38.673475 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:38.673967 11174 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:38.674386 11176 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:38.686206 11179 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:38.677315 14300 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:38.630594 14733 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:38.678450 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:38.691431 11326 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:38.679520 14621 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:38.630175 14720 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:38.680826 14707 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:38.674226 11182 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
W20260709 12:13:38.706367 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.684561 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:38.687304 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:38.687606 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:38.707072 11927 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:38.619711 14408 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:38.694069 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:38.694437 11174 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:13:38.709008 11330 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:38.694566 14733 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.695669 11179 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:38.711663 11326 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
W20260709 12:13:38.699923 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 62663.8us
I20260709 12:13:38.701458 14699 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:38.649246 14700 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.704041 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.704552 11933 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:38.684561 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:38.713666 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:38.688634 14228 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:38.715559 11930 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:38.715857 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:38.716531 11331 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:38.710964 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:38.716475 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:38.760972 11324 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 5.
I20260709 12:13:38.784441 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:38.711246 14513 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 12:13:38.796198 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 80423.7us
I20260709 12:13:38.766407 14494 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:13:38.804359 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87961.7us
W20260709 12:13:38.804576 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 169358us
I20260709 12:13:38.794037 14700 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:38.786983 14621 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:13:38.811249 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.789781 14714 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:13:38.847159 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 143224us
I20260709 12:13:38.806171 14494 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.780651 14389 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.810669 14519 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:38.807968 14716 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:38.794516 14669 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:13:38.815271 14621 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.816258 14464 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:38.866484 11180 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:13:38.847275 14670 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.859390 14519 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.895020 14519 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:38.859004 14494 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:38.908694 14670 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:38.910126 14711 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:38.913573 14744 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.867060 14228 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:13:38.919255 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:38.867766 14621 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:38.916252 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 103657us
I20260709 12:13:38.861660 14716 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.928612 11927 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:13:38.938683 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 85168.3us
I20260709 12:13:38.942973 11179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25983a67e23a4c708ae496061b992207"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:38.943408 14729 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:38.943892 14729 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.943956 11179 raft_consensus.cc:606] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:38.944413 11179 raft_consensus.cc:614] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:38.944916 14738 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:13:38.916029 14467 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:38.944375 14729 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:38.951857 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.867404 14539 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:13:38.964469 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:38.965557 11927 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:38.965651 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 155209us
I20260709 12:13:38.981410 14599 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:38.981976 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:38.983275 14733 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:38.985939 14731 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:13:38.986145 14445 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.989538 14670 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:38.861878 14408 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:38.991801 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.982211 14668 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:38.950133 14716 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:38.986565 14665 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.754s	user 0.006s	sys 0.006s
I20260709 12:13:38.952318 14467 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:39.018357 14467 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.011605 14055 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.019690 14055 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.026757 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:38.954916 14464 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:13:39.038242 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.038587 14445 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:39.058036 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 95262.4us
I20260709 12:13:38.985999 14599 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:13:39.070371 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:38.946940 14744 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:38.956562 14709 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:13:39.077215 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:39.078192 11933 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:39.075098 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:39.057973 14228 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:13:39.075040 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.082371 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:39.082569 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.083207 11256 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:39.086248 11249 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:39.080425 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.077235 11930 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:39.079252 11927 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:39.079087 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:39.088223 11928 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:39.080197 14690 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:39.096055 14744 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:39.096690 14389 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.099233 11255 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:39.100566 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.101254 11255 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:39.102236 11257 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:39.102473 11252 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:39.102779 14719 consensus_queue.cc:1175] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 9 status { last_received { term: 8 index: 4 } last_committed_idx: 3 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:39.103420 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.107959 11248 raft_consensus.cc:1217] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-8.7]   Dedup: 3.5->[6.6-8.7]
I20260709 12:13:39.108481 11248 pending_rounds.cc:85] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 5
I20260709 12:13:39.108739 11248 pending_rounds.cc:107] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:13:39.108981 11248 raft_consensus.cc:2889] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:39.105432 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.114444 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.115136 11254 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:39.134379 11249 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:39.151926 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.152735 11330 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 6.
I20260709 12:13:39.154819 11927 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:13:39.168217 11147 connection.cc:256] RPC call timeout handler was delayed by -1.8201001249999536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 12:13:39.168771 11147 connection.cc:256] RPC call timeout handler was delayed by -1.7688001249998706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 12:13:39.169054 11147 connection.cc:256] RPC call timeout handler was delayed by -1.6302001249998739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 12:13:39.169342 11147 connection.cc:256] RPC call timeout handler was delayed by -1.5078001249999033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20230000000000004s for a response
W20260709 12:13:39.169602 11147 connection.cc:256] RPC call timeout handler was delayed by -1.4034001249999619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
W20260709 12:13:39.171736 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.179383 14539 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:39.180357 14539 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.182029 14722 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.182770 14722 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.183863 11322 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:39.187760 11928 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:39.206879 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:39.219020 11256 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:39.224800 11927 raft_consensus.cc:2368] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
I20260709 12:13:39.222921 11254 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:39.228953 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.228310 11932 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:39.115990 14690 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.237864 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:39.241199 11249 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:39.194800 14703 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:39.238541 11253 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:39.246569 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.244112 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.247648 11255 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:39.248574 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.154305 14494 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.253343 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.254856 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:39.213753 14055 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.162304 14621 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.257666 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:39.257712 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 179426us
I20260709 12:13:39.259243 11256 raft_consensus.cc:2410] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:13:39.252435 14676 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:39.265316 11252 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:39.266999 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.268460 14704 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:39.245834 14677 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:13:39.269369 14552 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.279695 11253 raft_consensus.cc:2368] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
I20260709 12:13:39.284996 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.291638 11256 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:13:39.306586 14742 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.330219 14742 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:39.307560 14755 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:13:39.335175 14621 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:39.310424 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.319015 14729 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.322921 14539 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.324467 14494 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:39.267714 14703 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:39.288728 14754 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:13:39.347436 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.333535 14742 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:39.289709 14722 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:39.350204 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70559.2us
I20260709 12:13:39.352241 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.278697 14055 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 12:13:39.352543 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 85890.6us
I20260709 12:13:39.357434 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:39.359092 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74168.2us
I20260709 12:13:39.366616 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.360951 14621 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:39.370517 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:39.372638 11901 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:39.368824 14742 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.377839 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:39.378165 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:39.378999 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.360457 14539 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:13:39.382740 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.397280 11174 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:39.382980 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.374246 14519 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.383530 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.398984 11178 raft_consensus.cc:2368] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5.
I20260709 12:13:39.399653 14519 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.383674 11182 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:13:39.383802 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.402216 11181 raft_consensus.cc:2410] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:13:39.381577 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.386109 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.386760 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.404701 11177 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:13:39.376099 14552 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:39.387827 14742 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.388479 14450 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:39.374711 14548 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:39.389623 14729 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.390978 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.392273 14703 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.393008 14724 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:39.393347 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:39.426540 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:39.427038 14732 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.639s	user 0.004s	sys 0.000s
I20260709 12:13:39.427433 11177 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:39.427621 14724 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:39.427436 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:39.407146 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.409240 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.429852 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:39.430719 11322 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:39.409297 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.421897 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.432183 11331 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:39.424239 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45289.3us
I20260709 12:13:39.368130 14722 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:39.431819 11326 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:39.433220 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.428251 14724 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:39.421507 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.438951 11329 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:39.420567 14539 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.419569 14450 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:39.446406 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.420006 14704 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:39.443841 14621 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:39.434923 14747 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:39.429863 14467 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:39.448352 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.409155 14677 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:39.438171 14552 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:39.445973 14729 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:39.453325 14539 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.468104 14552 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.454492 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.470134 14539 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.413915 14716 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.466076 14677 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.453564 14450 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:39.487269 11179 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:13:39.488777 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:39.489979 11927 raft_consensus.cc:2368] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7.
I20260709 12:13:39.452646 14668 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.455200 14722 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:39.483143 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.505904 14450 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.486155 14677 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.480495 11328 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:39.526381 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.529310 11296 leader_election.cc:400] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
I20260709 12:13:39.531513 11296 leader_election.cc:403] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:39.532985 11255 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:39.538355 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.542169 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:39.546640 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 135951us
I20260709 12:13:39.506201 14668 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.530049 14722 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:39.553344 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.537155 14408 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:39.546861 14744 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:39.562022 14744 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:39.563098 14450 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.563001 14744 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.547190 14733 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:13:39.539645 14621 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:39.542348 14740 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:13:39.576408 14716 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:39.554412 14443 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:13:39.577318 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.578290 11325 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:39.578328 14668 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.555543 11324 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 4.
I20260709 12:13:39.579221 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.579779 11325 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.579828 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.581667 14742 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:39.582050 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.582952 11327 raft_consensus.cc:2410] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:39.587512 11325 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 5.
I20260709 12:13:39.587700 14460 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:39.589181 14690 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.482379 14704 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.563427 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.593703 11323 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:39.574295 14722 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.602804 11325 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 12:13:39.603940 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 174337us
W20260709 12:13:39.607878 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.608882 11330 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:39.615229 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.615903 11177 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:39.616109 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.616358 11331 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:39.616690 11328 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:39.614892 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:39.614840 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:39.617648 11928 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 6: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:39.618752 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.620143 11180 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:13:39.620395 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.616886 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.625541 11255 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:39.628050 11323 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:13:39.629431 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:39.620101 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.631546 11296 leader_election.cc:400] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
I20260709 12:13:39.632696 11296 leader_election.cc:400] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7.
I20260709 12:13:39.632956 11296 leader_election.cc:403] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:39.634099 11257 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:39.635324 11253 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:39.636946 11325 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:13:39.634497 11257 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:39.641011 11933 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:13:39.641320 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.642503 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.643155 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.643353 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.645264 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.646862 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:39.647722 11929 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:13:39.648780 14747 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:39.630625 14765 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:39.650038 14765 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:39.651796 11296 leader_election.cc:400] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5.
I20260709 12:13:39.652343 11296 leader_election.cc:403] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:13:39.653481 11900 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.992s (SENT)
I20260709 12:13:39.656373 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.650434 14765 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:39.658104 14744 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 4. Current term is 5.
I20260709 12:13:39.658490 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:39.658896 14744 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:39.659350 14408 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:39.660626 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.661731 14744 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:39.661819 14677 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.662434 14677 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.665422 11927 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:39.657059 11178 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:39.670747 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.672417 11254 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:13:39.673746 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:39.674919 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:39.676096 11900 leader_election.cc:336] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.013s (SENT)
I20260709 12:13:39.633348 14621 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.678357 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.678802 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.671780 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.681015 11177 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:39.682542 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.739276 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.645320 14733 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:13:39.679886 14742 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:39.749758 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.769183 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.753013 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.772760 11178 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:39.778956 11331 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:39.785904 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.682648 14408 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:39.682376 14753 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:39.811136 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.825649 11253 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:39.833957 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.796666 14703 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:39.795344 11935 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:39.797488 11254 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:13:39.840068 11254 pending_rounds.cc:85] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 4
I20260709 12:13:39.840363 11254 pending_rounds.cc:107] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:13:39.840596 11254 raft_consensus.cc:2889] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:39.840907 11254 pending_rounds.cc:107] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:13:39.841144 11254 raft_consensus.cc:2889] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:39.789459 14552 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.834968 11177 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:39.722582 14621 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:39.830972 14719 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.868706 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:39.834193 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 158144us
I20260709 12:13:39.853080 11331 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:39.854326 11323 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:39.855050 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.873677 11250 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:39.876545 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.790115 14742 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:13:39.831632 14740 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
I20260709 12:13:39.882388 14539 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:39.837782 14703 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:39.835343 14673 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.890229 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.889580 11327 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:39.889017 14724 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:13:39.889310 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:39.893934 11179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:39.898828 11253 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:39.919929 11179 raft_consensus.cc:606] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:39.920269 11179 raft_consensus.cc:614] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:39.897676 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:39.920653 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 238918us
I20260709 12:13:39.901939 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.922344 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:39.902818 11175 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:13:39.903946 11323 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 12:13:39.905040 11324 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:39.912082 11331 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:39.908196 11322 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:39.891943 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:39.892441 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 263080us
I20260709 12:13:39.870652 14552 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.909771 11173 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:39.917224 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.923513 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:39.933350 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:39.878821 14719 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:39.930006 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:39.931123 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:39.888998 14716 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:13:39.923569 11323 pending_rounds.cc:85] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 3
I20260709 12:13:39.937216 11323 pending_rounds.cc:107] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:13:39.937759 11323 raft_consensus.cc:2889] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:39.938270 11323 pending_rounds.cc:107] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:13:39.938619 11323 raft_consensus.cc:2889] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:39.940269 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.941532 11254 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:39.942364 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:39.943334 11173 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:39.945062 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:39.945880 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54044.2us
I20260709 12:13:39.947075 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:39.920214 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:39.900202 14703 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:40.003924 14539 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:39.942317 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:39.951565 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:39.953745 14677 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:40.028925 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:39.954576 14386 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:39.956702 11250 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:39.942687 14450 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:39.957880 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.961956 14552 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:39.953846 14494 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:39.965725 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:40.055691 11324 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:39.966635 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:39.913754 14673 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:39.969198 14761 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.547s	user 0.007s	sys 0.015s
I20260709 12:13:39.969069 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:39.969544 14775 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:39.972849 11182 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:39.973551 14598 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:39.974668 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:40.090628 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 116063us
I20260709 12:13:39.974861 11249 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:13:39.978525 11251 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:39.889264 11177 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:39.988234 11257 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:39.991556 14762 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.567s	user 0.006s	sys 0.006s
W20260709 12:13:39.993510 14768 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:13:40.096807 11257 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:39.994567 11254 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:39.995031 14751 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:39.995790 14719 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:39.931700 14599 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:39.914800 14704 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:39.932536 14740 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:39.942932 14753 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:39.893740 14766 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 12:13:39.998245 14621 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:40.002468 14765 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.091295 14494 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:40.008922 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79879.5us
I20260709 12:13:40.010033 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:40.149159 11929 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:40.029965 14690 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:39.942287 14408 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:40.032500 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.015472 14774 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:40.032500 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:40.052615 11328 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:40.167766 11327 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:40.102298 14742 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.064178 11253 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:40.088610 11323 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:40.101130 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.059026 14677 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:40.124423 11931 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:13:40.124902 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 177902us
I20260709 12:13:40.168735 11331 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:40.149206 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:40.167505 14774 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.172242 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:40.172391 14774 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:40.172971 11934 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:40.176348 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.177115 11255 raft_consensus.cc:2376] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10. Re-sending same reply.
I20260709 12:13:40.178288 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.178887 11255 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10. Re-sending same reply.
I20260709 12:13:40.118850 14599 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.138477 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:40.179690 14599 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:40.097523 14460 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:40.181734 14460 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.182214 14460 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:40.182266 14673 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:40.182914 14762 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:13:40.184518 14774 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.086427 14450 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.102276 14751 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:40.098811 14773 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.950191 11181 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:13:40.150652 11252 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:40.061422 11178 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:40.179767 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.005410 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:40.187955 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:40.191159 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.193228 11930 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:13:40.190629 11249 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:40.192940 11255 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:40.201387 11181 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:40.202637 11176 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:40.200922 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1713) took 1101 ms. Trace:
I20260709 12:13:40.209995 11327 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:40.210575 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.218200 11323 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:40.186756 14733 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7.
I20260709 12:13:40.205927 11248 rpcz_store.cc:276] 0709 12:13:39.099175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.107333 (+  8158us) service_pool.cc:224] Handling call
0709 12:13:39.107671 (+   338us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:39.138398 (+ 30727us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:39.243767 (+105369us) log.cc:844] Serialized 14 byte log entry
0709 12:13:39.361724 (+117957us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:39.363442 (+  1718us) log.cc:844] Serialized 52 byte log entry
0709 12:13:39.363954 (+   512us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:39.363987 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:39.364063 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:39.990724 (+626661us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:40.188042 (+197318us) raft_consensus.cc:1685] finished
0709 12:13:40.200677 (+ 12635us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:40.200843 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":183491,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":32741,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":13697,"spinlock_wait_cycles":19054336,"thread_start_us":186901,"threads_started":2,"wal-append.queue_time_us":162744}
W20260709 12:13:40.219813 14781 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:13:40.307303 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 296152us
I20260709 12:13:40.220707 14774 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:40.222872 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.179807 14704 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:13:40.169067 14753 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:40.225572 11934 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:40.230170 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.229741 14751 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.232297 14765 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:40.208593 14747 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:40.193449 14742 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.238276 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.202061 14773 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.207890 14665 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:13:40.241211 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.242136 14494 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.243098 14703 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.248059 14450 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:40.169139 14690 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:40.237118 14599 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.253755 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:40.255148 14460 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.255743 14729 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:40.255959 14720 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:40.260200 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1705) took 1190 ms. Trace:
I20260709 12:13:40.201610 14677 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:40.267304 14676 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:13:40.268170 14744 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
W20260709 12:13:40.287631 11325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.002s	sys 0.021s
W20260709 12:13:40.288138 11933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.009s	sys 0.014s
I20260709 12:13:40.293731 11249 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:40.304136 11929 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:40.304710 11250 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:40.395465 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.399291 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.515951 11254 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:40.436383 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.063s (SENT)
I20260709 12:13:40.506588 11328 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:13:40.443471 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:40.513248 11325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.002s	sys 0.022s
I20260709 12:13:40.463368 11252 raft_consensus.cc:2376] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10. Re-sending same reply.
I20260709 12:13:40.483903 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.492043 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.499848 11251 rpcz_store.cc:276] 0709 12:13:39.070166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.070414 (+   248us) service_pool.cc:224] Handling call
0709 12:13:40.259864 (+1189450us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54002a67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:40.260059 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011295104}
I20260709 12:13:40.505443 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.460597 14445 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.522874 11180 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:40.513475 11933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.010s	sys 0.014s
I20260709 12:13:40.525822 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.526495 11250 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:40.527463 11248 raft_consensus.cc:2368] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 10.
I20260709 12:13:40.528432 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.529503 11249 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:40.507699 14720 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:40.531404 14720 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:40.531958 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.532783 11254 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:40.532986 14742 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:40.534896 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 281232us
W20260709 12:13:40.533901 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 225396us
W20260709 12:13:40.538604 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.115s (SENT)
I20260709 12:13:40.493109 14751 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:40.539208 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:40.548555 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:40.549283 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.501533 14729 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:40.505683 14676 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:40.557655 11256 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:40.513978 14598 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:40.550923 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.565770 11255 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:40.528359 14744 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:40.573886 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.563961 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:40.507431 14539 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:40.575388 11257 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:40.576171 11252 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:13:40.574270 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:40.579816 11182 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:13:40.580286 11182 pending_rounds.cc:85] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 4
I20260709 12:13:40.580540 11182 pending_rounds.cc:107] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:13:40.580842 11182 raft_consensus.cc:2889] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:40.548857 14699 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:40.573029 14568 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:40.557958 14729 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:40.566550 14386 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:13:40.593775 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 457096us
I20260709 12:13:40.562865 14775 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:40.534587 14445 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.597146 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:40.576887 14539 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:40.582051 14673 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:40.533975 14753 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:40.603170 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64588.2us
W20260709 12:13:40.604543 11900 leader_election.cc:336] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.135s (SENT)
I20260709 12:13:40.566244 14716 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 12:13:40.599977 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 412067us
I20260709 12:13:40.582031 14742 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:40.592970 14733 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.609774 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:40.596316 14740 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.620330 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:40.599660 14744 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:40.621798 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:40.659657 11330 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:13:40.600075 14775 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:40.664512 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 491030us
I20260709 12:13:40.663399 11331 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:13:40.682039 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:40.684652 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:40.695068 11326 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:40.600804 14598 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:13:40.698016 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:40.620762 14673 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:40.684585 14786 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:13:40.707458 14784 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:13:40.724864 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 115159us
I20260709 12:13:40.582634 14690 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.610458 14386 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:13:40.726359 11252 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:40.728945 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:40.684969 14460 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:40.729895 11255 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:40.775749 14228 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:40.776299 14228 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:40.776736 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 644) took 1112 ms. Trace:
I20260709 12:13:40.777231 11933 rpcz_store.cc:276] 0709 12:13:39.664454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.678104 (+ 13650us) service_pool.cc:224] Handling call
0709 12:13:39.678394 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:39.723038 (+ 44644us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:39.723086 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:39.745107 (+ 22021us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.523731 (+778624us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:40.523788 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.523882 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:40.776498 (+252616us) raft_consensus.cc:1685] finished
0709 12:13:40.776521 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:40.776669 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48777,"raft.queue_time_us":60151,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":26074,"thread_start_us":753708,"threads_started":1,"wal-append.queue_time_us":925827}
I20260709 12:13:40.777238 14228 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.780997 14228 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:40.697368 14796 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:40.777601 11331 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
W20260709 12:13:40.754072 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 149557us
I20260709 12:13:40.708966 14443 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:40.788249 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.795950 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.798468 11250 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:40.761404 14704 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 12:13:40.799502 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 217856us
I20260709 12:13:40.664058 14740 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.721100 14714 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:40.804716 11257 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:40.731140 11251 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:13:40.784138 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:40.806164 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 209076us
I20260709 12:13:40.612990 14568 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.789858 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:40.662873 14742 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.766659 14690 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:40.695910 14621 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:13:40.685127 14744 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.610711 14677 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.799466 14703 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:40.684298 14733 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.847616 11326 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:13:40.823035 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61349.7us
I20260709 12:13:40.709198 14722 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:13:40.723165 14673 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.786111 14599 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:40.757668 14766 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:13:40.760038 14797 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.855432 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 645) took 1106 ms. Trace:
I20260709 12:13:40.856420 14445 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:40.857049 14445 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:40.857547 14445 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:40.843426 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47936.5us
I20260709 12:13:40.809554 14443 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:40.867182 14443 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:40.870409 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70183.5us
I20260709 12:13:40.873749 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:40.728892 14539 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.880012 14539 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:40.876462 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69473.2us
I20260709 12:13:40.877199 14798 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:40.882002 14798 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:40.883948 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:40.886606 11079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 242) took 15998 ms (client timeout 15000 ms). Trace:
I20260709 12:13:40.894465 11933 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:40.838012 14773 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:40.811450 14795 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:40.709470 14676 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:40.912323 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:40.903090 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:40.943562 11174 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:40.903715 11330 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:40.896587 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:40.913177 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:40.897537 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:40.945079 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50602.7us
I20260709 12:13:40.945518 11934 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:40.855860 11927 rpcz_store.cc:276] 0709 12:13:39.749388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.750438 (+  1050us) service_pool.cc:224] Handling call
0709 12:13:39.750878 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:39.828947 (+ 78069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:39.828983 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:39.829568 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.211828 (+382260us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:40.211890 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.211989 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:40.855213 (+643224us) raft_consensus.cc:1685] finished
0709 12:13:40.855233 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:40.855369 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33677,"raft.queue_time_us":51600,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":6150,"thread_start_us":333046,"threads_started":1,"wal-append.queue_time_us":633417}
I20260709 12:13:40.927188 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:40.941478 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68142.5us
I20260709 12:13:40.947068 11178 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:40.941663 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:40.947443 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:40.950079 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:40.943967 11323 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:40.836678 14668 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:13:40.956028 11323 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:13:40.958375 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:40.962240 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:40.887105 11079 rpcz_store.cc:269] 0709 12:13:24.891955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.894265 (+  2310us) service_pool.cc:224] Handling call
0709 12:13:40.886340 (+15992075us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:36.618443 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:36.619299 (+   856us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:36.619408 (+   109us) write_op.cc:435] Acquired schema lock
0709 12:13:36.619453 (+    45us) tablet.cc:662] Decoding operations
0709 12:13:36.634517 (+ 15064us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:36.634692 (+   175us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:36.634743 (+    51us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:13:36.640305 (+  5562us) tablet.cc:701] Row locks acquired
0709 12:13:36.640360 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:13:36.640589 (+   229us) write_op.cc:270] Start()
0709 12:13:36.640784 (+   195us) write_op.cc:276] Timestamp: P: 1783599216640503 usec, L: 0
0709 12:13:36.640836 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 12:13:36.763160 (+122324us) log.cc:844] Serialized 7467 byte log entry
0709 12:13:37.507122 (+743962us) op_driver.cc:464] REPLICATION: finished
0709 12:13:37.712100 (+204978us) write_op.cc:301] APPLY: starting
0709 12:13:37.712301 (+   201us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:37.778641 (+ 66340us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:37.799649 (+ 21008us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:37.935076 (+135427us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:37.935564 (+   488us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 12:13:37.935625 (+    61us) write_op.cc:312] APPLY: finished
0709 12:13:37.936092 (+   467us) log.cc:844] Serialized 132 byte log entry
0709 12:13:37.936288 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:37.936464 (+   176us) write_op.cc:454] Released schema lock
0709 12:13:37.936546 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:37.936928 (+   382us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:37.936956 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":620370,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":155457,"apply.run_cpu_time_us":6662,"apply.run_wall_time_us":239239,"num_ops":19,"prepare.queue_time_us":68896,"prepare.run_cpu_time_us":45452,"prepare.run_wall_time_us":357758,"raft.queue_time_us":328526,"raft.run_cpu_time_us":32662,"raft.run_wall_time_us":68643,"replication_time_us":866115,"thread_start_us":324324,"threads_started":4,"wal-append.queue_time_us":361898,"wal-append.run_cpu_time_us":17142,"wal-append.run_wall_time_us":1800244}]]}
W20260709 12:13:40.974117 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 159258us
I20260709 12:13:40.970736 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:40.928553 14055 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:40.942368 14450 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:40.986513 11174 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:40.941849 14408 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:40.952135 14774 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:40.993496 11327 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:41.004034 11326 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:41.002302 11934 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:40.971222 14676 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:41.011742 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.012573 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:41.028740 11331 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:40.980155 14809 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.980785 14766 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.978935 14703 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.953747 14773 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:13:40.926860 14665 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:13:40.978688 14539 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:13:41.014881 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 206602us
I20260709 12:13:40.979534 14704 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:41.036695 14810 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:40.992806 14450 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.989670 14055 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:41.045915 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:41.001977 14795 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.051961 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:41.102619 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50792.3us
I20260709 12:13:41.010643 14408 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:41.004043 14774 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:41.097662 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.008668 14719 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:13:40.855338 14797 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:40.804738 14796 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:41.050257 14568 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:41.059141 14539 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:41.096487 14450 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:41.038064 14599 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:13:41.104135 14408 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:41.058435 14598 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 12:13:40.882884 14798 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.050766 14703 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.114838 14445 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:41.050187 14766 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.050479 14809 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.586084 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:41.383630 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 419230us
W20260709 12:13:41.485191 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 391726us
I20260709 12:13:41.202060 14815 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:41.588796 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:41.589372 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:41.159046 14812 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.464s
I20260709 12:13:41.549784 14827 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:13:41.625440 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:41.430745 14796 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:41.590975 11254 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:13:41.591262 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:41.393258 14833 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:41.498478 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.103834 14817 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:13:41.660662 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.015s	sys 0.018s
W20260709 12:13:41.486944 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 496613us
W20260709 12:13:41.685428 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.015s	sys 0.022s
W20260709 12:13:41.405076 14804 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.543s	user 0.001s	sys 0.000s
I20260709 12:13:41.595868 11327 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:41.266690 11331 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:41.596625 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 203801us
I20260709 12:13:41.139750 14740 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:41.626986 11252 raft_consensus.cc:2410] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 12:13:41.403474 14801 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.452s	user 0.000s	sys 0.001s
W20260709 12:13:41.418624 14808 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:13:41.469087 14789 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.528s	user 0.001s	sys 0.000s
W20260709 12:13:41.394627 14799 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 12:13:41.469991 14774 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.152153 14719 consensus_queue.cc:1243] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:13:41.404500 14819 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:13:41.742182 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 48792.4us
W20260709 12:13:41.698623 14751 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.147s	user 0.012s	sys 0.000s
I20260709 12:13:41.402907 14747 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:41.683743 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 92562.6us
I20260709 12:13:41.696660 14810 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.626507 11251 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:41.428378 14673 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:41.692180 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:41.709059 14831 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:41.402750 11077 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:41.706766 11327 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:41.138610 14621 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.493s
I20260709 12:13:41.136608 14704 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:41.115473 14795 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.389317 14744 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:41.590993 14798 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:41.390658 14539 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:41.753749 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:41.689200 14714 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.792419 14831 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:41.815971 14774 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:41.818504 14828 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:13:41.819017 11077 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:41.824790 14690 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.987s	user 0.007s	sys 0.002s
I20260709 12:13:41.826128 11326 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:41.828027 11326 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:41.830722 14797 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:41.832909 14844 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:41.833447 14844 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:41.799496 14704 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:41.804585 14539 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:41.833948 14844 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:41.805804 14820 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
W20260709 12:13:41.835992 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.015s	sys 0.022s
W20260709 12:13:41.836385 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.015s	sys 0.022s
I20260709 12:13:41.804121 14765 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:13:41.810138 14714 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:41.840718 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:41.619149 14766 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:41.631201 14829 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.770546 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:41.202783 14818 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:41.693993 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:41.753724 14819 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:41.847419 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1867) took 1925 ms. Trace:
I20260709 12:13:41.811928 14840 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:13:41.847843 11173 rpcz_store.cc:276] 0709 12:13:39.921953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.955623 (+ 33670us) service_pool.cc:224] Handling call
0709 12:13:39.956059 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.088899 (+132840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.088935 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.089535 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.562640 (+473105us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:40.562710 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.883820 (+321110us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54000d8254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:40.883868 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.847049 (+963181us) raft_consensus.cc:1685] finished
0709 12:13:41.847071 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.847334 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3836,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5866,"spinlock_wait_cycles":304475520,"thread_start_us":394197,"threads_started":1,"wal-append.queue_time_us":710443}
I20260709 12:13:41.812963 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2155) took 1857 ms. Trace:
I20260709 12:13:41.776755 14810 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.849936 11329 rpcz_store.cc:276] 0709 12:13:39.955176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.085751 (+130575us) service_pool.cc:224] Handling call
0709 12:13:40.086237 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.505759 (+419522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.505801 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.522481 (+ 16680us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.808654 (+286173us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:40.847074 (+ 38420us) log.cc:844] Serialized 14 byte log entry
0709 12:13:40.967519 (+120445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.967619 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.812749 (+845130us) raft_consensus.cc:1685] finished
0709 12:13:41.812769 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.812893 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12196,"prepare.queue_time_us":39227,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":5480,"raft.queue_time_us":126233,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":6161,"spinlock_wait_cycles":43776,"thread_start_us":769423,"threads_started":3,"wal-append.queue_time_us":546403}
I20260709 12:13:41.773998 11256 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:13:41.722456 14826 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:13:41.689334 14821 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:41.831343 11077 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:41.781123 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:41.784796 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 92697.8us
I20260709 12:13:41.720865 14822 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:41.753933 14740 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.775516 14747 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.698824 14796 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.841699 11929 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:41.829486 11326 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:41.845211 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74741.9us
W20260709 12:13:41.875816 11900 leader_election.cc:336] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.530s (SENT)
W20260709 12:13:41.846273 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 152353us
I20260709 12:13:41.878118 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:41.854691 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:41.809523 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2140) took 2167 ms. Trace:
I20260709 12:13:41.879858 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:41.856315 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:41.857232 11327 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:41.852977 11331 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:41.842134 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:41.878158 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:41.885027 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:41.880750 11325 rpcz_store.cc:276] 0709 12:13:39.641982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.644165 (+  2183us) service_pool.cc:224] Handling call
0709 12:13:39.644441 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:39.649657 (+  5216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:39.649699 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:39.650239 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.517801 (+867562us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:40.718144 (+200343us) log.cc:844] Serialized 14 byte log entry
0709 12:13:40.726383 (+  8239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.726461 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.809209 (+1082748us) raft_consensus.cc:1685] finished
0709 12:13:41.809237 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.809417 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12832,"prepare.queue_time_us":39024,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":93,"raft.queue_time_us":156858,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":169,"thread_start_us":941657,"threads_started":2,"wal-append.queue_time_us":1007274}
W20260709 12:13:41.904567 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 2.186s (SENT)
I20260709 12:13:41.905197 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:41.832340 14806 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:41.915612 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:41.842762 14228 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.061s	user 0.010s	sys 0.003s
I20260709 12:13:41.258960 14055 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:41.809206 14720 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.277s	user 0.008s	sys 0.003s
W20260709 12:13:41.958300 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46043.2us
I20260709 12:13:41.904102 11077 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 3 to 4, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 4 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:41.693888 14833 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:41.803530 14450 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
W20260709 12:13:41.974790 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.996692 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:41.985879 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 242536us
I20260709 12:13:41.846961 14766 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:41.969767 11325 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:13:41.922026 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2159) took 1021 ms. Trace:
I20260709 12:13:42.002619 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1538) took 1005 ms. Trace:
I20260709 12:13:41.997496 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:42.003136 11325 rpcz_store.cc:276] 0709 12:13:40.997265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.905281 (+908016us) service_pool.cc:224] Handling call
0709 12:13:41.905611 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.002282 (+ 96671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.002554 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:41.831455 14841 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.002406 11328 rpcz_store.cc:276] 0709 12:13:40.900390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.903790 (+  3400us) service_pool.cc:224] Handling call
0709 12:13:40.952777 (+ 48987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:41.091747 (+138970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.091780 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:41.159817 (+ 68037us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.685576 (+525759us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:41.708024 (+ 22448us) log.cc:844] Serialized 14 byte log entry
0709 12:13:41.921454 (+213430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.921593 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.921687 (+    94us) raft_consensus.cc:1685] finished
0709 12:13:41.921707 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.921926 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61274,"prepare.queue_time_us":22213,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":194417,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":41129,"thread_start_us":571517,"threads_started":3,"wal-append.queue_time_us":540829}
I20260709 12:13:41.968420 14827 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:41.916581 14849 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:41.974123 14598 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:41.810621 14733 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:41.994125 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 89586.1us
I20260709 12:13:41.853067 14599 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.987983 14703 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:41.844483 14809 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:41.845732 14838 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.039453 14838 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:41.845865 14818 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:41.807741 14848 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.042289 14848 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.042737 14839 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:13:42.045151 14848 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:41.999116 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:41.908973 14668 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:41.825446 14690 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.157s
I20260709 12:13:41.843300 14835 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.841571 14714 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:41.833534 14719 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:41.991905 14775 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.284s	user 0.013s	sys 0.000s
W20260709 12:13:41.996189 14724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.007s	sys 0.040s
I20260709 12:13:41.996930 14720 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.989187 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:42.072307 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 83217.8us
I20260709 12:13:42.012286 11325 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:13:41.996450 14055 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:41.986838 14744 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:42.055725 14733 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.051025 14849 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:41.982393 14815 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:42.023221 11326 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:42.026875 14834 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:42.082819 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1540) took 1081 ms. Trace:
I20260709 12:13:42.059249 14818 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:42.083343 11326 rpcz_store.cc:276] 0709 12:13:41.001613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.018635 (+1017022us) service_pool.cc:224] Handling call
0709 12:13:42.022750 (+  4115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.082422 (+ 59672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.082740 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:42.027281 14832 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.027491 14836 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:13:42.071470 14724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.008s	sys 0.041s
I20260709 12:13:42.026999 11077 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:41.841686 14802 consensus_queue.cc:1175] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "786851d4227e4dd89b75ccdea0b1f03b" responder_term: 9 status { last_received { term: 7 index: 9 } last_committed_idx: 9 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:41.875598 14740 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:42.012447 14494 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.003s	sys 0.011s
I20260709 12:13:42.018918 14817 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:42.071732 14775 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.972301 14460 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:13:41.981135 14765 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:13:41.971328 14840 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:13:41.971711 14820 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:13:42.037842 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:42.096361 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58598.5us
I20260709 12:13:42.038549 14599 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:41.987485 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:42.047318 11328 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:42.048038 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.100459 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1541) took 1093 ms. Trace:
I20260709 12:13:42.100874 11328 rpcz_store.cc:276] 0709 12:13:41.008274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.022777 (+1014503us) service_pool.cc:224] Handling call
0709 12:13:42.047111 (+ 24334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.100211 (+ 53100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.100402 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:42.082257 14729 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.481s	user 0.014s	sys 0.000s
I20260709 12:13:41.917745 14796 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:41.998791 14673 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:13:41.875298 14822 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.059577 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.016031 14810 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.016703 14766 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:41.842689 14829 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:42.069515 11255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.008s	sys 0.021s
I20260709 12:13:42.069908 11324 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:42.007095 14450 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.050333 14668 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.058544 14704 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:41.993414 14821 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:42.080037 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1539) took 1080 ms. Trace:
I20260709 12:13:42.124629 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:41.846670 14819 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.029203 14716 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:42.100353 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:42.092537 14747 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.139494 14819 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:42.141708 14751 consensus_queue.cc:1175] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 9 status { last_received { term: 7 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: [8.5-8.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:42.086552 14842 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:13:42.142946 14842 consensus_queue.cc:1243] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:13:42.143477 14842 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:42.143862 14842 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:42.091256 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.123037 14668 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:42.144790 14842 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:42.185617 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:42.189826 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 199083us
I20260709 12:13:42.014377 14841 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.091619 14815 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.083051 14744 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:42.088305 14494 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.003s	sys 0.011s
I20260709 12:13:42.106302 11931 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:13:42.198496 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 199471us
I20260709 12:13:42.136245 11926 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:42.124873 14773 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.147445 14796 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.094115 14840 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:13:42.063637 14835 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:42.150368 14408 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.769s	user 0.007s	sys 0.003s
W20260709 12:13:42.151036 14676 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.110s	user 0.012s	sys 0.001s
I20260709 12:13:42.224539 14568 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:42.225023 14568 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:42.090358 14817 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.226042 14568 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.227488 14568 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:42.106365 14742 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.222952 14744 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.153571 14822 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.154241 14849 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:42.152745 14793 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.464s	user 0.010s	sys 0.017s
I20260709 12:13:42.071533 14719 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.231572 14719 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:42.160416 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.231369 14844 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.232192 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:42.232852 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2150) took 2339 ms. Trace:
I20260709 12:13:42.233289 14676 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:42.112665 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:42.191093 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.085273 14838 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.192061 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.124150 11325 rpcz_store.cc:276] 0709 12:13:41.000506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.011615 (+1011109us) service_pool.cc:224] Handling call
0709 12:13:42.011945 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.073239 (+ 61294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.079958 (+  6719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:42.192518 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.241012 11178 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
W20260709 12:13:42.119846 11255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.010s	sys 0.021s
I20260709 12:13:42.193821 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:42.241936 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 168451us
I20260709 12:13:42.242455 11177 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:13:42.242214 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.088119 14740 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.088589 14833 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.195497 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.201747 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.079550 14386 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:42.209277 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.203961 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:42.091765 14852 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.206678 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.305855 11257 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:42.233243 11179 rpcz_store.cc:276] 0709 12:13:39.893562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.944284 (+ 50722us) service_pool.cc:224] Handling call
0709 12:13:39.967455 (+ 23171us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.015051 (+ 47596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.015089 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.028155 (+ 13066us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.438163 (+410008us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:40.438245 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.438424 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.151845 (+1713421us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:42.232541 (+ 80696us) raft_consensus.cc:1685] finished
0709 12:13:42.232563 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.232766 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47144,"raft.queue_time_us":47652,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":18131,"spinlock_wait_cycles":180619008,"thread_start_us":409290,"threads_started":1,"wal-append.queue_time_us":557598}
I20260709 12:13:42.202196 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.307165 11252 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:42.211063 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.209280 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.106714 14848 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:42.212882 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.311750 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.313980 11253 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:42.314210 11177 raft_consensus.cc:2410] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:13:42.121723 14797 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:42.219484 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.233791 14676 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.234571 14855 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:42.105767 14861 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 12:13:42.089370 14818 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.198976 14716 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:42.419196 11177 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:42.316942 14852 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.086452 14832 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.221387 14842 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:42.185875 14766 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.134980 14450 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.222968 14835 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.138127 14829 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.223613 14408 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.225083 14841 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.151134 14795 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.151439 14704 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:42.206868 14747 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:13:42.161849 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.092952 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:42.237689 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:42.235570 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.238735 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 571) took 1293 ms. Trace:
W20260709 12:13:42.234568 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 122031us
I20260709 12:13:42.085301 14733 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.119794 14729 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.307243 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.308043 11249 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:42.212237 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.308400 11180 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:42.199630 14824 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:42.310338 11181 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:42.312259 11932 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:13:42.312824 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 108949us
I20260709 12:13:42.315208 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.207175 14673 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 12:13:42.316219 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.217332 14773 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.317215 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.763339 11252 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:42.185369 14720 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.233598 14844 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.304798 11254 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:42.304347 14719 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.253698 14598 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:42.243335 14494 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.306386 14386 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.303134 14621 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:42.313460 14836 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:42.327788 14833 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:42.745903 14824 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.315879 14851 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:42.259220 14798 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.322099 14676 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:42.783067 14866 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.783565 14840 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.746459 14729 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.205367 14055 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:13:42.433225 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:42.321213 14797 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.449179 11177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:42.420477 14810 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.457791 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.459491 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 605) took 1861 ms. Trace:
I20260709 12:13:42.448763 14855 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:42.450137 14822 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 12:13:42.717284 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:42.578044 14794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.779s	user 0.023s	sys 0.015s
W20260709 12:13:42.727209 14819 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.010s
I20260709 12:13:42.727514 14831 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.782548 14829 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:42.727849 14858 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.368s	user 0.000s	sys 0.008s
I20260709 12:13:42.485949 14864 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.725415 14775 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.732981 14568 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.736027 14871 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:42.564412 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 198967us
I20260709 12:13:42.765419 14740 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:42.743765 14880 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:42.486537 14825 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.138s	user 0.005s	sys 0.009s
I20260709 12:13:42.484007 14716 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.738430 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:42.514932 14865 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.738626 14539 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.738673 14809 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.489187 14862 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.797155 11177 raft_consensus.cc:606] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:42.815790 11177 raft_consensus.cc:614] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:42.453943 14722 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 12:13:42.518061 14868 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:13:42.509766 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:42.829156 11326 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:42.470703 11930 rpcz_store.cc:276] 0709 12:13:40.945737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.945855 (+   118us) service_pool.cc:224] Handling call
0709 12:13:40.979009 (+ 33154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:41.593299 (+614290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.593336 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:41.625250 (+ 31914us) log.cc:844] Serialized 26 byte log entry
0709 12:13:42.069321 (+444071us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:42.069357 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.069438 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.238517 (+169079us) raft_consensus.cc:1685] finished
0709 12:13:42.238540 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.238685 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":705292,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":43262,"spinlock_wait_cycles":16768,"thread_start_us":367523,"threads_started":2,"wal-append.queue_time_us":499533}
I20260709 12:13:42.447299 14849 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:13:42.724597 14867 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.838579 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.739428 14832 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.741662 14852 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.743209 14753 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.744583 11175 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:42.809785 14861 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:42.815057 14445 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.752686 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.753898 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:42.853833 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 100007us
I20260709 12:13:42.855877 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:42.784576 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.750502 14599 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.785173 14747 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.785770 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.806232 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.864307 11935 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:42.864424 11178 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:42.786362 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:42.789253 14824 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:42.795265 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 362121us
I20260709 12:13:42.816892 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.797323 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:42.817795 11927 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:13:42.803440 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:42.804912 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:42.793926 14668 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.644s	user 0.001s	sys 0.019s
I20260709 12:13:42.793326 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.798123 11322 rpcz_store.cc:276] 0709 12:13:40.598045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.598207 (+   162us) service_pool.cc:224] Handling call
0709 12:13:40.608005 (+  9798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:40.852868 (+244863us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:40.853201 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 12:13:40.897403 (+ 44202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:40.897465 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:40.897525 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.831636 (+934111us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b18004477c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:42.223066 (+391430us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:42.459241 (+236175us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400429ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:42.459293 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.459427 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24972,"prepare.queue_time_us":133778,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":82029,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":25366,"spinlock_wait_cycles":634748800,"thread_start_us":43814,"threads_started":1,"wal-append.queue_time_us":172792}
I20260709 12:13:42.785997 14386 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:42.838047 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 99707.5us
I20260709 12:13:42.884156 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:42.801458 14794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.002s	user 0.024s	sys 0.019s
I20260709 12:13:42.797262 14810 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.802117 14829 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.890823 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:42.742029 11928 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:42.803328 14821 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:42.796880 14797 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.748107 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.800177 14822 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:42.787305 14773 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:42.828562 14835 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:42.838976 14716 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:42.865983 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:42.792110 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.864454 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.868621 14720 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:42.869066 14774 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:42.878057 14875 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:42.869783 14831 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.870543 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:42.874369 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:42.926528 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.875643 11326 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:42.786567 14621 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.880219 11932 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:13:42.788597 14798 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.803972 14740 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.803189 14568 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.822685 14809 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.787902 14833 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.887751 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:42.878229 14747 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.822551 14539 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.897966 11177 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:13:42.899660 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:42.900228 11176 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:42.789868 14494 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.870501 14852 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:42.802627 14864 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.791510 14704 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.924728 14887 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:42.803573 14719 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.916007 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:42.916133 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:42.964891 11330 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:42.787508 14836 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.790398 14598 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.012058 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:42.789757 14866 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.815148 14865 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.803463 14871 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.919881 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.788215 14851 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.869678 14729 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.852134 14753 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.924202 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:42.804283 14880 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.873123 14766 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:13:42.928819 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:43.044950 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 116228us
I20260709 12:13:43.045751 11330 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:43.046713 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:43.047159 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:43.047308 11330 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:42.928969 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:42.927492 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:43.048684 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 121286us
I20260709 12:13:42.913954 14861 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:42.846530 14867 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.799012 14855 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.916935 14835 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.933110 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.933357 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:42.933913 14733 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.891858 14824 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:13:42.901605 14821 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:42.892588 14815 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:13:42.833940 14862 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.939747 11254 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:42.937422 14676 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.871243 14445 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.887055 14668 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:42.951481 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 72367.7us
I20260709 12:13:42.921280 14773 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.790813 14840 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:42.958153 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58577.8us
I20260709 12:13:42.960325 11323 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:42.959614 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:42.927309 14774 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:42.926026 14720 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:42.979482 14740 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:43.024775 14887 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.080842 14676 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:42.983497 14568 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:42.986537 14859 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.852s	user 0.002s	sys 0.012s
W20260709 12:13:42.990418 14890 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:13:42.992600 14494 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:42.993525 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.104416 11178 raft_consensus.cc:2376] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9. Re-sending same reply.
I20260709 12:13:43.104806 11322 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:42.995779 14747 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:13:43.002231 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1546) took 1095 ms. Trace:
I20260709 12:13:43.006704 14408 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:43.112113 11325 rpcz_store.cc:276] 0709 12:13:41.906433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.240918 (+334485us) service_pool.cc:224] Handling call
0709 12:13:42.241253 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:42.305747 (+ 64494us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:42.422933 (+117186us) log.cc:844] Serialized 14 byte log entry
0709 12:13:42.743919 (+320986us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:42.744516 (+   597us) log.cc:844] Serialized 52 byte log entry
0709 12:13:42.744697 (+   181us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:42.744947 (+   250us) log.cc:844] Serialized 14 byte log entry
0709 12:13:42.745167 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.745252 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:43.001854 (+256602us) raft_consensus.cc:1685] finished
0709 12:13:43.001890 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.002137 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":260320,"prepare.queue_time_us":438559,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":263386,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":819,"spinlock_wait_cycles":107396608,"thread_start_us":86181,"threads_started":2,"wal-append.queue_time_us":276475}
I20260709 12:13:43.010896 14819 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:42.878399 14450 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.951812 14833 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:43.119458 14445 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.119841 14862 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.120909 14835 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:43.121783 14889 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.184s	user 0.003s	sys 0.000s
I20260709 12:13:43.020313 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:43.124469 11329 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:43.020637 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 119137us
I20260709 12:13:43.011716 14844 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:43.032387 11251 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:43.031479 11252 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:42.961933 14842 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:43.037571 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1548) took 1054 ms. Trace:
W20260709 12:13:43.036990 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 164189us
W20260709 12:13:43.021247 14886 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:13:42.938760 14831 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.039835 11249 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:13:43.024204 14875 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:43.058641 11330 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:43.030556 14719 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.064237 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.054868 14820 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:43.053062 14599 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:43.054522 14877 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:43.070686 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:43.073359 11174 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:43.073246 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.077621 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:42.980962 14386 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.083055 11248 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:13:43.082868 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:43.158501 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 87904.4us
I20260709 12:13:43.159283 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:43.079087 14773 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.084331 14720 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.068241 14818 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:43.072235 14892 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.060s	user 0.001s	sys 0.001s
I20260709 12:13:43.165145 14887 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.091854 14867 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.093534 14809 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.094228 14539 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.094596 14865 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.094825 14798 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.095078 14810 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.096194 14753 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:43.097000 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.068244 14824 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:13:43.098735 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.105276 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1738) took 2317 ms. Trace:
I20260709 12:13:43.052026 14729 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.116425 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.117100 14812 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:43.117691 14880 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.118330 14871 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.128317 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.072214 14821 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.140614 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.144641 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.147574 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.132877 11327 rpcz_store.cc:276] 0709 12:13:41.982862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.260648 (+277786us) service_pool.cc:224] Handling call
0709 12:13:42.261056 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.458149 (+197093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.458189 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:42.462330 (+  4141us) log.cc:844] Serialized 26 byte log entry
0709 12:13:42.843499 (+381169us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:42.843524 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.843626 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:43.037300 (+193674us) raft_consensus.cc:1685] finished
0709 12:13:43.037324 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.037499 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44326,"raft.queue_time_us":44997,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6883,"thread_start_us":256004,"threads_started":2,"wal-append.queue_time_us":459519}
I20260709 12:13:43.146138 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.151533 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.154876 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.123260 14819 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:43.165032 14668 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.165478 14877 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:43.064347 14891 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:13:43.132220 14844 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:43.165282 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 88395.2us
I20260709 12:13:43.167609 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.168524 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:43.185427 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.137115 14450 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.279768 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 619) took 1461 ms. Trace:
I20260709 12:13:43.349419 11322 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.311741 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:43.350275 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:43.351055 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.248075 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.351850 11252 raft_consensus.cc:2410] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:13:43.351945 11327 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:43.352139 11180 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:43.352676 11249 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:13:43.353394 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 649) took 2423 ms. Trace:
I20260709 12:13:43.354638 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.273849 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2163) took 2362 ms. Trace:
I20260709 12:13:43.355242 11257 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:43.355724 11175 rpcz_store.cc:276] 0709 12:13:40.911154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.911321 (+   167us) service_pool.cc:224] Handling call
0709 12:13:40.955230 (+ 43909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.849672 (+1894442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.273386 (+423714us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54000d9154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:43.273782 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4114641152}
I20260709 12:13:43.162909 14795 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:43.349663 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.351267 11330 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.358078 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.169611 14818 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.368031 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.368983 11179 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:43.350200 11177 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.353823 11926 rpcz_store.cc:276] 0709 12:13:40.892834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.897685 (+  4851us) service_pool.cc:224] Handling call
0709 12:13:40.950099 (+ 52414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.208728 (+1258629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.279360 (+1070632us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400540454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:43.316297 (+ 36937us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2726829824}
I20260709 12:13:43.370359 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:43.360857 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 216299us
I20260709 12:13:43.158869 14875 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.373304 14875 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:43.376543 11147 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.653s (SENT) [suppressed 17 similar messages]
I20260709 12:13:43.380548 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.212523 11329 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:43.370376 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.370599 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:43.354950 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 186513us
I20260709 12:13:43.163708 14820 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:43.384737 14773 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:43.393775 11926 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:43.263254 14539 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.386740 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:43.398768 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49221.2us
I20260709 12:13:43.387372 11932 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.388073 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.272131 14851 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.167609 14747 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:43.402797 11177 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:43.389295 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.389045 14816 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:43.403879 11181 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.401324 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1868) took 1171 ms. Trace:
I20260709 12:13:43.392104 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.392261 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1866) took 1163 ms. Trace:
I20260709 12:13:43.170539 14861 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:43.393872 11179 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:13:43.263697 14445 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.384619 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.385926 11249 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:43.408668 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.342073 14719 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:43.394155 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 82486us
I20260709 12:13:43.395279 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:43.395231 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:43.395679 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:43.397217 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.395839 14670 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:43.411499 11322 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:43.349774 11255 rpcz_store.cc:276] 0709 12:13:40.787468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.789294 (+  1826us) service_pool.cc:224] Handling call
0709 12:13:40.808976 (+ 19682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:41.027273 (+218297us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:41.073533 (+ 46260us) log.cc:844] Serialized 14 byte log entry
0709 12:13:42.805384 (+1731851us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b180040a020. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a
0709 12:13:43.061997 (+256613us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:43.067371 (+  5374us) log.cc:844] Serialized 26 byte log entry
0709 12:13:43.067502 (+   131us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:43.067524 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.067570 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:43.089798 (+ 22228us) raft_consensus.cc:1685] finished
0709 12:13:43.104938 (+ 15140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.105190 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":391641,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":220856,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":21389,"spinlock_wait_cycles":28073856,"thread_start_us":1469142,"threads_started":4,"wal-append.queue_time_us":1084472}
I20260709 12:13:43.350396 14824 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:43.349778 11331 rpcz_store.cc:276] 0709 12:13:41.818731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.147194 (+328463us) service_pool.cc:224] Handling call
0709 12:13:42.147553 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.797332 (+649779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.797380 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.797440 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:42.797487 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.279539 (+482052us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54001c52d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:43.279585 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.279709 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1493922688}
I20260709 12:13:43.361610 14795 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:43.385067 14740 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:43.314314 14864 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:43.271886 14869 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.546s	user 0.002s	sys 0.000s
I20260709 12:13:43.389254 14821 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:43.405264 11252 rpcz_store.cc:276] 0709 12:13:42.229207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.140179 (+910972us) service_pool.cc:224] Handling call
0709 12:13:43.392198 (+252019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.446386 11177 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:43.447368 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.404758 11257 rpcz_store.cc:276] 0709 12:13:42.229794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.386489 (+1156695us) service_pool.cc:224] Handling call
0709 12:13:43.401248 (+ 14759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.394001 14865 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.408658 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.398898 14539 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.409024 11929 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.409222 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1743) took 1176 ms. Trace:
I20260709 12:13:43.409425 11927 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.408690 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:43.409902 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.411688 11329 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:43.458658 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2168) took 1203 ms. Trace:
I20260709 12:13:43.457844 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.459275 11329 rpcz_store.cc:276] 0709 12:13:42.255719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.396920 (+1141201us) service_pool.cc:224] Handling call
0709 12:13:43.458558 (+ 61638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.360028 14812 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:43.457546 11249 rpcz_store.cc:276] 0709 12:13:42.233043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.385085 (+1152042us) service_pool.cc:224] Handling call
0709 12:13:43.385435 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.408972 (+ 23537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.409169 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.413269 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:43.462119 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 48917.5us
I20260709 12:13:43.400413 14815 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:43.464884 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.414269 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.415017 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:43.415791 11179 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:43.417526 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.403053 14668 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.420651 14840 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.403322 14747 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.398823 14819 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.427192 14386 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:43.428669 14904 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:13:43.430733 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1859) took 1228 ms. Trace:
I20260709 12:13:43.405679 14871 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.449265 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:43.486694 11248 rpcz_store.cc:276] 0709 12:13:42.202105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.202213 (+   108us) service_pool.cc:224] Handling call
0709 12:13:43.430468 (+1228255us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400277c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:43.430661 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775390464}
I20260709 12:13:43.395948 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:43.453140 11926 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:43.455902 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 79394.6us
I20260709 12:13:43.460227 11331 raft_consensus.cc:2410] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:13:43.464151 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:43.488794 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.468514 14861 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:43.465853 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.467677 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.463502 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:43.491569 14812 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:43.496372 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:43.496832 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.496749 14812 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:43.473011 14408 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:43.398900 14820 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.499361 14720 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:43.489425 14831 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.499923 14720 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.500311 14831 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:43.500841 14831 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.474588 14836 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:43.432940 14795 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:43.468675 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54513.4us
W20260709 12:13:43.469239 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55862.4us
I20260709 12:13:43.478554 14891 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:43.431926 14851 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:43.513078 14851 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:43.513700 14851 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:43.514014 14905 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:13:43.479557 14668 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.432118 14740 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.474326 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:43.478828 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.454808 14864 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:43.468079 14815 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.418872 14809 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:43.517306 14824 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.522265 14864 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.488070 11322 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:43.418589 14445 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.479382 14747 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:43.418382 14719 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.493186 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:43.494319 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1745) took 1256 ms. Trace:
I20260709 12:13:43.406553 14844 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:43.507474 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:43.469712 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59799.4us
I20260709 12:13:43.460718 14865 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.516984 11927 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
W20260709 12:13:43.588872 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 95778.8us
I20260709 12:13:43.516947 11255 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:13:43.522938 14740 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:43.523530 14668 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.505980 14871 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.530030 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.506268 14836 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:43.594216 11331 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:43.445410 14877 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.528292 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 621) took 1479 ms. Trace:
I20260709 12:13:43.506335 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:43.595449 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 89184.3us
I20260709 12:13:43.596472 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:43.568491 11248 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:13:43.598691 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1746) took 1360 ms. Trace:
I20260709 12:13:43.597476 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:43.594976 11934 rpcz_store.cc:276] 0709 12:13:42.049081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.049238 (+   157us) service_pool.cc:224] Handling call
0709 12:13:42.060704 (+ 11466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.077743 (+ 17039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.077777 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:42.085660 (+  7883us) log.cc:844] Serialized 26 byte log entry
0709 12:13:42.384976 (+299316us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:42.385041 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.385138 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:43.271525 (+886387us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b2400031148. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363e09d 00007f3ec627010e 00007f3ec628788c 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221
0709 12:13:43.527965 (+256440us) raft_consensus.cc:1685] finished
0709 12:13:43.527996 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.528215 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12195,"raft.queue_time_us":12537,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":5027,"spinlock_wait_cycles":4591872,"thread_start_us":252183,"threads_started":1,"wal-append.queue_time_us":326732}
I20260709 12:13:43.599129 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.525563 14829 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:43.560824 14851 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.530913 14809 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.532341 14539 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.593191 14775 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:43.590121 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:43.591859 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.634014 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.503361 14819 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:43.566123 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:43.641922 11322 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:43.642452 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 52415.1us
I20260709 12:13:43.505209 14902 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:43.557401 14864 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:43.560585 14408 consensus_queue.cc:1175] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fecfc03de52a4308bc9b045cad5aac1e" responder_term: 8 status { last_received { term: 8 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 8 index: 6 } }
I20260709 12:13:43.599265 11248 rpcz_store.cc:276] 0709 12:13:42.238735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.505477 (+1266742us) service_pool.cc:224] Handling call
0709 12:13:43.568144 (+ 62667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:43.598419 (+ 30275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.598633 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.610548 14773 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:43.519830 14891 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.596143 14835 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:13:43.585525 11249 rpcz_store.cc:276] 0709 12:13:42.238039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.493615 (+1255576us) service_pool.cc:224] Handling call
0709 12:13:43.493956 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.494097 (+   141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:43.494113 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:43.494139 (+    26us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:43.494157 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.494190 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.494277 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.530735 14815 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:43.567349 14729 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.634042 14775 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.557303 14831 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:43.618908 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.602130 14896 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:13:43.627501 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.628324 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1849) took 3057 ms. Trace:
I20260709 12:13:43.629231 11252 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:43.592971 14895 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:43.572072 11926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:13:43.634718 11934 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:43.591665 14821 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:43.596957 14836 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:43.638953 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47159.2us
I20260709 12:13:43.650328 14668 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:43.644057 11329 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:43.600736 14841 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:43.607905 14766 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:43.608361 14742 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:43.671164 11331 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:13:43.667877 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.671907 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1550) took 1613 ms. Trace:
I20260709 12:13:43.672224 11331 rpcz_store.cc:276] 0709 12:13:42.058183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.662016 (+1603833us) service_pool.cc:224] Handling call
0709 12:13:43.670922 (+  8906us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:43.671619 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.671857 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.630884 14809 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:43.651199 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 622) took 1603 ms. Trace:
I20260709 12:13:43.669968 14821 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:43.676337 14716 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.003s
I20260709 12:13:43.670157 14841 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:43.679174 14841 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:43.670349 14833 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.001s
I20260709 12:13:43.681214 14833 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.682305 14851 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.683035 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.683432 14833 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:43.668769 14668 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.683705 11932 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:43.684019 14668 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:43.662742 14775 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:43.678634 11322 rpcz_store.cc:276] 0709 12:13:42.048170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.648052 (+1599882us) service_pool.cc:224] Handling call
0709 12:13:43.650402 (+  2350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.650684 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:43.650726 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:43.650794 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:43.650848 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.650936 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.651110 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.687651 14821 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.671358 14742 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.689832 14450 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.690116 14742 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:43.690788 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 618) took 1883 ms. Trace:
I20260709 12:13:43.691509 11253 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 5.
I20260709 12:13:43.691170 11328 rpcz_store.cc:276] 0709 12:13:41.807117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.127444 (+320327us) service_pool.cc:224] Handling call
0709 12:13:42.205927 (+ 78483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.790990 (+585063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.791032 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.791069 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:42.791102 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.690568 (+899466us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54000e5e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:43.690613 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.690726 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345081216}
I20260709 12:13:43.693122 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:43.693735 11328 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:43.693845 14621 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.694331 14445 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:43.695555 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1749) took 1374 ms (client timeout 1596 ms). Trace:
W20260709 12:13:43.695990 11251 rpcz_store.cc:269] 0709 12:13:42.321378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.457094 (+135716us) service_pool.cc:224] Handling call
0709 12:13:43.695352 (+1238258us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540033ccd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:43.695495 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497013120}
I20260709 12:13:43.697176 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.670476 14896 consensus_queue.cc:1243] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:13:43.698897 11174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1880) took 1502 ms (client timeout 1753 ms). Trace:
I20260709 12:13:43.699028 14871 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.670020 14766 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:43.699299 11174 rpcz_store.cc:269] 0709 12:13:42.196607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.235291 (+ 38684us) service_pool.cc:224] Handling call
0709 12:13:43.698668 (+1463377us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54003480d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:43.698815 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769422720}
I20260709 12:13:43.699972 11328 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:43.700075 14766 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:43.700985 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2166) took 1446 ms. Trace:
I20260709 12:13:43.703931 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.704617 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:43.704551 14775 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.706279 14719 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:43.707166 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2153) took 3752 ms. Trace:
I20260709 12:13:43.669234 14895 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.707594 14740 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.707609 11324 rpcz_store.cc:276] 0709 12:13:39.954529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.085812 (+131283us) service_pool.cc:224] Handling call
0709 12:13:40.086237 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.122172 (+2035935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.706922 (+1584750us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54001c3ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:43.707103 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4557140608}
I20260709 12:13:43.598232 14704 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:13:43.608150 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.712450 11251 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:43.658524 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:43.658540 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.659907 11255 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:43.655624 14891 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:43.665205 11926 raft_consensus.cc:606] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:43.661484 11256 rpcz_store.cc:276] 0709 12:13:40.570907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.571034 (+   127us) service_pool.cc:224] Handling call
0709 12:13:40.581908 (+ 10874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.852910 (+1271002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.628060 (+1775150us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540002e9d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:43.628259 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2736270592}
I20260709 12:13:43.650519 14861 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:43.644722 14902 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:43.666723 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:43.605537 14901 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:13:43.650434 14720 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.646126 14539 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:13:43.701308 11328 rpcz_store.cc:276] 0709 12:13:42.255017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.699491 (+1444474us) service_pool.cc:224] Handling call
0709 12:13:43.699808 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.700740 (+   932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.700928 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.711681 11324 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:13:43.611691 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:43.713474 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.730527 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2167) took 1475 ms. Trace:
I20260709 12:13:43.714087 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.715766 11926 raft_consensus.cc:614] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:43.731076 11324 rpcz_store.cc:276] 0709 12:13:42.255401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.711268 (+1455867us) service_pool.cc:224] Handling call
0709 12:13:43.711521 (+   253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:43.730172 (+ 18651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.730449 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.531214 14824 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:43.724534 11331 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [8.5-9.6]
I20260709 12:13:43.736378 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1551) took 1641 ms. Trace:
I20260709 12:13:43.737136 11331 rpcz_store.cc:276] 0709 12:13:42.094888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.672837 (+1577949us) service_pool.cc:224] Handling call
0709 12:13:43.724131 (+ 51294us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:43.735928 (+ 11797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.736284 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.732515 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:43.720739 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 106291us
I20260709 12:13:43.742138 11324 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:43.747536 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:43.721998 14833 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:43.750866 14740 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.751432 14866 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.752045 14865 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:43.755849 11327 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:43.756582 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2170) took 1500 ms. Trace:
I20260709 12:13:43.756911 11327 rpcz_store.cc:276] 0709 12:13:42.256307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.755325 (+1499018us) service_pool.cc:224] Handling call
0709 12:13:43.755659 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.756321 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.756522 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:43.759975 11326 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:13:43.782315 14895 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:44.111567 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 503485us
I20260709 12:13:44.097143 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:44.096576 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2174) took 1838 ms. Trace:
W20260709 12:13:43.916237 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1753) took 1468 ms (client timeout 1713 ms). Trace:
W20260709 12:13:44.113901 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1874) took 1922 ms (client timeout 1595 ms). Trace:
W20260709 12:13:44.099139 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.792428 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:44.115621 11323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2177) took 1809 ms (client timeout 1959 ms). Trace:
I20260709 12:13:44.107990 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:44.110319 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 443624us
W20260709 12:13:44.114295 11254 rpcz_store.cc:269] 0709 12:13:42.448037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.791818 (+343781us) service_pool.cc:224] Handling call
0709 12:13:43.915913 (+1124095us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b540010b654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:43.916141 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56730880}
I20260709 12:13:44.117247 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:44.117851 11249 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:13:43.725792 14902 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:43.799088 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.120121 11328 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:44.111073 11256 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:13:44.113773 11326 rpcz_store.cc:276] 0709 12:13:42.258044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.759384 (+1501340us) service_pool.cc:224] Handling call
0709 12:13:43.759749 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:43.761061 (+  1312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.096511 (+335450us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:44.111907 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:44.123286 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.112954 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:44.123399 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1751) took 1702 ms. Trace:
I20260709 12:13:44.123823 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:44.116251 11323 rpcz_store.cc:269] 0709 12:13:42.305807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.865678 (+559871us) service_pool.cc:224] Handling call
0709 12:13:44.115449 (+1249771us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b540042aed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:44.115563 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105374976}
I20260709 12:13:44.125031 11326 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
W20260709 12:13:44.115016 11182 rpcz_store.cc:269] 0709 12:13:42.191040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.191147 (+   107us) service_pool.cc:224] Handling call
0709 12:13:44.113683 (+1922536us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b540016eed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:44.113838 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926363648}
I20260709 12:13:44.127013 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.127898 11326 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:44.128504 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 446) took 1290 ms. Trace:
I20260709 12:13:44.128930 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:44.128887 11249 rpcz_store.cc:276] 0709 12:13:42.838335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.117244 (+1278909us) service_pool.cc:224] Handling call
0709 12:13:44.117538 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.118412 (+   874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.128441 (+ 10029us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:44.130735 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 387401us
I20260709 12:13:44.131780 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:43.731846 14450 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:13:44.052454 14860 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.910s	user 0.000s	sys 0.005s
I20260709 12:13:44.135361 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:44.125070 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.124794 11256 rpcz_store.cc:276] 0709 12:13:42.421327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.748708 (+1327381us) service_pool.cc:224] Handling call
0709 12:13:44.110848 (+362140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.123124 (+ 12276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.123345 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.136677 11251 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:13:44.138937 11326 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:13:44.139708 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2175) took 1881 ms. Trace:
I20260709 12:13:44.140028 11326 rpcz_store.cc:276] 0709 12:13:42.258516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.138375 (+1879859us) service_pool.cc:224] Handling call
0709 12:13:44.138704 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.139443 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.139653 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.141079 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.143146 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.133653 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:44.143879 11323 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:44.144753 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.145359 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:44.147090 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:44.147943 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.156309 11254 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:13:44.157087 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.158049 11328 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:44.159914 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.160311 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:44.160905 11256 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:44.160956 14742 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:43.839684 14775 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:44.163008 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65964.8us
I20260709 12:13:44.118963 14896 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.030220 14895 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:44.180598 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:44.161795 11323 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:13:44.177235 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:44.231392 11249 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:13:44.127637 14861 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:44.175405 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:44.232499 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55343.9us
I20260709 12:13:44.135221 14831 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:44.240578 11331 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:44.084275 14747 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.095935 14841 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.842805 14668 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:43.839921 14766 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.232684 14816 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:44.177048 14864 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:44.158946 14875 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:44.247044 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:44.247283 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.080060 14815 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.158764 14445 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:44.257896 14450 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:44.246016 14896 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.232326 14802 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:13:44.251487 14670 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:44.251857 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:44.251956 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:44.319849 11932 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.137807 14833 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:44.168392 14704 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.331866 11932 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:44.175127 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:44.260603 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1256736220000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
I20260709 12:13:44.253221 14895 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.257586 14840 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:44.342036 14840 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.343411 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.344295 14840 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:44.346499 14896 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.348107 14818 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:44.348124 14668 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.263648 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 578) took 1135 ms. Trace:
W20260709 12:13:44.215986 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.154868 14835 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 12:13:44.262836 11934 raft_consensus.cc:2955] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:44.350369 11928 rpcz_store.cc:276] 0709 12:13:43.128490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.129020 (+   530us) service_pool.cc:224] Handling call
0709 12:13:43.129451 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.160134 (+ 30683us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:43.412197 (+252063us) log.cc:844] Serialized 14 byte log entry
0709 12:13:43.478004 (+ 65807us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:43.478074 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:43.478130 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.263334 (+785204us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b54000a7154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:44.263397 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.263572 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46289,"prepare.queue_time_us":317959,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":15540,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":2840,"spinlock_wait_cycles":69932800,"thread_start_us":290881,"threads_started":2,"wal-append.queue_time_us":116111}
I20260709 12:13:44.276563 11248 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:44.276827 11323 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:44.272727 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 359231us
I20260709 12:13:44.357054 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.358120 11323 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:44.278079 11326 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:44.359364 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.154748 14740 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:44.281894 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.235153 14861 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:44.294729 11929 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:44.296624 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.363344 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:44.255824 14796 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:44.293334 14766 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.252750 14831 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.326500 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:44.330520 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 78741.2us
I20260709 12:13:44.367735 11256 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.158126 14722 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:13:44.291093 14747 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.289608 14841 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.278508 14816 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:44.241837 14775 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:44.364598 14861 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:44.340895 14891 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.001s
I20260709 12:13:44.291633 14815 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:44.319444 14670 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:44.444201 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:44.355741 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.279628 11174 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:44.447464 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:44.443804 14809 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.770s	user 0.009s	sys 0.001s
I20260709 12:13:44.372400 14796 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.364410 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.365299 11323 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:44.463378 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.447472 14840 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:44.368547 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:44.333822 14833 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:44.388003 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.389590 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1747) took 2150 ms. Trace:
I20260709 12:13:44.485423 11255 rpcz_store.cc:276] 0709 12:13:42.239398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.757830 (+1518432us) service_pool.cc:224] Handling call
0709 12:13:43.760962 (+  3132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.761191 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:44.113063 (+351872us) log.cc:844] Serialized 14 byte log entry
0709 12:13:44.387784 (+274721us) log.cc:844] Serialized 14 byte log entry
0709 12:13:44.389085 (+  1301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.389161 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:44.389225 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.389324 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.389498 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":530217,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":4112,"raft.queue_time_us":1127,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6405,"spinlock_wait_cycles":153436416,"thread_start_us":587348,"threads_started":2}
I20260709 12:13:44.489017 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:44.373616 14824 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.001s
I20260709 12:13:44.445189 11173 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:13:44.460939 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 110503us
I20260709 12:13:44.492125 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.492931 11932 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:44.337378 14720 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:44.464080 11323 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:44.293006 14450 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.471666 11256 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:44.499611 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:44.447899 14891 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.463384 14809 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.470752 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.472573 14766 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:44.472575 14840 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:44.335156 11328 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
I20260709 12:13:44.473927 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:44.513463 11222 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.559s (SENT)
I20260709 12:13:44.480600 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.502875 14833 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:44.490347 11255 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:44.520628 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.521270 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:44.522100 11255 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:44.523880 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:44.471876 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 103465us
I20260709 12:13:44.525005 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:44.506079 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:44.371079 14795 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.826s	user 0.008s	sys 0.005s
I20260709 12:13:44.515581 11173 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:13:44.496033 14720 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.503399 14450 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.494710 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:44.503167 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.521317 11928 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:44.469408 14670 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.506940 14895 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.526947 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:44.574827 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 47958.3us
W20260709 12:13:44.575906 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.576095 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.575012 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.578163 11175 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:44.574131 11173 pending_rounds.cc:85] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 9
I20260709 12:13:44.577127 11933 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:44.579391 11323 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:13:44.579501 11173 pending_rounds.cc:107] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:13:44.527861 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.580667 11173 raft_consensus.cc:2889] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:44.535868 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:44.507889 14766 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.535828 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:44.582777 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 402964us
I20260709 12:13:44.593920 11932 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:44.584208 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1553) took 1849 ms. Trace:
I20260709 12:13:44.594884 11323 rpcz_store.cc:276] 0709 12:13:42.739448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.578595 (+1839147us) service_pool.cc:224] Handling call
0709 12:13:44.579064 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.583902 (+  4838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.584146 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:44.597093 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.497252 14816 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:44.602627 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.606362 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:44.610102 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2172) took 2353 ms. Trace:
I20260709 12:13:44.610574 11327 rpcz_store.cc:276] 0709 12:13:42.256898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.759309 (+1502411us) service_pool.cc:224] Handling call
0709 12:13:44.099880 (+340571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.100349 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.100389 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.100925 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.375063 (+274138us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:44.375135 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.375796 (+   661us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.609875 (+234079us) raft_consensus.cc:1685] finished
0709 12:13:44.609895 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.610026 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":273638,"threads_started":1,"wal-append.queue_time_us":503271}
I20260709 12:13:44.544014 14896 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:44.582419 14720 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:44.619475 14809 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:44.621592 14923 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.172s	user 0.001s	sys 0.001s
I20260709 12:13:44.621820 14704 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:44.623558 14796 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:44.624435 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2169) took 2368 ms. Trace:
I20260709 12:13:44.599839 14766 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.624884 11324 rpcz_store.cc:276] 0709 12:13:42.256011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.743584 (+1487573us) service_pool.cc:224] Handling call
0709 12:13:44.095010 (+351426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.095418 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.095447 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.095932 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.327872 (+231940us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:44.499215 (+171343us) log.cc:844] Serialized 14 byte log entry
0709 12:13:44.499438 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.499523 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.624196 (+124673us) raft_consensus.cc:1685] finished
0709 12:13:44.624219 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.624367 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":176831,"raft.queue_time_us":4074,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":3190,"spinlock_wait_cycles":11395840,"thread_start_us":379980,"threads_started":2,"wal-append.queue_time_us":352944}
I20260709 12:13:44.627107 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:44.629004 11324 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:44.630208 11932 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:13:44.614763 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.631387 11928 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:44.631930 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 621) took 2627 ms. Trace:
I20260709 12:13:44.566831 14599 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:44.511731 14902 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.583904 14775 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:44.616847 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:44.617799 11327 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:13:44.632437 11329 rpcz_store.cc:276] 0709 12:13:42.004739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.511968 (+1507229us) service_pool.cc:224] Handling call
0709 12:13:43.512410 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.668919 (+156509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.631706 (+962787us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400285ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:44.631874 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302050560}
I20260709 12:13:44.637058 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1560) took 1775 ms. Trace:
I20260709 12:13:44.637405 11327 rpcz_store.cc:276] 0709 12:13:42.862113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.612377 (+1750264us) service_pool.cc:224] Handling call
0709 12:13:44.612739 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.636816 (+ 24077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.637006 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.597534 14670 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:44.637864 11329 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:44.638808 11327 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:13:44.646219 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2181) took 1600 ms. Trace:
I20260709 12:13:44.646219 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2180) took 1724 ms. Trace:
I20260709 12:13:44.639410 14599 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:44.640785 11935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 652) took 1824 ms (client timeout 1644 ms). Trace:
I20260709 12:13:44.594362 14795 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.643766 14774 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.639535 14815 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:44.646647 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1561) took 1766 ms. Trace:
I20260709 12:13:44.646734 11327 rpcz_store.cc:276] 0709 12:13:43.046183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.638170 (+1591987us) service_pool.cc:224] Handling call
0709 12:13:44.638486 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.645952 (+  7466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.646156 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.648141 11329 rpcz_store.cc:276] 0709 12:13:42.921949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.637275 (+1715326us) service_pool.cc:224] Handling call
0709 12:13:44.637563 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.645952 (+  8389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.646156 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.631121 14896 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.659212 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.646880 14809 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.656165 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 627) took 1526 ms. Trace:
W20260709 12:13:44.651508 11935 rpcz_store.cc:269] 0709 12:13:42.816626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.818262 (+  1636us) service_pool.cc:224] Handling call
0709 12:13:44.640524 (+1822262us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b5400780354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:44.640675 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56863488}
I20260709 12:13:44.656165 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2182) took 1609 ms. Trace:
I20260709 12:13:44.656621 11324 rpcz_store.cc:276] 0709 12:13:42.879774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.628241 (+1748467us) service_pool.cc:224] Handling call
0709 12:13:44.628623 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.629409 (+   786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.646579 (+ 17170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.663915 11329 rpcz_store.cc:276] 0709 12:13:43.130243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.655390 (+1525147us) service_pool.cc:224] Handling call
0709 12:13:44.655707 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.655866 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.655891 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.655931 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:44.655959 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.656008 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.656114 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38400}
I20260709 12:13:44.666605 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.664108 11327 rpcz_store.cc:276] 0709 12:13:43.046497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.655359 (+1608862us) service_pool.cc:224] Handling call
0709 12:13:44.655707 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.655866 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.655891 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.655931 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:44.655959 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.656008 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.656104 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19712}
I20260709 12:13:44.664655 11173 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:13:44.581425 14840 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:44.670053 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 227005us
I20260709 12:13:44.671188 11329 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:44.671913 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 629) took 1259 ms. Trace:
I20260709 12:13:44.672417 11329 rpcz_store.cc:276] 0709 12:13:43.413025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.670224 (+1257199us) service_pool.cc:224] Handling call
0709 12:13:44.670518 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.671689 (+  1171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.671861 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.673730 11327 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:44.661829 14896 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.674362 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 631) took 1252 ms. Trace:
I20260709 12:13:44.676662 14450 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:44.677933 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.685235 11933 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:44.687295 14795 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:44.665235 14836 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.970s	user 0.004s	sys 0.009s
I20260709 12:13:44.690382 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:44.697122 14840 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:44.695602 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.703933 11177 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:44.699745 11926 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 4.
I20260709 12:13:44.700522 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:44.690871 11329 rpcz_store.cc:276] 0709 12:13:43.422019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.673328 (+1251309us) service_pool.cc:224] Handling call
0709 12:13:44.673698 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.673872 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.673897 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.674124 (+   227us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:44.674161 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.674206 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.674314 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.664588 14774 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:13:44.707357 11930 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:44.707835 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.707836 14774 consensus_queue.cc:1243] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:13:44.664865 14809 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.711520 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:44.716512 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 628) took 1586 ms. Trace:
I20260709 12:13:44.717307 11324 rpcz_store.cc:276] 0709 12:13:43.130608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.664990 (+1534382us) service_pool.cc:224] Handling call
0709 12:13:44.665300 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.665500 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:44.665779 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 12:13:44.716175 (+ 50396us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.716233 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:44.716275 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.716338 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.716454 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":537,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":2075,"raft.queue_time_us":99,"thread_start_us":50056,"threads_started":1}
I20260709 12:13:44.718487 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.719131 14599 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:44.719545 11324 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:44.719646 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.720279 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.720759 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:44.721316 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.721689 11324 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:13:44.722024 11175 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:44.722167 11177 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:13:44.719175 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.723294 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:44.722545 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:44.724161 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2176) took 2418 ms. Trace:
I20260709 12:13:44.724377 11324 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:44.725482 11324 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.724552 11326 rpcz_store.cc:276] 0709 12:13:42.305511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.522619 (+2217108us) service_pool.cc:224] Handling call
0709 12:13:44.522906 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.523303 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.523341 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.523799 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.658038 (+134239us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:44.658097 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.658202 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.723969 (+ 65767us) raft_consensus.cc:1685] finished
0709 12:13:44.723988 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.724107 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54400,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":9590,"thread_start_us":132619,"threads_started":1,"wal-append.queue_time_us":185913}
I20260709 12:13:44.729709 11329 raft_consensus.cc:2955] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:13:44.731187 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.735409 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:44.736172 11324 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:13:44.736634 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:44.737455 11250 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:44.732084 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.738921 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1760) took 1361 ms. Trace:
I20260709 12:13:44.739642 14668 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.740345 14668 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.740665 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:44.742316 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:44.739010 14599 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:44.743341 14599 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.744503 14774 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:44.745065 14774 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:44.704334 14836 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.745946 14774 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:44.743592 14722 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:44.747079 14722 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.747972 11257 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:44.732041 14902 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:44.739614 11255 rpcz_store.cc:276] 0709 12:13:43.377938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.522595 (+1144657us) service_pool.cc:224] Handling call
0709 12:13:44.522899 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.523292 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.523331 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.523799 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.652941 (+129142us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:44.653002 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.653115 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.738647 (+ 85532us) raft_consensus.cc:1685] finished
0709 12:13:44.738678 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.738843 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4356,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12314,"thread_start_us":126188,"threads_started":1,"wal-append.queue_time_us":188044}
I20260709 12:13:44.747483 14722 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:44.763765 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:44.764674 11255 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:44.755024 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.766598 11175 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:44.745486 14599 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:44.793663 14865 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:44.797973 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:44.799450 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.800083 11173 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:44.800168 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.800980 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.793744 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.802809 11179 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:44.803139 14865 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:44.803544 14865 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:44.803668 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.804819 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1556) took 2064 ms. Trace:
I20260709 12:13:44.805281 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:44.805879 11180 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:44.806245 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1763) took 1427 ms (client timeout 1706 ms). Trace:
W20260709 12:13:44.806654 11252 rpcz_store.cc:269] 0709 12:13:43.378866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.446995 (+ 68129us) service_pool.cc:224] Handling call
0709 12:13:44.806094 (+1359099us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54000f8cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:44.806191 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415528832}
I20260709 12:13:44.807806 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:44.804431 14865 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:44.809127 14865 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:44.805234 11328 rpcz_store.cc:276] 0709 12:13:42.740329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.581851 (+1841522us) service_pool.cc:224] Handling call
0709 12:13:44.611330 (+ 29479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.615535 (+  4205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.615686 (+   151us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.616334 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.693852 (+ 77518us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:44.693900 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.693977 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.804617 (+110640us) raft_consensus.cc:1685] finished
0709 12:13:44.804636 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.804763 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":18394,"thread_start_us":75171,"threads_started":1,"wal-append.queue_time_us":179278}
I20260709 12:13:44.822840 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:44.825084 11324 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:44.831312 14668 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:44.755229 14836 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.063752 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.079844 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.081302 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.082485 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.083526 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.084129 11175 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:45.084930 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.085615 11175 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:45.086519 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.087512 11175 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:13:45.088348 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.089524 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.090659 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.091728 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.092501 11175 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 7: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:45.093161 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.094013 11175 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:13:45.094767 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.096427 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.098317 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.102290 14815 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:45.106665 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1552) took 3011 ms. Trace:
I20260709 12:13:45.107131 11322 rpcz_store.cc:276] 0709 12:13:42.095285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.686708 (+1591423us) service_pool.cc:224] Handling call
0709 12:13:44.115588 (+428880us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:44.115987 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:44.506497 (+390510us) log.cc:844] Serialized 14 byte log entry
0709 12:13:44.715323 (+208826us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:44.764038 (+ 48715us) log.cc:844] Serialized 52 byte log entry
0709 12:13:44.764380 (+   342us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:44.764437 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.764538 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.801486 (+ 36948us) raft_consensus.cc:1685] finished
0709 12:13:44.801517 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.106455 (+304938us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540042aed0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:45.106596 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":884,"prepare.queue_time_us":342076,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":11274,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":9518,"spinlock_wait_cycles":985390976,"thread_start_us":505717,"threads_started":2,"wal-append.queue_time_us":247467}
W20260709 12:13:45.140671 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.148699 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.149472 11250 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:45.150451 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.151011 11250 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:44.804833 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.155879 14729 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:45.156330 14729 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:45.158299 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:45.163354 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:45.170100 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.173803 14815 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:45.174414 14815 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.176530 14815 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.179224 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.180044 11255 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:45.180941 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.182642 11179 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:45.181537 11255 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:45.205255 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.205976 11255 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:45.207885 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1759) took 1830 ms. Trace:
I20260709 12:13:45.157171 14729 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.208380 11249 rpcz_store.cc:276] 0709 12:13:43.377583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.129762 (+752179us) service_pool.cc:224] Handling call
0709 12:13:44.130138 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.234859 (+104721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.207545 (+972686us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400109854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:45.207805 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232186240}
I20260709 12:13:45.209996 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.211136 11249 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:45.212481 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.215473 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.216321 11249 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:45.220501 11178 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2176) took 2342 ms (client timeout 1559 ms). Trace:
W20260709 12:13:45.221010 11178 rpcz_store.cc:269] 0709 12:13:42.878465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.928714 (+ 50249us) service_pool.cc:224] Handling call
0709 12:13:45.220252 (+2291538us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b54002323d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:45.220396 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128921856}
I20260709 12:13:45.225554 14450 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.209497 14729 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:45.235185 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:45.235877 11328 raft_consensus.cc:2376] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9. Re-sending same reply.
W20260709 12:13:45.251458 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.257102 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1744) took 3019 ms. Trace:
I20260709 12:13:45.259029 14720 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.265883 11181 raft_consensus.cc:2368] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:45.269330 11257 rpcz_store.cc:276] 0709 12:13:42.237293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.492447 (+1255154us) service_pool.cc:224] Handling call
0709 12:13:43.492845 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.748523 (+1255678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.256740 (+508217us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540007ec54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:45.257010 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2885844224}
I20260709 12:13:45.272223 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 524) took 1118 ms. Trace:
I20260709 12:13:45.277218 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.273507 14720 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.273790 11182 rpcz_store.cc:276] 0709 12:13:44.154031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.155972 (+  1941us) service_pool.cc:224] Handling call
0709 12:13:44.174933 (+ 18961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.276259 (+101326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.276303 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.276365 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:44.276411 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.271926 (+995515us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54002e40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:45.271984 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.272155 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232449920}
I20260709 12:13:45.280423 11257 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:45.272136 11180 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:13:45.284654 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.290871 14809 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:45.291750 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.296721 14742 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.302361 11249 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:45.305030 14844 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:45.331148 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:45.331620 14877 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:45.332499 14877 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:45.332876 14877 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.333202 14877 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:44.840519 14896 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.333807 14809 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:45.334242 14809 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:45.335114 14809 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:45.312628 14840 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.336308 14809 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:45.336802 14840 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:45.337975 11177 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:45.339136 14840 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:45.316708 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:45.341202 11328 raft_consensus.cc:2376] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8. Re-sending same reply.
W20260709 12:13:45.320072 14861 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.872s	user 0.016s	sys 0.000s
I20260709 12:13:45.342521 14861 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.345058 14861 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:45.354277 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.355156 11322 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 6.
W20260709 12:13:45.355481 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 532690us
I20260709 12:13:45.356103 14445 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.356627 14445 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.336319 14896 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.359087 14445 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.355347 14824 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:45.359834 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:45.360595 11328 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.358903 14720 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:45.362358 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.351802 14877 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.320839 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.328191 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:44.735471 14833 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:45.362645 14720 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:45.400189 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.363078 11254 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:45.405728 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:45.363379 11248 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1768) took 1754 ms (client timeout 1574 ms). Trace:
W20260709 12:13:45.408334 11248 rpcz_store.cc:269] 0709 12:13:43.608810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.658140 (+ 49330us) service_pool.cc:224] Handling call
0709 12:13:45.363175 (+1705035us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400109d54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:45.363312 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390023936}
I20260709 12:13:45.409623 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:45.411713 11331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1562) took 1698 ms (client timeout 2053 ms). Trace:
I20260709 12:13:45.412123 11248 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:45.414196 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 653) took 2588 ms. Trace:
I20260709 12:13:45.367630 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.414613 11930 rpcz_store.cc:276] 0709 12:13:42.826152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.630315 (+1804163us) service_pool.cc:224] Handling call
0709 12:13:44.630588 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.708063 (+ 77475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.413937 (+705874us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b54003b5454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:45.414133 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176056576}
I20260709 12:13:45.415396 14836 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:13:45.416446 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69072.1us
W20260709 12:13:45.419114 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.424417 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.418641 14877 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:45.434919 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:45.368321 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:45.436596 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.438216 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:45.438865 11929 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:45.390230 11257 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:45.441771 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.442281 11257 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:45.392787 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.446092 11256 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:45.447892 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:45.448513 11930 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:13:45.448835 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.429268 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:45.449290 11178 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:45.388940 11323 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:45.449484 11181 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:45.365422 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:45.412189 11331 rpcz_store.cc:269] 0709 12:13:43.713091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.738469 (+ 25378us) service_pool.cc:224] Handling call
0709 12:13:45.411512 (+1673043us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400331dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:45.411650 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284925184}
I20260709 12:13:45.450891 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.449965 11178 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:45.451076 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 85814.6us
I20260709 12:13:45.386584 11176 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:45.404206 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:45.416405 11250 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
W20260709 12:13:45.455557 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51418.4us
I20260709 12:13:45.457168 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.457911 11181 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:45.442886 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.458724 11257 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:45.458863 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.454478 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.463255 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.466805 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.467999 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.464120 11176 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:45.451881 11178 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:45.472795 11178 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.476784 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:45.466619 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:45.477444 11929 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.470788 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:45.479640 11255 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:13:45.479264 11328 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
W20260709 12:13:45.387475 14670 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.005s
I20260709 12:13:45.482014 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.482481 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.489854 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:45.490525 11927 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:45.496762 11255 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:45.474825 14676 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:13:45.463601 14861 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:45.501596 14840 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:45.502895 14891 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:45.484715 14875 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:45.503968 14875 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.504317 14875 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:45.504355 14670 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:45.505846 14747 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.505723 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.506042 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.506548 11328 raft_consensus.cc:2376] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10. Re-sending same reply.
I20260709 12:13:45.393832 14833 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.501711 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.505754 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.506738 11330 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:45.508096 11929 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:45.464226 14796 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:45.508260 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.511418 11178 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:13:45.511885 11178 pending_rounds.cc:85] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 5
I20260709 12:13:45.512216 11178 pending_rounds.cc:107] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:13:45.512565 11178 raft_consensus.cc:2889] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:45.514510 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.516958 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.471895 14818 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:45.515166 11248 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:45.503510 14896 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:45.498580 14820 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:45.519563 11322 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:45.521997 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.525143 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.487344 14815 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:13:45.510149 14796 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.513582 14676 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:45.528522 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.506997 14598 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:13:45.529340 14408 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:45.543681 14829 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:45.527407 14833 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:45.525179 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.522676 14896 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:45.535521 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 630) took 2114 ms. Trace:
I20260709 12:13:45.531121 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:45.546554 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.547144 11255 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:45.524812 14820 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.515766 14747 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:45.550910 14747 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.547048 11327 rpcz_store.cc:276] 0709 12:13:43.421119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.668568 (+1247449us) service_pool.cc:224] Handling call
0709 12:13:44.668865 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.674130 (+  5265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.535268 (+861138us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b5400429ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:45.535456 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9318144}
I20260709 12:13:45.551832 14820 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:45.552872 14766 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.926s	user 0.008s	sys 0.005s
I20260709 12:13:45.546438 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.548400 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.538599 14742 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:45.539283 14796 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:45.555011 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.547230 11248 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:45.554550 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.555402 11255 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:13:45.556468 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.556604 11254 raft_consensus.cc:2410] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:13:45.562337 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.522836 14818 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.539547 14676 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:45.557500 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:45.565793 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.568799 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1565) took 1460 ms. Trace:
I20260709 12:13:45.569288 11329 rpcz_store.cc:276] 0709 12:13:44.108658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.706751 (+598093us) service_pool.cc:224] Handling call
0709 12:13:44.707081 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.715992 (+  8911us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:44.817752 (+101760us) log.cc:844] Serialized 14 byte log entry
0709 12:13:45.362321 (+544569us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:45.362808 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:13:45.362944 (+   136us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:45.362974 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.363052 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:45.568584 (+205532us) raft_consensus.cc:1685] finished
0709 12:13:45.568605 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.568737 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69879,"raft.queue_time_us":72595,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":12435,"spinlock_wait_cycles":1941760,"thread_start_us":65667,"threads_started":1,"wal-append.queue_time_us":720740}
I20260709 12:13:45.571902 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:45.562925 11255 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.547744 14720 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:45.546644 14833 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:45.557587 14742 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:45.588624 14742 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:45.559361 14747 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:45.580192 14829 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:45.553334 14766 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.586589 14833 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:45.601146 11255 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:45.603439 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:45.597496 14829 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.607448 11255 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:45.608319 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:45.608867 11255 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:45.599045 14820 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:45.616773 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 133046us
I20260709 12:13:45.594344 14896 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.618467 14896 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:45.620875 14815 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:45.622248 11900 leader_election.cc:400] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:45.622591 11900 leader_election.cc:403] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:45.623467 11929 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:13:45.625280 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:45.625602 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:45.626785 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:45.623410 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 136271us
I20260709 12:13:45.631435 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:45.616647 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.635545 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:45.634842 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:45.614216 14829 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:45.639882 11322 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:45.639978 14598 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:13:45.627363 14861 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:45.638244 11934 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:13:45.637228 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:45.641975 11327 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:45.633291 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:45.636859 11329 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:45.636487 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.621491 14815 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.646524 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.644151 14408 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:45.610682 14676 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.655073 14818 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:45.657749 14816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.013s	sys 0.010s
I20260709 12:13:45.639011 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:45.651073 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.660450 11330 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:45.651829 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.661945 11331 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:45.649212 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.666522 14445 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:45.667361 14820 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:45.660328 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:45.663434 14816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.013s	sys 0.010s
I20260709 12:13:45.641278 14747 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:45.670223 14766 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.637846 14891 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:45.670253 14945 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:13:45.668146 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:45.689504 11327 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:45.690065 11322 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:45.681977 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:45.684724 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:45.691107 11331 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:45.659557 14809 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:45.667900 14815 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:45.686703 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.710032 11330 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
W20260709 12:13:45.687201 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48267.9us
I20260709 12:13:45.694969 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.711593 11223 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:45.695194 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.713990 11322 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:45.648871 14861 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.698164 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 525) took 1458 ms. Trace:
I20260709 12:13:45.701705 14809 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.704205 14539 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:45.715363 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:45.715890 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:45.716837 11174 rpcz_store.cc:276] 0709 12:13:44.239876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.250983 (+ 11107us) service_pool.cc:224] Handling call
0709 12:13:44.251296 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.446343 (+195047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.697878 (+1251535us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b54001f40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:45.698086 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64545152}
I20260709 12:13:45.717438 11248 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:45.718430 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:45.729476 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.730858 11248 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:45.731801 11329 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:45.731927 11327 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:45.732381 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1765) took 2352 ms. Trace:
I20260709 12:13:45.732885 11248 rpcz_store.cc:276] 0709 12:13:43.379490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.729980 (+2350490us) service_pool.cc:224] Handling call
0709 12:13:45.730678 (+   698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.732108 (+  1430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.732323 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:45.613584 11930 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:45.725039 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:45.666162 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.692606 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:45.730865 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:45.731099 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:45.738657 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.688875 14598 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:13:45.748709 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82620.3us
I20260709 12:13:45.751269 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:45.752719 11322 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:45.753027 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:45.754575 11929 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.756847 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:45.728229 14891 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:45.730822 14809 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:45.730273 14820 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:45.730283 14539 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:45.728034 14815 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:45.736202 14450 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9.
I20260709 12:13:45.754174 14408 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:45.749642 14896 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:45.705456 14742 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.754698 14720 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:13:45.777848 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 139078us
I20260709 12:13:45.771090 14891 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:45.770092 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 77548.1us
I20260709 12:13:45.750058 14818 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:45.747180 14766 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:45.751953 14747 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:13:45.794597 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69684.8us
I20260709 12:13:45.790441 14720 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:45.770408 14950 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:13:45.778378 14598 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:45.788302 14896 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:45.809278 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51487.4us
I20260709 12:13:45.818195 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:45.792495 14774 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:13:45.805564 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:45.777509 11929 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:45.778496 14729 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:13:45.778669 14949 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:13:45.784308 14951 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:45.804687 14891 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.802666 14824 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:45.804055 14818 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:45.762758 14875 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.821230 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:45.753863 14829 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:45.788682 14815 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:45.807789 14958 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:45.816780 14720 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:45.821897 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 360676us
I20260709 12:13:45.799700 11329 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:45.858456 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1761) took 2480 ms. Trace:
I20260709 12:13:45.892051 11255 rpcz_store.cc:276] 0709 12:13:43.378264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.622609 (+2244345us) service_pool.cc:224] Handling call
0709 12:13:45.623038 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:45.632466 (+  9428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:45.632498 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:45.642346 (+  9848us) log.cc:844] Serialized 26 byte log entry
0709 12:13:45.782862 (+140516us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:45.782900 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.783086 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:45.858232 (+ 75146us) raft_consensus.cc:1685] finished
0709 12:13:45.858253 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.858392 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10566,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3312,"thread_start_us":113611,"threads_started":1,"wal-append.queue_time_us":167917}
I20260709 12:13:45.894217 11255 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:45.895133 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1769) took 2285 ms. Trace:
I20260709 12:13:45.895718 11255 rpcz_store.cc:276] 0709 12:13:43.609778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.893559 (+2283781us) service_pool.cc:224] Handling call
0709 12:13:45.893976 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.894746 (+   770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.894988 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:45.897099 11255 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
W20260709 12:13:45.867329 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53374us
I20260709 12:13:45.898897 11901 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:45.879122 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78228us
I20260709 12:13:45.904016 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:45.905505 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:45.897565 11255 pending_rounds.cc:85] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 4
I20260709 12:13:45.907369 11255 pending_rounds.cc:107] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:13:45.907823 11255 raft_consensus.cc:2889] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:45.908819 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1770) took 2294 ms. Trace:
I20260709 12:13:45.909310 11255 rpcz_store.cc:276] 0709 12:13:43.614601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.896451 (+2281850us) service_pool.cc:224] Handling call
0709 12:13:45.896755 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.908476 (+ 11721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.908748 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:45.973780 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:45.975273 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 83604.4us
I20260709 12:13:45.978348 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:45.843187 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71536.6us
W20260709 12:13:45.870610 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 99290.9us
I20260709 12:13:45.990213 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:45.994863 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 173693us
I20260709 12:13:46.000306 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:46.001157 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:46.001657 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 196123us
I20260709 12:13:46.001668 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:46.002453 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:46.015347 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:46.016271 11930 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.999058 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:45.852866 11110 debug-util.cc:398] Leaking SignalData structure 0x7b080003bd40 after lost signal to thread 10014
I20260709 12:13:45.845180 14818 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:45.819231 14896 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:45.873879 14598 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:45.852694 14829 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.852428 14445 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:45.873663 14841 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:45.834412 14954 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.849156 14958 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.834779 14742 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.973289 14747 raft_consensus.cc:2764] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:13:45.849323 14824 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:45.856102 14815 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.874284 14861 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:45.973289 14949 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.858084 14960 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:46.042824 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:45.993508 14969 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:46.062983 14969 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.063454 14969 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:45.856057 14796 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.067865 14796 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.867017 14833 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.070580 14833 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:45.854481 14961 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.192s
I20260709 12:13:46.027513 14729 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:46.052695 11930 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:46.080279 14969 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.038635 14959 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:46.038040 14841 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:46.040716 14958 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:46.092278 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 89896.7us
I20260709 12:13:46.054586 14742 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:46.061233 14445 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:46.046401 14968 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:46.082780 14950 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:46.108487 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 209652us
W20260709 12:13:46.109352 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 205453us
I20260709 12:13:46.092458 14965 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:46.093747 14841 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.092245 14815 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:46.110183 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:46.090386 14967 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:46.091822 14729 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:46.045719 14954 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.115464 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:46.105365 14959 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.056443 14970 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:13:46.081933 14949 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.122165 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:46.120296 14967 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.115388 14815 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:46.106492 14973 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:13:46.145323 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:46.210798 11249 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:46.121078 14861 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:46.174073 14967 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:46.174144 14766 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:46.207487 14408 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:46.149262 14970 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:13:46.113983 14875 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.120167 14968 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:46.207962 14976 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:13:46.213546 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 91439.3us
I20260709 12:13:46.218575 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:46.222057 11110 debug-util.cc:398] Leaking SignalData structure 0x7b080008bcc0 after lost signal to thread 11113
I20260709 12:13:46.222860 11256 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:13:46.224822 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1558) took 3376 ms. Trace:
I20260709 12:13:46.233590 11323 rpcz_store.cc:276] 0709 12:13:42.848115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.609715 (+1761600us) service_pool.cc:224] Handling call
0709 12:13:44.610055 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.450258 (+840203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.224491 (+774233us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b540042b8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:46.224754 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790840064}
I20260709 12:13:46.111135 14950 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:46.222355 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 179644us
I20260709 12:13:46.214867 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:46.238276 11249 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:46.233455 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 452) took 2091 ms. Trace:
I20260709 12:13:46.146255 14979 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.239012 11256 rpcz_store.cc:276] 0709 12:13:44.141594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.222290 (+2080696us) service_pool.cc:224] Handling call
0709 12:13:46.222642 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.233177 (+ 10535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.233399 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:46.139773 14980 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:46.240607 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 456) took 2096 ms. Trace:
I20260709 12:13:46.242954 11249 rpcz_store.cc:276] 0709 12:13:44.143729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.237831 (+2094102us) service_pool.cc:224] Handling call
0709 12:13:46.238112 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.238647 (+   535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.240550 (+  1903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:46.209708 14815 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:46.110375 14966 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
W20260709 12:13:46.220052 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 218512us
I20260709 12:13:46.148079 14949 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:46.255479 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.256031 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:46.256300 11250 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:13:46.257015 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 454) took 2114 ms. Trace:
I20260709 12:13:46.155388 14954 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:46.251319 11249 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:13:46.136895 14824 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.143050 14964 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:13:46.241568 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:46.232721 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 126424us
I20260709 12:13:46.258285 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:46.257328 11250 rpcz_store.cc:276] 0709 12:13:44.142258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.232683 (+2090425us) service_pool.cc:224] Handling call
0709 12:13:46.256079 (+ 23396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.256766 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.256962 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:46.251938 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 259461us
I20260709 12:13:46.232445 14976 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:46.232399 14861 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:46.265579 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1778) took 2080 ms. Trace:
I20260709 12:13:46.137748 14959 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.267076 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:46.147652 14977 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 12:13:46.232338 14766 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.275688 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:46.275909 11249 rpcz_store.cc:276] 0709 12:13:44.185341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.250402 (+2065061us) service_pool.cc:224] Handling call
0709 12:13:46.250943 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.262851 (+ 11908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.265513 (+  2662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:46.234741 14408 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.277841 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:46.263387 14875 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:46.265008 14980 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:46.245095 14842 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:13:46.268455 14954 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:46.282522 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:46.295046 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:46.304363 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:46.300763 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:46.242250 14729 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.274140 14976 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.307523 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:46.306504 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:46.262642 14984 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:13:46.280730 14986 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:46.311635 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:46.292514 14971 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:46.296854 14875 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.313047 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:46.314386 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:46.314855 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:46.260108 14979 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.310238 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:46.300246 14954 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:46.318337 11175 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:46.318133 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:46.300089 14445 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:46.335418 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:46.340729 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:46.342933 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:46.343549 11934 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:13:46.349237 14961 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:46.349941 14961 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.350397 14961 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:46.315469 14986 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:46.357137 11110 debug-util.cc:398] Leaking SignalData structure 0x7b0800313f20 after lost signal to thread 11206
I20260709 12:13:46.332320 14965 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:46.397681 14996 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:46.315579 14983 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:46.356719 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:46.398536 14983 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:46.399200 14983 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.332440 14875 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.401024 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1775) took 2741 ms. Trace:
I20260709 12:13:46.315212 14742 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:46.310722 14985 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:46.316390 14971 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:46.298231 14987 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 12:13:46.397818 11110 debug-util.cc:398] Leaking SignalData structure 0x7b0800353c80 after lost signal to thread 11281
I20260709 12:13:46.401923 14973 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:46.401469 11255 rpcz_store.cc:276] 0709 12:13:43.659574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.910214 (+2250640us) service_pool.cc:224] Handling call
0709 12:13:46.000253 (+ 90039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.000469 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:46.232009 (+231540us) log.cc:844] Serialized 14 byte log entry
0709 12:13:46.400676 (+168667us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:46.400729 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:46.400776 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.400842 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.400965 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228749,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"thread_start_us":367312,"threads_started":2,"wal-append.queue_time_us":168653}
W20260709 12:13:46.404920 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 125445us
I20260709 12:13:46.318189 14774 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:46.397593 14965 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.409405 11934 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:46.409622 11110 debug-util.cc:398] Leaking SignalData structure 0x7b0800360000 after lost signal to thread 11355
I20260709 12:13:46.413815 11175 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:46.344591 14970 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:46.414853 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:46.420112 11110 debug-util.cc:398] Leaking SignalData structure 0x7b08003fc4e0 after lost signal to thread 11356
I20260709 12:13:46.419953 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:46.425673 11110 debug-util.cc:398] Leaking SignalData structure 0x7b08003f2240 after lost signal to thread 11959
W20260709 12:13:46.427115 11110 debug-util.cc:398] Leaking SignalData structure 0x7b08003cab20 after lost signal to thread 14901
W20260709 12:13:46.427572 11110 debug-util.cc:398] Leaking SignalData structure 0x7b08002f9e40 after lost signal to thread 14916
I20260709 12:13:46.430482 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:46.403321 14986 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:46.408167 14971 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:46.394976 14995 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:46.392752 14896 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:46.418614 14981 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.259s	user 0.001s	sys 0.001s
I20260709 12:13:46.443337 14445 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:46.421994 14840 raft_consensus.cc:2764] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:13:46.454722 11255 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:46.304291 14968 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.414791 14742 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:46.443832 14969 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.442962 14796 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:46.454128 14774 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:46.462457 14995 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:46.345468 14861 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.486032 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:46.486085 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 71755.9us
I20260709 12:13:46.487802 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:46.489727 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:46.490037 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:46.490820 11934 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.477849 14445 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:46.492563 14445 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.493330 14408 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.501691 11934 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:46.508909 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:46.510006 11934 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:46.514221 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:46.524634 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 208451us
W20260709 12:13:46.525844 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 244092us
W20260709 12:13:46.526407 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 234862us
I20260709 12:13:46.529165 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:46.526865 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 290088us
I20260709 12:13:46.534101 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:46.537181 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.550144 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:46.552361 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:46.553563 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:46.538692 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:46.482658 14833 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.564388 14833 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.567166 14833 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:46.568665 14896 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:46.560968 14977 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:46.485651 14950 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.522586 14982 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:46.461668 14954 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.561863 14408 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:46.418411 14980 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.562742 14970 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:13:46.602697 14977 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:46.606626 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:46.613598 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2207) took 1165 ms. Trace:
I20260709 12:13:46.483987 14796 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.609830 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:46.615181 11934 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:46.611634 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 83799.4us
I20260709 12:13:46.607247 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:46.594079 15004 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:46.578527 14959 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:46.614028 11180 rpcz_store.cc:276] 0709 12:13:45.447853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.462549 (+ 14696us) service_pool.cc:224] Handling call
0709 12:13:46.602904 (+1140355us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7e9c 00007f3ec35c7a5b 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:13:46.613525 (+ 10621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2725760}
I20260709 12:13:46.575222 14766 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.616056 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:46.618111 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:46.625238 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:46.613782 14865 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.804s	user 0.022s	sys 0.000s
I20260709 12:13:46.606431 14408 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:46.607111 14992 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:46.632750 11252 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:46.616928 14950 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.609436 15009 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:46.616956 15004 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.620134 14982 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:46.651122 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 457) took 2056 ms. Trace:
I20260709 12:13:46.627596 14954 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:46.631108 14992 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:46.671110 14992 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.624229 14774 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.658666 15008 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:46.676724 15008 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:46.677145 15008 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:46.678131 15008 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.679476 15008 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:46.613018 14742 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.685107 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:46.632397 14408 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.630234 14979 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.629211 14896 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:13:46.667991 14841 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
W20260709 12:13:46.668849 14722 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.855s	user 0.008s	sys 0.006s
I20260709 12:13:46.630344 14959 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:46.711293 14959 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.711838 14959 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:46.711517 14722 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.691905 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:46.713899 11325 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:46.714892 14722 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.715217 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:46.716043 11325 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:46.717080 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:46.661959 11250 rpcz_store.cc:276] 0709 12:13:44.594960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.279363 (+1684403us) service_pool.cc:224] Handling call
0709 12:13:46.279696 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.282007 (+  2311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.282037 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:46.292840 (+ 10803us) log.cc:844] Serialized 26 byte log entry
0709 12:13:46.458746 (+165906us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:46.462716 (+  3970us) log.cc:844] Serialized 14 byte log entry
0709 12:13:46.462927 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.463004 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:46.650891 (+187887us) raft_consensus.cc:1685] finished
0709 12:13:46.650913 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.651059 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1912,"prepare.queue_time_us":2758,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":25946,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":13790,"thread_start_us":165572,"threads_started":1,"wal-append.queue_time_us":265309}
I20260709 12:13:46.718436 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:46.719340 11250 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:46.720296 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:46.634016 14676 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:46.720889 11250 raft_consensus.cc:2376] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10. Re-sending same reply.
I20260709 12:13:46.724301 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:46.725068 11250 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:46.726112 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:46.727623 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:46.729102 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:46.640838 15009 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:46.659962 14720 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.804s	user 0.010s	sys 0.001s
I20260709 12:13:46.638228 14766 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:46.734100 11325 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:46.646754 15004 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.736724 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:46.739172 11324 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.660363 14865 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:13:46.707852 14957 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.889s	user 0.002s	sys 0.009s
I20260709 12:13:46.734972 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:46.735921 14994 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:46.744341 15002 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:13:46.733224 14977 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.763538 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:46.710470 14896 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:46.619870 14995 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.662722 14950 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:46.822031 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49151.4us
I20260709 12:13:46.737406 15009 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:46.768754 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 215234us
I20260709 12:13:46.775713 11324 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:46.778754 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 249738us
I20260709 12:13:46.731786 14841 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:46.790812 14815 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.006s
I20260709 12:13:46.734524 14766 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:46.826737 14729 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:46.829272 14959 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.830631 14995 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:46.833976 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:46.797765 14896 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:46.782974 14891 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.938s	user 0.012s	sys 0.000s
I20260709 12:13:46.837859 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:46.857095 11325 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:46.770874 14978 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:46.820767 14955 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:46.764237 14445 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.737197 14720 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.866487 11326 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:46.755256 14983 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.835178 14982 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.770627 14994 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.866333 14985 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:46.848865 14766 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:46.866273 14955 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:46.857301 14815 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.869395 15019 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:46.894838 11934 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:46.916909 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:46.886646 14818 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.848s	user 0.009s	sys 0.004s
I20260709 12:13:46.862720 14978 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.886145 14841 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:46.819921 14986 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:46.889845 14958 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.000s
W20260709 12:13:46.884301 14809 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.095s	user 0.011s	sys 0.002s
W20260709 12:13:46.894549 14991 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.489s	user 0.001s	sys 0.000s
I20260709 12:13:46.891541 14955 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:46.937851 14958 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.958534 14809 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.890328 14985 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.867272 14865 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:13:46.888903 14875 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.929840 14978 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:46.922467 14818 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.135097 11256 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:47.136969 14818 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.129078 14958 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:47.140404 14971 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.667s	user 0.013s	sys 0.006s
W20260709 12:13:47.048921 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 223098us
I20260709 12:13:47.050549 11251 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:13:47.052431 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 204165us
I20260709 12:13:47.148099 15004 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.057801 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:47.150871 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:47.152254 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95114.9us
I20260709 12:13:47.152858 15004 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:46.924667 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.998984 15023 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:47.172883 15023 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.176568 15023 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.181571 15008 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:47.089725 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.090116 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:47.092690 11329 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:47.095799 11331 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:13:47.100407 15033 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:47.104053 11933 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:47.114703 14729 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:13:47.117744 11322 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:13:46.869891 14445 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:46.934564 14720 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:46.881784 14842 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.881234 14861 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.119728 14875 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:47.193984 14842 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.194245 14861 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:46.965323 14815 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.127068 14950 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.128360 11255 raft_consensus.cc:2368] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9.
I20260709 12:13:47.129047 14809 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.132926 14676 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.836082 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:47.215734 14969 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:47.216472 14969 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:13:47.216696 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 380568us
I20260709 12:13:47.216948 14969 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:47.218710 15033 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.013016 14983 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.222218 14809 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:47.222394 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:47.222889 14809 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:47.223084 11326 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:13:47.007344 14985 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.010443 14986 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:46.979717 14994 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.088281 11328 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:47.227563 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.228372 11328 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:47.153551 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:47.229974 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:47.230134 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 76901.8us
W20260709 12:13:47.170135 14967 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.937s	user 0.018s	sys 0.003s
I20260709 12:13:47.233745 14967 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.236564 14967 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.150887 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:46.867452 14891 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:47.238377 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 87769.3us
I20260709 12:13:47.239410 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:47.240301 14891 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.240715 15019 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:47.241221 14891 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:47.205579 15005 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.205739 14989 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:13:47.206087 15010 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.208289 14445 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:47.224206 14809 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.242025 14720 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:47.243520 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:47.246109 11249 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:47.248340 15033 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:47.249087 11325 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:47.215823 14676 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.284051 14676 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.252588 14729 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:47.254356 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.236594 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:47.254796 14861 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.255903 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:47.209471 15009 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.257668 14842 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:47.202893 15032 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.089s	user 0.002s	sys 0.000s
I20260709 12:13:47.148484 14971 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.210378 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.244751 14539 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.279636 11257 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11. Re-sending same reply.
I20260709 12:13:47.279820 14841 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.281350 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:47.290233 14961 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.860s	user 0.014s	sys 0.006s
W20260709 12:13:47.290879 14976 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.956s	user 0.017s	sys 0.018s
I20260709 12:13:47.290984 14949 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:47.298941 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:47.291317 15009 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.292196 11250 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:47.292385 14408 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.006s
W20260709 12:13:47.303622 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71213.7us
I20260709 12:13:47.287279 14720 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:47.314438 11329 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:47.315696 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:47.304883 14976 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.330411 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.330713 11928 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:47.302747 14841 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.295877 14961 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.331687 14976 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:47.327981 11930 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
W20260709 12:13:47.333320 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57707.9us
I20260709 12:13:47.334254 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:47.308300 14831 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.331867 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.341893 11325 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:47.348501 14954 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.010s
I20260709 12:13:47.348829 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1776) took 3164 ms. Trace:
I20260709 12:13:47.349422 11256 rpcz_store.cc:276] 0709 12:13:44.184713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.239815 (+2055102us) service_pool.cc:224] Handling call
0709 12:13:46.240110 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:47.135923 (+895813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.348277 (+212354us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54002a6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:47.348674 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2056700160}
I20260709 12:13:47.351014 14539 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:47.350742 14955 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.351513 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:47.352383 11256 raft_consensus.cc:2376] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12. Re-sending same reply.
I20260709 12:13:47.353458 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:47.309881 14408 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.354270 11256 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:47.355288 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:47.355670 14408 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.356539 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:47.357657 14994 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:47.313198 15010 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.358006 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:47.358889 15033 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:47.359481 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:47.360085 15010 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:47.360414 11256 raft_consensus.cc:2368] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8.
I20260709 12:13:47.360809 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:47.360761 15010 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:47.362457 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:47.363130 11329 raft_consensus.cc:2376] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11. Re-sending same reply.
I20260709 12:13:47.366791 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:47.368017 15006 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:47.368396 11323 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:47.369251 11328 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:47.308923 15005 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.370290 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.369989 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:47.306305 14720 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.378345 14949 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:47.379586 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:47.380837 14950 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:47.381470 14950 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:47.335326 14967 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.363986 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.371922 15005 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:47.375613 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45374.6us
I20260709 12:13:47.376801 14971 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:47.380019 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 84746.4us
I20260709 12:13:47.336675 11324 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:47.368479 15006 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:47.371596 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.378387 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:47.358131 14994 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:47.359602 15033 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:47.358706 14976 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:47.367501 14961 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:47.382052 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47981.7us
I20260709 12:13:47.373064 11323 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:47.397405 11326 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:47.398428 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:47.409076 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:47.388098 14955 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:47.423962 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:47.436178 15033 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.442173 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:47.443625 11934 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 6: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:47.444597 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57708.6us
W20260709 12:13:47.445936 14766 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.517s	user 0.016s	sys 0.008s
I20260709 12:13:47.446448 14766 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.446795 14966 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:47.447433 14966 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:47.449105 14766 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:47.451571 14896 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.011s
I20260709 12:13:47.462352 14896 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.465093 14896 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.455788 14969 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.477767 14994 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.479768 14994 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:47.482442 14950 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:47.491992 14978 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.513s	user 0.018s	sys 0.001s
I20260709 12:13:47.492688 14978 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.494349 15006 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.496109 15006 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:47.498358 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 91208.9us
I20260709 12:13:47.500595 14954 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:47.501112 14954 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:47.501508 14954 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:47.762364 15008 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.003s
I20260709 12:13:47.422557 14967 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:47.794728 14842 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.795434 14842 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.796461 15010 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.801627 14842 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:47.804026 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.804157 14978 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.816877 14967 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.788808 15008 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:47.904191 15010 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:47.961902 15010 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.967315 15036 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:47.971016 14969 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:47.981401 14968 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.023s
W20260709 12:13:47.988698 14968 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.004s	sys 0.023s
W20260709 12:13:47.990376 15033 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.052s	sys 0.051s
W20260709 12:13:47.994988 15033 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.053s	sys 0.051s
I20260709 12:13:47.993151 15036 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:48.016394 15043 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:13:48.019863 15036 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:47.999943 14954 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.011368 14841 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:48.021433 15006 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.525s	user 0.014s	sys 0.000s
I20260709 12:13:48.038435 14982 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:48.039073 14982 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.040514 14982 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:48.043637 14539 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.692s	user 0.013s	sys 0.001s
W20260709 12:13:48.047610 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78348.2us
I20260709 12:13:48.045535 15036 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.048957 14954 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:48.053812 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.054301 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.054674 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.054915 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.056293 14539 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:48.055971 14982 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.053828 15036 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.074572 11329 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:48.075912 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.076423 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.077574 14841 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:13:48.079322 14994 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.599s	user 0.007s	sys 0.007s
I20260709 12:13:48.080062 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:48.081302 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:48.058678 14539 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.088564 14539 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.091598 14969 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:48.092505 14969 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.094122 14969 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.062112 15006 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:48.098667 15006 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.114131 11296 leader_election.cc:400] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8.
I20260709 12:13:48.114639 11296 leader_election.cc:403] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:47.408443 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1898) took 1089 ms. Trace:
I20260709 12:13:48.144410 11257 rpcz_store.cc:276] 0709 12:13:46.313309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.378075 (+1064766us) service_pool.cc:224] Handling call
0709 12:13:47.402432 (+ 24357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.146021 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.089442 14539 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:48.148684 14539 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.148782 11257 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:48.149160 14539 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.149942 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.150789 11257 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:48.151665 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.152303 11257 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:48.152670 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1910) took 1031 ms. Trace:
I20260709 12:13:48.152951 11257 rpcz_store.cc:276] 0709 12:13:47.121595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.151359 (+1029764us) service_pool.cc:224] Handling call
0709 12:13:48.152624 (+  1265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.116196 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:48.116816 15006 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.143723 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1591) took 1453 ms. Trace:
I20260709 12:13:48.146950 14796 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.162843 15033 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:48.163512 15033 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:48.164077 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.165030 11257 raft_consensus.cc:2410] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:13:48.164613 15033 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.165503 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1895) took 1855 ms. Trace:
I20260709 12:13:48.165872 11257 rpcz_store.cc:276] 0709 12:13:46.309921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.153589 (+1843668us) service_pool.cc:224] Handling call
0709 12:13:48.165438 (+ 11849us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.167013 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.167833 11930 raft_consensus.cc:2368] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10.
I20260709 12:13:48.168941 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.116760 14841 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8.
I20260709 12:13:48.169533 11934 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.162611 11322 rpcz_store.cc:276] 0709 12:13:46.690686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.691686 (+  1000us) service_pool.cc:224] Handling call
0709 12:13:46.735781 (+ 44095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:47.210126 (+474345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.143316 (+933190us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54000c6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:48.143606 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876364032}
I20260709 12:13:48.174809 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.162870 14965 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.176146 14965 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.179255 14965 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.179798 14796 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:48.181036 14796 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.183326 14796 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.185220 14796 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.185876 14796 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.188442 14796 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.191984 11934 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:48.202276 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.228596 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1912) took 1019 ms. Trace:
I20260709 12:13:48.229152 11257 rpcz_store.cc:276] 0709 12:13:47.209646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.172698 (+963052us) service_pool.cc:224] Handling call
0709 12:13:48.228512 (+ 55814us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.233376 15051 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.234457 15051 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.233328 14965 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:48.238006 14965 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.239310 15051 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.236325 11257 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:13:48.241590 14965 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.231875 14539 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.242340 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1782) took 3531 ms. Trace:
I20260709 12:13:48.244107 14539 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.243319 11257 rpcz_store.cc:276] 0709 12:13:44.709966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.235396 (+3525430us) service_pool.cc:224] Handling call
0709 12:13:48.235995 (+   599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.241125 (+  5130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.241415 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:48.250257 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1874) took 3512 ms (client timeout 1844 ms). Trace:
W20260709 12:13:48.250761 11252 rpcz_store.cc:269] 0709 12:13:44.737510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.807570 (+ 70060us) service_pool.cc:224] Handling call
0709 12:13:48.246960 (+3439390us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b540010a754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:48.250037 (+  3077us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4195535104}
I20260709 12:13:48.248337 15052 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.255678 15052 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.259302 15052 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:48.262326 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 865879us
I20260709 12:13:48.263451 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.267768 15006 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:48.268711 15006 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:48.269555 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:48.269786 14998 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:48.270354 14998 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.270655 14539 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:48.272220 14539 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.274400 14539 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.279093 14966 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.282778 14998 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.283398 14966 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:48.284438 14998 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.286573 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:48.290704 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63965.2us
I20260709 12:13:48.294994 11930 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:48.300635 14829 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.300765 11257 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:48.301368 14829 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.301753 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.302201 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1872) took 3565 ms. Trace:
I20260709 12:13:48.302630 11257 rpcz_store.cc:276] 0709 12:13:44.736881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.286825 (+3549944us) service_pool.cc:224] Handling call
0709 12:13:48.299472 (+ 12647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.301849 (+  2377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.302112 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.303884 14829 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.303985 11257 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:48.297755 15052 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.308372 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:48.308581 15052 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.309171 11934 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:48.310369 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:48.311009 15052 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.313073 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.315034 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.299032 14796 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:48.321573 14796 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.324247 14796 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.326215 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:48.317791 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.320207 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:48.326828 14796 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:48.326893 15051 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:48.327812 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.331264 11930 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.327752 15052 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:48.331465 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1786) took 2826 ms. Trace:
I20260709 12:13:48.332082 11257 rpcz_store.cc:276] 0709 12:13:45.481074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.303427 (+2822353us) service_pool.cc:224] Handling call
0709 12:13:48.303796 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.307134 (+  3338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.307370 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.327764 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 458) took 3690 ms. Trace:
I20260709 12:13:48.319561 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.319585 11933 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:13:48.332993 11249 rpcz_store.cc:276] 0709 12:13:44.636893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.286304 (+1649411us) service_pool.cc:224] Handling call
0709 12:13:46.286608 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:47.280177 (+993569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.327473 (+1047296us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b540033ccd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:48.327707 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205561088}
I20260709 12:13:48.346817 14998 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.347599 14998 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.350019 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.350828 11176 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:48.351416 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.352124 11179 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:48.330885 15051 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.353502 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.357503 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:48.356882 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.359364 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:48.359993 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:48.364614 11330 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:48.364750 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.366291 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.369705 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.372118 15051 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.373157 11249 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:48.373479 11173 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:48.356999 14796 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.374315 14998 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.379060 14966 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.379626 14966 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.381876 14966 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.383247 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.383960 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.384298 11323 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:48.384714 11329 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.395277 11329 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
W20260709 12:13:48.412273 11222 connection.cc:256] RPC call timeout handler was delayed by -1.0564043739998397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260709 12:13:48.414665 11933 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:13:48.415654 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.415675 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.416551 11935 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:48.417829 14951 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.417939 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.418481 14951 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.422101 11331 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:48.422567 11257 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.424070 14951 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.419436 14877 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.425014 14877 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.428000 14877 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.437861 14877 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:48.411755 14796 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.447346 14877 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:48.449705 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.441216 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 684) took 1952 ms. Trace:
I20260709 12:13:48.450248 11928 rpcz_store.cc:276] 0709 12:13:46.488939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.489417 (+   478us) service_pool.cc:224] Handling call
0709 12:13:46.489737 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:47.331188 (+841451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.440765 (+1109577us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b540034f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:48.441098 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933085056}
I20260709 12:13:48.450980 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:48.451885 11935 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:48.453239 11930 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:48.466519 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.467629 11175 raft_consensus.cc:2410] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:13:48.469689 11257 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:48.471065 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.471944 11257 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:48.472775 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.473316 11257 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:48.385492 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.458640 14816 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.459195 14998 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:13:48.459574 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.596s (SENT) [suppressed 45 similar messages]
I20260709 12:13:48.459842 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.483506 11324 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:13:48.460975 14965 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.461277 14877 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.475024 11329 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:13:48.477376 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.484999 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:48.485486 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.455953 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1867) took 6226 ms (client timeout 1901 ms). Trace:
I20260709 12:13:48.485168 14816 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:48.486125 11253 rpcz_store.cc:269] 0709 12:13:42.229491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.314881 (+ 85390us) service_pool.cc:224] Handling call
0709 12:13:48.455574 (+6140693us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b540010ac54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:48.455753 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2131906944}
I20260709 12:13:48.465314 14959 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.473451 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.490110 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 651) took 1102 ms. Trace:
I20260709 12:13:48.490551 11324 rpcz_store.cc:276] 0709 12:13:47.386310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.408850 (+ 22540us) service_pool.cc:224] Handling call
0709 12:13:48.488617 (+1079767us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.490974 11326 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 11 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:48.491550 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 649) took 1117 ms. Trace:
I20260709 12:13:48.491916 11326 rpcz_store.cc:276] 0709 12:13:47.373743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.473174 (+1099431us) service_pool.cc:224] Handling call
0709 12:13:48.491498 (+ 18324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.485314 14998 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:48.493716 14998 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:48.482288 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 155868us
I20260709 12:13:48.488009 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.494876 14998 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.495985 11253 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:48.494968 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1901) took 1900 ms (client timeout 1714 ms). Trace:
I20260709 12:13:48.496522 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.496649 14965 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.497263 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.498425 11253 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:48.499285 15057 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.499832 15057 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.501055 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1787) took 2486 ms. Trace:
I20260709 12:13:48.501844 14796 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:48.496587 11251 rpcz_store.cc:269] 0709 12:13:46.595010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.684854 (+ 89844us) service_pool.cc:224] Handling call
0709 12:13:48.494637 (+1809783us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54002a58d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:48.494832 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837733760}
I20260709 12:13:48.503024 15057 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.503582 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1602) took 1372 ms. Trace:
I20260709 12:13:48.503736 11251 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:13:48.504068 11251 pending_rounds.cc:85] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 7
I20260709 12:13:48.504292 11251 pending_rounds.cc:107] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:13:48.504580 11251 raft_consensus.cc:2889] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:48.504316 14966 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:48.505378 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1790) took 2464 ms. Trace:
I20260709 12:13:48.505775 11251 rpcz_store.cc:276] 0709 12:13:46.040761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.502952 (+2462191us) service_pool.cc:224] Handling call
0709 12:13:48.503366 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.505165 (+  1799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.505325 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.506803 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.507491 11251 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.508282 11323 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:48.509187 14816 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.510324 11256 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:48.510584 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.503950 11329 rpcz_store.cc:276] 0709 12:13:47.112739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.474410 (+1361671us) service_pool.cc:224] Handling call
0709 12:13:48.474780 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.485070 (+ 10290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.485314 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.513487 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.513762 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.514292 11323 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.513895 14796 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.514971 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.516322 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.516848 11329 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:48.517084 14796 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.501417 11253 rpcz_store.cc:276] 0709 12:13:46.014196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.499075 (+2484879us) service_pool.cc:224] Handling call
0709 12:13:48.499440 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.500678 (+  1238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.500706 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:48.500767 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:48.500798 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.500856 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.500974 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87}
I20260709 12:13:48.501720 14965 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:48.531013 14796 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:48.496572 14998 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.535069 11329 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:13:48.536027 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 52085.8us
I20260709 12:13:48.533407 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:48.532368 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.545905 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.547153 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.547711 11256 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:48.548506 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.548889 11256 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.555224 14984 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:48.555879 14984 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:48.537817 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.559839 11251 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:48.528123 14968 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:48.563381 14840 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.531740 14816 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:13:48.564404 14824 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:48.564594 14816 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:48.564690 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.565018 14816 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.564970 14824 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.566232 14816 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.565615 11251 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:48.567407 14824 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:13:48.567528 14816 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.560871 11253 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:48.568342 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:48.570796 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 60311.8us
I20260709 12:13:48.571532 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:48.573798 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 114250us
I20260709 12:13:48.575172 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1886) took 3476 ms. Trace:
I20260709 12:13:48.577836 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:48.577978 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:48.576620 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.580571 11257 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:48.580942 11296 leader_election.cc:400] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10.
I20260709 12:13:48.581300 11296 leader_election.cc:403] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:13:48.590287 14959 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.591486 14796 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:48.591257 14959 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.592044 14796 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.593675 14959 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.596709 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1603) took 1483 ms. Trace:
I20260709 12:13:48.596428 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.607182 11324 rpcz_store.cc:276] 0709 12:13:47.113191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.491367 (+1378176us) service_pool.cc:224] Handling call
0709 12:13:48.491743 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.492097 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.492127 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:48.492765 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.574434 (+ 81669us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:48.574690 (+   256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.574756 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:48.596481 (+ 21725us) raft_consensus.cc:1685] finished
0709 12:13:48.596503 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.596653 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":81270,"threads_started":1,"wal-append.queue_time_us":92309}
I20260709 12:13:48.611030 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:48.610690 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.614125 11253 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:48.612012 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.615609 11324 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:48.617847 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:48.622778 15034 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.273s	user 0.011s	sys 0.008s
I20260709 12:13:48.616360 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.625566 11253 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:48.627862 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.631268 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.633428 11253 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.636206 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:48.625900 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47978.6us
W20260709 12:13:48.637317 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.637411 11147 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.706s (SENT)
I20260709 12:13:48.637872 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.638962 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.626199 11324 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
W20260709 12:13:48.641780 11325 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.299s	user 0.021s	sys 0.004s
I20260709 12:13:48.642385 11325 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:48.642948 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 647) took 1365 ms. Trace:
I20260709 12:13:48.636111 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:48.646445 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2215) took 1421 ms. Trace:
I20260709 12:13:48.643287 11325 rpcz_store.cc:276] 0709 12:13:47.277272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.280974 (+  3702us) service_pool.cc:224] Handling call
0709 12:13:48.642895 (+1361921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.647130 11323 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:48.647576 11256 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:48.646940 11326 rpcz_store.cc:276] 0709 12:13:47.224580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.492667 (+1268087us) service_pool.cc:224] Handling call
0709 12:13:48.492988 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.493256 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:48.493931 (+   675us) log.cc:844] Serialized 14 byte log entry
0709 12:13:48.637315 (+143384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:48.637363 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:48.646189 (+  8826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.646262 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.646385 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":198,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":78,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":3047,"thread_start_us":142800,"threads_started":1}
I20260709 12:13:48.648924 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.649699 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.649967 11328 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:48.650421 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69622.8us
I20260709 12:13:48.650529 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1606) took 1506 ms. Trace:
I20260709 12:13:48.575565 11252 rpcz_store.cc:276] 0709 12:13:45.094395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.251706 (+3157311us) service_pool.cc:224] Handling call
0709 12:13:48.252065 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.269898 (+ 17833us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:48.342406 (+ 72508us) log.cc:844] Serialized 14 byte log entry
0709 12:13:48.570348 (+227942us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:48.570401 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:48.570445 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.570507 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.570624 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88608,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":61,"raft.queue_time_us":1603,"thread_start_us":273329,"threads_started":2}
I20260709 12:13:48.650930 11328 rpcz_store.cc:276] 0709 12:13:47.144158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.371290 (+227132us) service_pool.cc:224] Handling call
0709 12:13:48.650472 (+1279182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.654820 11256 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:48.654763 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:48.654744 15057 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.658145 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.656255 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.661880 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.667430 11257 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:48.662245 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.663772 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.669694 11178 raft_consensus.cc:2410] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:13:48.667011 11328 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:48.671048 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.671640 11328 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:48.665562 11248 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:13:48.672586 11248 pending_rounds.cc:85] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 2
I20260709 12:13:48.672523 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.657747 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.673857 11325 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:48.673971 11175 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:48.674721 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.661026 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.660174 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.675992 11322 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:48.661378 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.676987 11173 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:48.677323 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.676121 11179 raft_consensus.cc:2410] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:13:48.678503 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.675432 11325 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.679708 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.680456 11322 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:48.681164 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:48.673347 11328 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.658399 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.673015 11248 pending_rounds.cc:107] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:13:48.688539 11248 raft_consensus.cc:2889] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:48.665818 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:48.692373 11253 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:48.692462 11322 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:48.675938 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:48.695572 11252 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:48.663627 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1881) took 3570 ms (client timeout 3265 ms). Trace:
W20260709 12:13:48.696476 11254 rpcz_store.cc:269] 0709 12:13:45.093033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.424119 (+331086us) service_pool.cc:224] Handling call
0709 12:13:48.663241 (+3239122us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54001cf7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:48.663460 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300283136}
I20260709 12:13:48.697614 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.698261 11254 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:48.698300 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.699530 11254 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:13:48.699738 11323 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:13:48.700125 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1899) took 2384 ms. Trace:
I20260709 12:13:48.700289 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2216) took 1475 ms. Trace:
I20260709 12:13:48.700464 11254 rpcz_store.cc:276] 0709 12:13:46.315895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.698943 (+2383048us) service_pool.cc:224] Handling call
0709 12:13:48.699321 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.699886 (+   565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.700079 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.700656 11323 rpcz_store.cc:276] 0709 12:13:47.224847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.699201 (+1474354us) service_pool.cc:224] Handling call
0709 12:13:48.699489 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.700078 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.700243 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.702332 11180 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:13:48.687779 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:48.703984 11251 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:48.695564 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.705039 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:48.714099 11254 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:13:48.715960 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1908) took 1613 ms. Trace:
I20260709 12:13:48.716691 11322 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:48.716377 11254 rpcz_store.cc:276] 0709 12:13:47.102975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.701356 (+1598381us) service_pool.cc:224] Handling call
0709 12:13:48.702814 (+  1458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.715616 (+ 12802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.715877 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.720429 11325 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:48.728554 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 60456.3us
I20260709 12:13:48.733672 11254 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:48.733706 11322 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:13:48.734820 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2217) took 1437 ms. Trace:
I20260709 12:13:48.735241 11322 rpcz_store.cc:276] 0709 12:13:47.297560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.718896 (+1421336us) service_pool.cc:224] Handling call
0709 12:13:48.719350 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.734585 (+ 15235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.734768 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.737660 11325 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:48.738495 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 648) took 1365 ms. Trace:
I20260709 12:13:48.738672 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 650) took 1362 ms. Trace:
I20260709 12:13:48.738862 11325 rpcz_store.cc:276] 0709 12:13:47.373409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.723398 (+1349989us) service_pool.cc:224] Handling call
0709 12:13:48.734495 (+ 11097us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.738225 (+  3730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.738446 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.739194 11326 rpcz_store.cc:276] 0709 12:13:47.373996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.732136 (+1358140us) service_pool.cc:224] Handling call
0709 12:13:48.732450 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.735843 (+  3393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.735886 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:48.735931 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:48.735965 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.736022 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.736143 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:48.741956 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 128331us
I20260709 12:13:48.732365 15051 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:48.756774 11252 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:48.761128 11252 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:48.763285 11177 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:13:48.763643 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1590) took 2047 ms. Trace:
I20260709 12:13:48.764196 11252 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:13:48.766201 15068 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:48.766728 11252 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:13:48.768293 11252 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:13:48.768044 15068 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:48.758162 15051 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:48.769662 15068 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:48.770990 15068 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:48.771498 15068 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.769544 15051 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.764072 11327 rpcz_store.cc:276] 0709 12:13:46.690376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.690754 (+   378us) service_pool.cc:224] Handling call
0709 12:13:46.691085 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.691418 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.691445 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:48.671074 (+1979629us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b2c00126d18. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:13:48.671623 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.725167 (+ 53544us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:48.725211 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.725288 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:48.728022 (+  2734us) raft_consensus.cc:1685] finished
0709 12:13:48.728042 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.737437 (+  9395us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16092,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5272,"spinlock_wait_cycles":12477696,"thread_start_us":53064,"threads_started":1,"wal-append.queue_time_us":52628}
I20260709 12:13:48.762399 11254 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:48.775233 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1909) took 1669 ms. Trace:
I20260709 12:13:48.745473 15065 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:48.775651 11254 rpcz_store.cc:276] 0709 12:13:47.106396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.717328 (+1610932us) service_pool.cc:224] Handling call
0709 12:13:48.733511 (+ 16183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.774955 (+ 41444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.775166 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.795717 15057 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10.
I20260709 12:13:48.772432 15068 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.800341 11254 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:13:48.809392 11328 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:13:48.810283 11323 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:48.811830 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 52162.9us
I20260709 12:13:48.812742 11254 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:48.821714 11254 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:48.822187 11254 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:48.819985 15069 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.836133 11328 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:13:48.847025 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 211027us
I20260709 12:13:48.849356 11254 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.850750 11254 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:48.826900 15065 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:48.825412 15073 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:48.852497 15073 raft_consensus.cc:2764] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:13:48.827020 15067 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.856504 11328 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:48.857795 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:48.594111 14796 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.860211 11254 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:13:48.861548 11254 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:48.829239 15064 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:48.870805 15064 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.871851 15064 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.879238 11328 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:13:48.884675 15078 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:48.885159 11328 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:13:48.881385 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 149712us
I20260709 12:13:48.888520 11256 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:13:48.889284 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:48.844853 15069 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.905426 11328 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:48.896483 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.909931 14815 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.910728 14815 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:48.864287 15067 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:48.914114 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59871.2us
I20260709 12:13:48.924438 15067 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.925760 14796 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:13:48.925480 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:48.926586 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.918839 11328 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:48.930387 11328 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:13:48.922667 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 150358us
I20260709 12:13:48.939332 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:48.931006 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:48.913605 14815 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:48.942279 14796 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:48.936012 15067 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:48.943516 15067 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:48.944424 15067 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:48.915681 15070 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:48.947672 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:48.948680 15074 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:48.949301 15074 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.949775 15074 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.948348 11328 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:13:48.953434 11328 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:48.921077 15069 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:48.957443 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:48.962939 11223 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:48.947315 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:48.922446 15064 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:13:48.977378 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1608) took 1700 ms. Trace:
I20260709 12:13:48.966331 15078 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:48.954700 15075 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:48.973326 14992 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:48.987432 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:48.988451 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:48.946815 15070 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:48.989614 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:48.977808 11323 rpcz_store.cc:276] 0709 12:13:47.276858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.701508 (+1424650us) service_pool.cc:224] Handling call
0709 12:13:48.810014 (+108506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.810747 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.977193 (+166446us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400459cd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:48.977323 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245090688}
W20260709 12:13:48.984843 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58407.1us
I20260709 12:13:48.999415 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:49.000787 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57154.5us
I20260709 12:13:48.976845 15069 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:13:49.007707 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.003291 14951 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 12:13:49.009775 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62565.9us
I20260709 12:13:49.009711 14951 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:49.010659 15065 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:49.010912 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1593) took 2319 ms. Trace:
I20260709 12:13:48.985706 15075 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.011543 15075 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:49.011318 11330 rpcz_store.cc:276] 0709 12:13:46.691238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.691353 (+   115us) service_pool.cc:224] Handling call
0709 12:13:46.691615 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.365353 (+1673738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:49.010575 (+645222us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54003228d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:49.010827 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1431774464}
I20260709 12:13:49.012619 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:49.013921 14796 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:48.977430 15081 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:49.014564 15082 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:49.012663 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:49.000380 14952 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.020838 11330 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:49.023303 14982 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:49.000005 15074 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.024413 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:49.027853 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:49.028107 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:49.030908 11328 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:49.024305 15071 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:49.032178 15071 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:49.031875 14952 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.032361 15070 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.033172 15071 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.033176 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:49.018360 15081 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.041818 15081 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:49.027935 14982 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.043032 14982 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:49.046429 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.047147 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.048512 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.049023 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.033787 15070 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:49.050422 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.047854 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.051745 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.052189 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.052938 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:49.039482 15069 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:49.049836 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.049786 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.053601 15069 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:49.054113 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.003229 15078 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:13:49.027315 15064 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:49.041249 11329 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:13:49.052982 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.049881 11180 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:49.038986 11328 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:49.054064 11933 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:49.058427 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:49.027257 15065 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:49.026983 15073 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:49.074559 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.077726 11179 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:49.078796 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.058478 15064 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:49.061041 15082 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:49.081416 11180 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:13:49.054693 15069 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.078181 14951 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:13:49.080751 14952 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:49.055498 14992 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:13:49.106385 11178 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:49.095284 15057 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:49.080714 14816 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.002s
I20260709 12:13:49.106693 15067 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:49.114068 15067 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.114209 15069 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.114491 15067 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.119625 14982 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.122038 15081 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.124279 15081 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:49.126976 14976 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:49.127277 11935 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:13:49.128504 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.129539 11179 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:49.126577 15075 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.108958 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:49.077360 15073 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:49.146658 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64027.1us
I20260709 12:13:49.106009 15064 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:49.106372 11324 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:49.168149 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:49.179337 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:49.107575 14951 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:49.138176 14976 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:49.137805 14796 consensus_queue.cc:1243] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:49.154484 14816 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:13:49.147629 14815 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:49.192884 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:49.198132 14976 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.199746 14976 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:49.200254 11257 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:13:49.201459 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:49.130136 15057 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:49.198094 14815 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:49.198629 14796 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.193563 11253 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:49.195212 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:49.209120 11325 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.202618 11249 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:49.200240 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:49.193360 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 135011us
I20260709 12:13:49.148561 14539 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:49.207141 11252 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:49.157338 15089 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:49.155059 14982 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:49.148904 15081 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:13:49.187014 14992 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:49.213881 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:49.217751 11257 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.215693 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:49.215794 15089 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.220901 15089 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:49.209434 14815 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.224787 11325 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:49.226756 15074 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:49.209746 15071 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:13:49.230406 15071 consensus_queue.cc:1243] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:49.234748 14824 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:49.235317 14824 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:49.235728 14824 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:49.237850 15073 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.239449 15073 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:49.248674 14995 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:49.249122 14995 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:49.251454 15075 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:49.251658 15052 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.252342 15052 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.252197 14951 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.254383 14951 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:49.254733 15052 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:49.255734 14995 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.256875 14995 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:49.202154 14952 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:13:49.263465 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46984.9us
I20260709 12:13:49.206404 15057 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:49.265120 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:49.265988 11325 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:49.266777 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 267416us
I20260709 12:13:49.262555 14539 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:49.274881 14539 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.265336 15071 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.277180 15071 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.280354 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2224) took 1122 ms. Trace:
I20260709 12:13:49.280798 11331 rpcz_store.cc:276] 0709 12:13:48.171306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.357866 (+186560us) service_pool.cc:224] Handling call
0709 12:13:49.280107 (+922241us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b54000e6354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:49.280285 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118241024}
I20260709 12:13:49.281304 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:49.282829 15071 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:49.286490 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71036.9us
I20260709 12:13:49.289740 11325 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:49.299232 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:49.303253 11330 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:13:49.306249 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:49.245225 15067 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.298278 15075 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:49.266134 14796 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.316409 14815 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.317193 14796 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.304528 15071 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.326013 15071 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.328284 15071 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.317880 15075 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:49.329514 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:49.330217 11252 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:49.330258 15064 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.329888 15075 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.331789 15075 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:49.332566 15064 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.333760 11249 raft_consensus.cc:1217] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[5.5-10.8]   Dedup: 7.6->[8.7-10.8]
I20260709 12:13:49.338634 11249 pending_rounds.cc:85] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 6
I20260709 12:13:49.338904 11249 pending_rounds.cc:107] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:13:49.339130 11249 raft_consensus.cc:2889] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:49.340301 14796 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:49.333971 11177 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:49.335436 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:49.341073 14796 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:49.298190 14992 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:49.342077 14796 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.317867 11935 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:13:49.342502 14992 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.346066 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:49.345441 14840 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.782s	user 0.004s	sys 0.008s
I20260709 12:13:49.348882 14840 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:49.349195 14824 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.354151 11322 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:49.355036 15067 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.308794 15090 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:49.356915 15090 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:49.358337 14824 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.351480 11252 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:49.352511 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:49.366839 15052 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:49.367408 15052 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.367861 15052 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.370584 15088 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:49.378831 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:49.379731 15090 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:49.380142 15090 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:49.380545 15090 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.393756 15090 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.394688 15065 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:49.396571 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:49.399828 14824 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:49.400265 14824 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:49.400658 14824 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.404685 14965 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:49.406215 11927 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:49.411363 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1795) took 2922 ms (client timeout 2586 ms). Trace:
W20260709 12:13:49.412012 11250 rpcz_store.cc:269] 0709 12:13:46.488555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.728848 (+240293us) service_pool.cc:224] Handling call
0709 12:13:49.411051 (+2682203us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b5400277c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:49.411269 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168319232}
I20260709 12:13:49.412719 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.413686 11180 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:49.417958 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.409399 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 679) took 3420 ms. Trace:
W20260709 12:13:49.401438 11929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 591) took 3775 ms (client timeout 1945 ms). Trace:
I20260709 12:13:49.419678 11927 rpcz_store.cc:276] 0709 12:13:45.989079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.999603 (+ 10524us) service_pool.cc:224] Handling call
0709 12:13:49.405775 (+3406172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.406759 (+   984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:49.409121 (+  2362us) spinlock_profiling.cc:243] Waited 3.41 s on lock 0x7b54005afd50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:49.409302 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7832473472}
W20260709 12:13:49.423287 11929 rpcz_store.cc:269] 0709 12:13:45.626301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.626556 (+   255us) service_pool.cc:224] Handling call
0709 12:13:49.401073 (+3774517us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54005afd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:49.401254 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291287808}
I20260709 12:13:49.376693 15089 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.430855 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.432150 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.405496 14965 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:49.432785 11175 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:49.439819 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.441614 11180 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:49.439222 11178 raft_consensus.cc:2376] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11. Re-sending same reply.
I20260709 12:13:49.409852 14815 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:49.377329 15052 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.449651 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.446651 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:49.453044 11257 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:49.452143 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.457008 11173 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:49.458714 11927 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:13:49.451503 11250 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:49.460938 11176 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:13:49.451531 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:49.464427 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:49.467290 11927 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:13:49.467384 15057 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:49.468032 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.468488 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:49.468750 15068 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.469303 15068 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:49.470459 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.450416 14976 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:49.465204 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:49.475046 11178 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:49.475728 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:49.476284 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.447623 14965 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:49.499956 11932 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:49.500593 11173 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:49.498943 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:49.504662 11176 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:49.504985 11250 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:49.505831 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 203090us
I20260709 12:13:49.506068 11331 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:49.513653 11180 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:49.474661 14840 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.474407 14969 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.469534 15057 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:49.477056 15052 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.516479 15072 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:13:49.519197 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.519663 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.505302 14976 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.474771 15006 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:49.498587 14815 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:13:49.524482 14840 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.537971 15052 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:49.498123 15068 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:49.538820 14840 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:49.472752 14824 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:49.548698 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.551335 11178 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:49.551656 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:49.521409 15081 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:49.555806 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:49.559723 11324 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:49.533147 14976 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.524722 14969 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:49.562220 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.562080 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:49.562885 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:49.538846 15052 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:13:49.543495 15057 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:49.568375 11326 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.566498 14965 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:49.578504 15104 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:13:49.604625 11330 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:13:49.605090 11330 pending_rounds.cc:85] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:13:49.605378 11330 pending_rounds.cc:107] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:13:48.557134 14984 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.606356 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.606601 14984 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:49.606966 11175 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:49.608616 11330 raft_consensus.cc:2889] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:49.574069 15108 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:13:49.578470 14824 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:13:49.580224 11930 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:49.580616 14969 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.581727 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.583084 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:49.550834 15070 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:13:49.583621 15101 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:13:49.586828 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:49.542768 15006 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:49.593070 14998 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.047s	user 0.012s	sys 0.001s
I20260709 12:13:49.593950 15089 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.564803 15090 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:49.594331 11327 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:49.615146 11177 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:13:49.594621 11328 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 8: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:49.594868 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.598616 14952 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:49.608979 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.621449 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:49.609997 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 147631us
W20260709 12:13:49.610498 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63654.5us
I20260709 12:13:49.624929 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:49.625514 11257 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:49.611565 11324 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:49.613363 11331 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
W20260709 12:13:49.608522 11352 debug-util.cc:398] Leaking SignalData structure 0x7b080003d680 after lost signal to thread 10014
I20260709 12:13:49.618181 11177 pending_rounds.cc:85] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 9
I20260709 12:13:49.628171 11177 pending_rounds.cc:107] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:13:49.628449 11177 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:49.615769 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.609774 11323 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
W20260709 12:13:49.628734 11352 debug-util.cc:398] Leaking SignalData structure 0x7b08000382c0 after lost signal to thread 11113
W20260709 12:13:49.630473 11352 debug-util.cc:398] Leaking SignalData structure 0x7b08000530a0 after lost signal to thread 11206
I20260709 12:13:49.631970 11329 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
W20260709 12:13:49.632740 11352 debug-util.cc:398] Leaking SignalData structure 0x7b0800301ec0 after lost signal to thread 11281
I20260709 12:13:49.634541 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.636009 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:49.639472 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:49.640334 11256 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:13:49.642191 11182 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:46.399407 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:49.644523 14998 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:49.565887 15081 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:49.647444 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.614677 15070 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:49.626901 14952 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.646318 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:49.668561 11930 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:49.637485 11174 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:49.670743 11250 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:49.671572 11326 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:49.671478 11178 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
W20260709 12:13:49.672506 11352 debug-util.cc:398] Leaking SignalData structure 0x7b0800263360 after lost signal to thread 11355
W20260709 12:13:49.673028 11352 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9740 after lost signal to thread 11356
I20260709 12:13:49.673870 11180 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:49.672896 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.676249 11173 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:49.680712 11181 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:13:49.683368 11178 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:13:49.684432 11173 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:49.689384 11174 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:13:49.769459 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 100549us
I20260709 12:13:49.799980 11175 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:13:49.814327 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 96102.3us
W20260709 12:13:49.815804 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 134120us
I20260709 12:13:49.845679 11180 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:49.845824 14539 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:49.851022 14816 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:49.857134 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 368092us
I20260709 12:13:49.858708 11257 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
W20260709 12:13:49.890599 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.939074 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:49.942989 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:49.945055 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87109.6us
I20260709 12:13:49.946950 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:49.951561 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:49.954597 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:49.955490 11930 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.963474 14539 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:49.965607 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:49.969499 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.970016 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.970178 11180 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:49.970608 11178 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:49.973879 14840 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:49.974498 14840 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:49.762418 15089 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.195s
I20260709 12:13:49.976989 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:49.942202 15083 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:13:49.927212 14824 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.982899 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.984850 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:49.990254 11178 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:49.990931 11930 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:49.994189 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.995751 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.996416 11178 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:49.989801 14539 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:50.006166 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:49.979210 15075 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:13:50.007278 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:50.006507 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.008431 11175 raft_consensus.cc:2410] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:13:49.996567 15006 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:49.944828 15070 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.011787 11178 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:50.000077 14982 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:50.016645 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:49.991087 14824 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.020910 11928 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:13:50.020756 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.031327 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.032555 11178 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:50.015364 15051 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:50.034804 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.036618 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.006515 14969 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:13:50.021382 14982 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:50.040288 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.050987 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.051893 11177 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:50.053802 11326 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:13:50.006593 15099 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:13:50.058714 11327 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 12:13:50.020757 15068 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:13:50.052608 15090 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
W20260709 12:13:50.065013 15114 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:13:50.066138 14952 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.033809 14539 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:50.072934 15036 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:50.073781 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.074492 11352 debug-util.cc:398] Leaking SignalData structure 0x7b0800093aa0 after lost signal to thread 11959
I20260709 12:13:50.078054 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.080503 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1766) took 6473 ms. Trace:
I20260709 12:13:50.053133 15051 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:50.082690 11933 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:50.119148 11248 rpcz_store.cc:276] 0709 12:13:43.606702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.733974 (+2127272us) service_pool.cc:224] Handling call
0709 12:13:48.665073 (+2931099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.731521 (+ 66448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.080254 (+1348733us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b54001cf7d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:50.080416 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6691974912}
I20260709 12:13:50.083179 14824 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:50.041271 15065 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:50.017481 14840 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.054289 15089 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:13:50.057081 14982 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:50.085928 15057 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:50.080912 14976 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.100044 15099 consensus_queue.cc:1243] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:13:50.119060 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.137712 11901 connection.cc:256] RPC call timeout handler was delayed by -1.1719167510000261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
I20260709 12:13:50.126454 15065 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.125384 15051 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.126700 11935 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:50.126168 14952 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:50.122615 11933 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:50.140076 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.153548 11326 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:13:50.154021 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.155213 11327 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
W20260709 12:13:50.157100 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.157238 11934 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:50.145296 15065 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:50.144949 15036 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.165514 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.166826 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:50.166668 11933 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.170327 11933 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:50.167994 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72377.5us
W20260709 12:13:50.172626 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.641s (SENT)
I20260709 12:13:50.173062 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.140579 15057 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.175277 11327 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:50.176218 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:50.178980 11931 raft_consensus.cc:2376] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10. Re-sending same reply.
I20260709 12:13:50.183271 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:50.184533 11932 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:13:50.183759 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:50.174070 15036 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.187955 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.191172 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.175423 14824 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:50.159332 14539 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.204519 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:50.205160 14965 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:13:50.195355 14952 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:50.247478 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:50.248433 14965 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:50.248838 14965 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.249197 14965 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:50.224684 11176 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:13:50.195134 14984 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:50.196390 15108 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:13:50.254464 15108 consensus_queue.cc:1243] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:13:50.257292 15090 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.257810 15090 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.259956 15090 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:50.260893 11327 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:50.260620 14965 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.260996 15090 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:13:50.263855 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.229424 15091 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:50.239500 15057 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:50.240618 15117 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:13:50.243510 15036 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:50.244369 14976 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:50.262409 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.282415 11329 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:50.265489 11327 raft_consensus.cc:1217] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-9.9]   Dedup: 6.7->[8.8-9.9]
I20260709 12:13:50.283499 11327 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [6.7-9.9]
I20260709 12:13:50.259951 14877 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.284792 14877 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.287717 14877 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.287514 14982 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.270372 11933 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:13:50.289793 14877 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.290599 14982 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:50.290441 14877 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:50.293239 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.296463 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.267131 15091 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:50.266855 14829 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 12:13:50.309828 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.298259 15088 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:13:50.317130 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:50.317960 11331 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:50.319689 14877 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.320273 14877 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.320415 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.320751 14877 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.321101 11248 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:13:50.322484 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.322909 14877 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:50.323146 11248 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:50.314036 14952 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:50.321211 11256 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:50.324661 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.325436 11248 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:50.325721 15091 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.326715 15091 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:50.327142 11248 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:13:50.326998 15065 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.328648 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.329051 15065 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.330117 11331 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:50.316278 14965 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:50.333849 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.334806 11248 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:50.338765 11248 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:13:50.338675 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.341398 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.343736 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.343827 15081 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.345583 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.346586 11256 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:50.346704 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.352016 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2220) took 2965 ms. Trace:
I20260709 12:13:50.352499 11322 rpcz_store.cc:276] 0709 12:13:47.387115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.735955 (+1348840us) service_pool.cc:224] Handling call
0709 12:13:48.736259 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.354563 (+618304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.351739 (+997176us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b54004597d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:50.351941 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420490624}
I20260709 12:13:50.360580 14841 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.361200 14841 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.364168 14841 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:50.369131 15129 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:13:50.370404 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.372427 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.374719 11248 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:13:50.376087 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.377234 11248 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 12:13:50.378206 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.378799 11248 raft_consensus.cc:2376] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10. Re-sending same reply.
I20260709 12:13:50.379683 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.380234 11248 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:50.381090 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.382052 15070 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:50.382632 11256 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:13:50.384119 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.384848 11256 raft_consensus.cc:2376] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11. Re-sending same reply.
I20260709 12:13:50.385880 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.387686 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.388406 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.389261 11249 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:50.391814 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.394647 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1822) took 1210 ms. Trace:
I20260709 12:13:50.388386 11256 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:50.396828 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.397516 11248 raft_consensus.cc:2368] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10.
I20260709 12:13:50.400882 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:50.401310 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.401538 11930 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:50.402027 11252 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:50.395081 11254 rpcz_store.cc:276] 0709 12:13:49.184587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.069693 (+885106us) service_pool.cc:224] Handling call
0709 12:13:50.070207 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:50.070745 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.070797 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:50.071536 (+   739us) log.cc:844] Serialized 52 byte log entry
0709 12:13:50.166360 (+ 94824us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:50.261931 (+ 95571us) log.cc:844] Serialized 14 byte log entry
0709 12:13:50.270436 (+  8505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.270550 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:50.378181 (+107631us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b180040a200. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:50.394328 (+ 16147us) raft_consensus.cc:1685] finished
0709 12:13:50.394436 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.394570 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117632,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":8276,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":19785,"spinlock_wait_cycles":5048448,"thread_start_us":180115,"threads_started":2,"wal-append.queue_time_us":220771}
I20260709 12:13:50.407930 15131 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.408429 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:50.408547 15131 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.409888 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:50.410972 15131 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.413476 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.414422 11254 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:13:50.415591 15112 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:50.417222 15112 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:50.417616 15112 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.417999 15112 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:50.418562 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:50.419210 11933 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:50.422575 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.423537 14841 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:50.427446 15131 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:50.427922 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:50.428210 15131 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:50.429484 15131 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.431556 11173 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:13:50.434741 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:50.435864 15132 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:50.444852 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.450835 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.451886 11250 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 6.
I20260709 12:13:50.456485 15081 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.457204 15081 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.459887 15081 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:50.570511 11222 leader_election.cc:336] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.596s (SENT)
I20260709 12:13:50.571226 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:50.571640 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.572080 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.572501 15131 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:50.575281 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:50.576028 15132 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:50.577781 15132 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:50.579037 15131 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:50.579214 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:50.579573 15131 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:50.579912 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 550) took 1106 ms. Trace:
I20260709 12:13:50.580467 15131 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.580333 11182 rpcz_store.cc:276] 0709 12:13:49.473062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.498477 (+ 25415us) service_pool.cc:224] Handling call
0709 12:13:50.579695 (+1081218us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54002b7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:50.579821 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245490560}
I20260709 12:13:50.585099 15132 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:50.585804 15132 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:50.586760 15132 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.590198 14841 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:50.593842 15139 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:50.590081 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 164468us
I20260709 12:13:50.602394 15132 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:50.601501 15139 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:50.604205 15139 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:50.607309 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.608652 14841 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:50.610563 14841 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:50.609073 15139 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.611667 15139 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:50.612321 15091 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:50.614732 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.617956 15091 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:50.620538 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.620906 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.620059 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:50.621685 11251 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:50.622009 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.623140 15142 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:50.623512 15142 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:50.625684 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:50.626183 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.014s	sys 0.016s
W20260709 12:13:50.626566 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.014s	sys 0.016s
I20260709 12:13:50.626748 11256 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:50.626863 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 635032us
I20260709 12:13:50.627897 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:50.628266 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.629206 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:50.630420 11256 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:13:50.631494 14840 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:50.632366 11256 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: [8.7-10.8]
I20260709 12:13:50.633664 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.624081 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.634485 11251 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:50.624226 15091 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:50.634881 15139 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:50.635301 15091 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.635898 11256 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:50.636330 15116 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:50.636772 15116 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.625460 15142 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.625402 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:50.637110 15139 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:50.642894 15139 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:50.643586 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:50.638691 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.645242 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.645618 11251 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:50.623762 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.633816 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2220) took 1027 ms. Trace:
I20260709 12:13:50.641458 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.640535 14841 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:50.637123 15116 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:50.653760 14816 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:50.638321 14840 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:13:50.638326 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.655288 11248 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:50.644035 15139 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.639681 15142 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:50.637959 15089 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:13:50.635840 15091 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:50.635800 14824 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:50.659389 15089 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:50.668111 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.668897 11175 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:50.653808 14841 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:50.641047 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.670054 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.670017 14841 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.670761 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.671279 15068 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:50.671907 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:50.672128 15068 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:50.673265 11296 leader_election.cc:400] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10.
I20260709 12:13:50.665900 11181 rpcz_store.cc:276] 0709 12:13:49.606627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.648773 (+ 42146us) service_pool.cc:224] Handling call
0709 12:13:49.680456 (+ 31683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.681266 (+   810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.633595 (+952329us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b54000d9b50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:50.633754 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64688896}
I20260709 12:13:50.668787 15089 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:13:50.674037 15057 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:50.674432 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.674817 15057 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:50.675487 15057 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:50.675643 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.665997 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.668838 14994 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:50.659982 11253 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:13:50.666951 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.676599 11182 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:50.677709 11251 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:50.671310 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.664908 15116 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.681900 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.690404 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:50.691645 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:50.698879 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55399.5us
I20260709 12:13:50.700109 14796 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:50.701061 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:50.701211 15145 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.701826 15145 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.702410 15146 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.702929 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:50.703006 15146 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.703809 15145 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:50.704878 15145 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.705212 15146 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.705474 15145 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.708302 15145 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.650388 15143 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:13:50.709270 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.710455 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.676970 14824 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:50.676019 15057 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:50.673527 15068 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:13:50.716642 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.709206 15139 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:50.722880 11182 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:50.725363 14840 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:50.712476 14994 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:50.726168 14994 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:50.716146 15057 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:50.726683 11329 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:50.739490 15150 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:50.740428 15150 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.744792 15150 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.745719 15150 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:50.746433 15150 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.718186 15143 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:50.747839 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.716338 15146 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:13:50.757543 11182 raft_consensus.cc:2368] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10.
I20260709 12:13:50.761025 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.762131 11182 raft_consensus.cc:2410] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:13:50.726197 14840 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:50.716542 15145 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:13:50.766202 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.766861 11248 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:50.766326 11251 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:13:50.768217 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.768823 11182 raft_consensus.cc:2376] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10. Re-sending same reply.
I20260709 12:13:50.769589 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.770170 11182 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:13:50.770215 15065 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:50.778505 11182 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
W20260709 12:13:50.779616 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.779456 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:50.780581 11253 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:50.780527 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.780833 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.781385 11256 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:50.781407 11251 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:50.757159 15143 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:50.780009 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.787385 11182 raft_consensus.cc:2376] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9. Re-sending same reply.
W20260709 12:13:50.787894 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48120.2us
I20260709 12:13:50.790077 11251 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:13:50.789780 15149 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:50.790766 11256 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 12:13:50.791599 11256 pending_rounds.cc:85] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 3
I20260709 12:13:50.791848 11256 pending_rounds.cc:107] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 12:13:50.792092 11256 raft_consensus.cc:2889] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:50.792578 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.794219 11251 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:13:50.792119 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:50.788990 15143 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:50.802333 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.803305 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.803777 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.804996 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.807549 11322 raft_consensus.cc:2368] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9.
I20260709 12:13:50.807915 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:50.808285 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.808633 11251 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:50.809165 11182 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:50.764182 14824 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.812772 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.813033 11148 leader_election.cc:400] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9.
I20260709 12:13:50.801259 14841 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:50.814160 11148 leader_election.cc:403] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:50.814198 11322 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:50.816040 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.816056 11323 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:13:50.816707 11182 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:13:50.817492 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:50.821131 15152 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.821373 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.822091 15151 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.822661 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:50.823259 11323 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:50.822669 15151 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.826567 15150 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:50.827248 15150 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:50.810187 15146 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.833137 15146 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.835989 15146 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.815723 14841 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:50.850378 14841 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:50.818367 14840 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.853936 15116 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:50.854578 15112 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:50.860996 15152 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.864087 15152 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:50.898645 11322 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:13:50.821547 14824 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:50.950974 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 138103us
I20260709 12:13:50.954280 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:50.956796 11323 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:13:50.959156 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 267550us
I20260709 12:13:50.960594 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:50.961364 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:50.962541 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:50.963295 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:50.964629 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:50.966046 11323 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:50.972633 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:50.978029 11322 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:13:50.978792 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.981145 11182 raft_consensus.cc:2376] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11. Re-sending same reply.
I20260709 12:13:50.984340 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.986692 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.988025 11182 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:50.990131 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:50.991827 15150 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:50.992468 15150 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:50.992777 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.996050 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:50.997287 11182 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:50.999634 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:51.000629 11182 raft_consensus.cc:2376] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9. Re-sending same reply.
I20260709 12:13:51.001863 15151 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.002751 15151 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:51.003366 15151 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:51.004626 15151 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.006947 15151 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:50.960976 15065 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9.
I20260709 12:13:51.008428 15147 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:51.010726 15143 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.002737 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:51.017462 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.018600 15145 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:13:51.018484 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45971.7us
I20260709 12:13:51.019241 15145 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.019726 15145 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:51.020272 15070 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:50.968194 15146 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:51.022472 15146 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:51.023876 15146 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:50.965176 15116 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:13:51.026255 15147 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:51.026832 15147 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:50.965777 15068 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:51.028086 15068 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.028793 15068 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:51.028012 15147 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.033854 15147 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:51.040162 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.042922 11182 raft_consensus.cc:2376] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9. Re-sending same reply.
I20260709 12:13:51.049050 15091 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.055511 15150 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.079604 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:51.080729 11182 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:51.068710 14841 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.069283 15131 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.073850 14539 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:13:51.074951 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.042239 14796 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:13:51.088857 11180 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.118s	user 0.018s	sys 0.000s
I20260709 12:13:51.090662 11248 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:13:51.093236 11180 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7.
I20260709 12:13:51.093825 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1956) took 1126 ms. Trace:
I20260709 12:13:51.094759 11177 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:51.098683 11323 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:13:51.100034 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1793) took 4754 ms (client timeout 1706 ms). Trace:
I20260709 12:13:51.094143 11180 rpcz_store.cc:276] 0709 12:13:49.967453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.969247 (+  1794us) service_pool.cc:224] Handling call
0709 12:13:51.093765 (+1124518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:51.109530 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.110327 11180 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
W20260709 12:13:51.100562 11255 rpcz_store.cc:269] 0709 12:13:46.346035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.487031 (+140996us) service_pool.cc:224] Handling call
0709 12:13:51.089754 (+4602723us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54000f8cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:51.099962 (+ 10208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460755200}
I20260709 12:13:50.970225 15149 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:51.114310 11330 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:13:51.115222 11326 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:51.091960 14952 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.116425 14829 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:51.128753 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 164154us
I20260709 12:13:51.129897 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.101133 15143 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.091899 14539 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:51.133026 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.136026 11182 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 12:13:51.138865 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46156.9us
I20260709 12:13:51.135674 14539 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.128304 14796 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:51.145182 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.125990 14824 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:51.132017 15112 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.148661 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:51.152724 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 360706us
I20260709 12:13:51.153399 11182 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:51.108623 15131 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.156276 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:51.137199 14952 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.158632 15057 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.159670 14952 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.161046 15057 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.161494 15153 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:51.162199 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.164179 11180 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:13:51.166827 11180 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 10. Ops: []
I20260709 12:13:51.146840 15150 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:51.169144 11180 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
I20260709 12:13:51.171286 11180 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 8. Current term is 9. Ops: []
W20260709 12:13:51.172408 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.152660 14824 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:51.188035 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.144462 14840 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:13:51.160898 15068 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.148967 15143 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:13:51.151304 15116 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:13:51.199615 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.190129 15153 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:51.190523 14824 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:51.191645 14796 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.202795 15161 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.201573 15112 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:13:51.213387 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 80455.3us
W20260709 12:13:51.214872 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.216215 15131 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:51.213207 15127 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.929s	user 0.000s	sys 0.005s
I20260709 12:13:51.212016 15142 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:51.204809 15159 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:13:51.226430 15159 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:51.226871 15159 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:51.227799 15159 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.229074 15159 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:51.211112 14796 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.232225 14796 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.233183 15091 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:51.214118 15112 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.235536 15112 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:51.238554 14998 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:51.238977 14998 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:51.239950 14998 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.240942 14998 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:51.244031 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44526.3us
I20260709 12:13:51.244949 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.246378 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.249473 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.250325 15068 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.250356 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2241) took 1215 ms. Trace:
I20260709 12:13:51.251359 11178 rpcz_store.cc:276] 0709 12:13:50.034868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.042126 (+  7258us) service_pool.cc:224] Handling call
0709 12:13:51.250295 (+1208169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:51.250850 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2229) took 1435 ms. Trace:
I20260709 12:13:51.253369 11173 rpcz_store.cc:276] 0709 12:13:49.815249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.941253 (+126004us) service_pool.cc:224] Handling call
0709 12:13:51.250676 (+1309423us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b54002b6454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:51.250794 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118052096}
I20260709 12:13:51.255735 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:51.258178 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.258713 15091 raft_consensus.cc:2764] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:13:51.257261 15151 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:13:51.262156 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.262879 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:51.263947 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:51.264194 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:51.265086 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.265488 15150 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:13:51.265735 15165 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:13:51.265973 15150 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:51.266381 15150 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:51.267532 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.267738 14824 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.271075 15091 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:51.271687 15091 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:51.272715 15091 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.272802 15112 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.273777 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:51.278661 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:51.269539 14829 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:51.282120 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.282258 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.277513 15151 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:51.289703 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.280709 15051 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:51.290408 15162 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.280354 15163 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:51.292043 14796 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:51.294143 14796 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:51.295015 14796 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.298574 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.299831 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:51.300441 11329 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:51.291594 14824 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:51.299332 11328 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:51.306936 15051 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:13:51.307623 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.308248 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.308552 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.308774 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:51.309002 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.308936 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.310365 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:51.310611 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.311483 11254 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:13:51.315822 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.318007 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.319119 15151 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:51.322631 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.324332 11327 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:13:51.319615 15151 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.290670 14998 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:51.327000 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.335047 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.335445 11328 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:51.333339 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.323508 15160 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:51.317569 15142 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:51.325798 15131 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.349864 14539 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:51.350219 14992 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:13:51.351662 11250 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:13:51.353756 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:51.355804 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:51.356436 11257 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:13:51.356706 11326 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:13:51.358062 11326 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:51.364974 11252 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:51.324062 14952 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.341044 15112 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:51.337818 14998 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.336376 15151 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:51.341152 15153 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.358261 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:51.374817 11257 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:51.366493 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:51.376223 11248 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:51.376727 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.339764 11329 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:51.366523 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:51.369817 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.380685 11928 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:51.381402 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.381822 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:51.370311 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.384380 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:51.388185 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:51.397439 11329 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 12:13:51.356117 15116 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:13:51.403499 11248 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:13:51.390170 14877 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.399099 15065 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:51.390622 15139 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:51.378189 15143 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.366928 15081 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:13:51.408396 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.384841 15131 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:51.415411 14994 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.466812 14877 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.474401 14994 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:51.475750 14877 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.476282 14877 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:51.373420 15099 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:13:51.368904 15145 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.421269 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.372891 14952 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.399385 15153 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:51.429808 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.502556 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 94275.5us
I20260709 12:13:51.503741 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.504001 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:51.504909 11248 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:51.460237 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 90018.4us
W20260709 12:13:51.468020 11278 debug-util.cc:398] Leaking SignalData structure 0x7b0800164fa0 after lost signal to thread 10014
I20260709 12:13:51.468698 11329 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:51.470959 11257 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:51.472157 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:51.508183 11930 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:51.472777 15143 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.480856 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:51.461524 15159 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:51.488387 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.510848 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:13:51.513810 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92657.5us
I20260709 12:13:51.515157 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.515789 11329 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:51.531121 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:51.530637 11248 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:51.534737 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.535017 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.530467 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:51.512765 15068 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:51.483973 15065 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:51.537433 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.476711 14877 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.551826 14877 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.552484 14877 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.539201 15068 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:51.530366 15131 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:51.534056 15147 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:51.522789 15180 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:51.538295 14998 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.563997 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.567155 11329 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:51.563550 14829 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.555070 14877 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:51.583828 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.585704 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.523710 15159 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.537163 14992 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:13:51.531385 15150 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:51.591480 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60430.8us
I20260709 12:13:51.585382 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:51.588461 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.561369 15147 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.584702 15131 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:51.587646 15174 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:51.643126 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:51.643586 15169 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:13:51.643988 11175 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:51.644716 14952 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:51.645046 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.645347 14952 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:51.645982 15175 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:13:51.610404 11176 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:13:51.647876 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.649139 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.630407 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.640298 14992 consensus_queue.cc:1243] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:13:51.640107 15174 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:51.650396 14992 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:13:51.632302 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47003.7us
I20260709 12:13:51.651343 11175 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:51.651875 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.652351 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.643292 15099 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.638203 14877 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:13:51.654613 15099 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.655896 14877 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.656581 14877 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.657006 15099 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:51.666426 15170 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 12:13:51.668869 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:51.670697 15173 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:13:51.672950 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.588088 14829 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.674814 11322 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:13:51.676142 14829 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.675748 15065 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.677857 11329 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:51.653299 15088 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.608922 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.517040 15153 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.650573 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:51.842514 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 144342us
I20260709 12:13:51.721576 11325 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:51.651260 15174 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.653443 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:51.844751 11176 raft_consensus.cc:2410] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:13:51.704735 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.688061 11331 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:51.881793 15057 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:51.883797 11175 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:51.884415 15145 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.886060 15159 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.886621 15159 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.886853 15153 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.887081 15159 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.640524 15161 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:51.887946 15161 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:51.888319 15161 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:51.889194 15159 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:51.884127 15057 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.890826 15162 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.891294 15057 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.887430 15153 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.892040 15153 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.891314 15162 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.894162 15153 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.894779 15162 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.898121 11178 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:13:51.898571 11178 pending_rounds.cc:85] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 5
I20260709 12:13:51.899253 11178 pending_rounds.cc:107] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 8.6
W20260709 12:13:51.899531 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.899991 11178 raft_consensus.cc:2889] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:51.901821 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:51.904062 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 199417us
I20260709 12:13:51.639147 15147 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:51.919217 11248 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:13:51.922276 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:51.916860 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1655) took 1209 ms. Trace:
I20260709 12:13:51.922551 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:51.916751 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.924849 11257 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:13:51.638751 15152 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.928856 11323 rpcz_store.cc:276] 0709 12:13:50.707234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.967158 (+259924us) service_pool.cc:224] Handling call
0709 12:13:50.967472 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.099280 (+131808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.916603 (+817323us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54003241d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:51.916796 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300701056}
I20260709 12:13:51.931316 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2238) took 1908 ms. Trace:
W20260709 12:13:51.933862 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 427453us
I20260709 12:13:51.935613 11900 leader_election.cc:400] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10.
I20260709 12:13:51.935938 11900 leader_election.cc:403] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:51.685412 15150 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:51.941591 15057 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.921733 15153 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:51.942389 15153 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:51.943254 11278 debug-util.cc:398] Leaking SignalData structure 0x7b0800164c20 after lost signal to thread 11113
I20260709 12:13:51.943511 11068 maintenance_manager.cc:419] P bbfb3415e272477dac9ba599cc40a89d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033780
I20260709 12:13:51.944940 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.945271 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:51.939041 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:51.946257 11248 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:51.932068 11177 rpcz_store.cc:276] 0709 12:13:50.009707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.114369 (+104662us) service_pool.cc:224] Handling call
0709 12:13:51.094591 (+980222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.095155 (+   564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.917778 (+822623us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54000d7850. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:51.917903 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235890944}
I20260709 12:13:51.934669 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.945593 11325 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:51.939466 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:51.948944 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:51.949657 11325 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:51.948915 11323 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:51.951378 11327 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:13:51.921875 15162 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:51.980296 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:51.925391 15159 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:51.981760 11257 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:13:51.641799 15151 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:51.935302 15145 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:51.982020 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48454.9us
W20260709 12:13:51.984165 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48689.2us
W20260709 12:13:51.981321 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.993163 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:52.006534 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:52.012238 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.899680 14829 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 12:13:51.687871 15131 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:51.896665 15099 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:13:51.996415 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:52.007359 11257 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:52.050495 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 105327us
W20260709 12:13:52.078210 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.935094 15088 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 12:13:52.146064 11250 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.007s	sys 0.022s
I20260709 12:13:52.097395 11248 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.097834 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:52.094470 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:52.151652 11930 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:52.151683 11257 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:52.106431 11255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.005s	sys 0.022s
W20260709 12:13:52.152964 11255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.005s	sys 0.023s
I20260709 12:13:51.988420 15159 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:51.945937 15152 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:52.172232 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 175913us
I20260709 12:13:52.178128 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:52.193841 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:52.195484 11325 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.197176 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:52.144115 15162 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:52.093534 15188 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.094225 15132 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:52.216099 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:52.210902 11322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.025s
I20260709 12:13:51.982692 15189 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.245s
I20260709 12:13:52.215474 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.225330 15193 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:52.215827 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.191021 15174 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:52.234683 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.234984 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:52.235381 11175 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:52.196451 15159 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:52.230479 11173 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:52.236636 11322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.005s	sys 0.025s
I20260709 12:13:52.226962 11930 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:52.195644 15194 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:52.052229 15145 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.239267 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:52.248739 11255 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:52.259924 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 224502us
I20260709 12:13:52.262419 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:52.224505 15162 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:52.235834 15153 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.218719 15190 raft_consensus.cc:2764] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:13:52.210706 15075 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:13:52.246351 15194 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:52.296476 15153 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.297039 15153 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.299207 15153 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.232632 15188 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.302292 11257 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:13:52.303351 15194 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.304638 15194 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:52.279729 15162 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:52.277717 15132 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.310600 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:52.273802 15192 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.317304 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:52.198287 15065 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.230202 15193 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.314298 15186 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:52.323431 15153 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:52.346701 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:52.351202 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:52.343405 14829 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:52.301397 15161 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.354357 11324 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:52.356367 15161 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:52.357735 14829 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.361492 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:52.362656 14952 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.717s	user 0.005s	sys 0.005s
I20260709 12:13:52.363314 14952 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:52.364184 14952 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:13:52.366307 15180 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:52.366564 15145 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.367072 15180 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:52.368441 15180 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.376144 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:52.320001 15132 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.349159 14994 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:52.377897 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.378830 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.339130 15150 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:52.326361 15065 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.384830 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.381937 11175 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:13:52.387975 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 190922us
I20260709 12:13:52.392791 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:52.389240 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.394158 11175 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:52.393579 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:52.398447 15150 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:52.398904 15150 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.399263 15150 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:52.373739 15151 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:52.388990 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 110902us
I20260709 12:13:52.403725 11251 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.322389 15192 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.380534 15199 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.410265 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:52.418269 14965 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:52.407406 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:52.392918 15153 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:13:52.419843 15195 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:13:52.407790 15188 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.396363 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:52.380517 15198 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
W20260709 12:13:52.449676 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.372146 15186 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.432011 15153 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.434690 15152 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:52.456918 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.437222 14984 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:52.436198 14994 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:52.460860 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.464330 11176 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:13:52.467967 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.434250 15159 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.456117 15153 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:52.453409 15209 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:52.472970 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.456748 15075 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.461745 14984 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:52.476750 11294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.017s	sys 0.021s
W20260709 12:13:52.478809 11294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.018s	sys 0.022s
I20260709 12:13:52.477363 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.420367 15199 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.480816 14984 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:52.497993 11255 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.499888 15206 raft_consensus.cc:2764] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:13:52.475528 15209 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:52.506320 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:52.487914 15081 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.508985 11933 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:52.514134 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:52.514816 11255 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.515823 14994 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.494489 15152 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:52.509974 15206 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.523795 11926 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:52.524856 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:52.505935 15099 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:52.481000 15075 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.512226 15215 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.511063 15209 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:52.528350 15152 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.537199 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:52.537463 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.538272 11173 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:52.537878 11926 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:52.533524 15132 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.546377 15192 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.546660 15186 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.522210 15081 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.548237 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:52.549176 11255 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.519723 15205 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:13:52.528327 15202 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.554313 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:52.528375 15065 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.559017 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:52.560047 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 93923.9us
I20260709 12:13:52.568174 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.566891 11926 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.568884 11173 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:52.560261 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 163947us
I20260709 12:13:52.566093 15163 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:52.570875 15211 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:13:52.572722 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.573522 11177 raft_consensus.cc:2376] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 7. Re-sending same reply.
I20260709 12:13:52.582660 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.578698 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:52.531513 15099 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.590478 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:52.591892 11255 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:13:52.592393 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.593070 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.599942 15124 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 2.308s	user 0.008s	sys 0.013s
I20260709 12:13:52.601460 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1962) took 2582 ms. Trace:
I20260709 12:13:52.602123 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:52.603932 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:52.576743 15152 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.601919 11179 rpcz_store.cc:276] 0709 12:13:50.019426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.125073 (+105647us) service_pool.cc:224] Handling call
0709 12:13:50.125448 (+   375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:50.129991 (+  4543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.130033 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:50.137740 (+  7707us) log.cc:844] Serialized 52 byte log entry
0709 12:13:50.196227 (+ 58487us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:50.196304 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.196403 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.547741 (+2351338us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:52.601094 (+ 53353us) raft_consensus.cc:1685] finished
0709 12:13:52.601119 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.601374 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":155,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6530,"spinlock_wait_cycles":20369280,"thread_start_us":35471,"threads_started":1,"wal-append.queue_time_us":140785}
I20260709 12:13:52.609560 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.609911 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:52.532475 15206 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.613951 15207 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:52.614432 15207 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.614796 15207 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:52.616287 15206 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.617304 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:52.624634 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:52.617959 15065 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:52.533375 15215 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:52.645277 15182 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.123s	user 0.002s	sys 0.008s
I20260709 12:13:52.647168 15215 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.647073 15153 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.648705 14841 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:52.649858 11179 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:13:52.594172 15147 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.647s	user 0.013s	sys 0.000s
I20260709 12:13:52.625965 15052 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:13:52.626920 15210 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:13:52.628794 15075 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.630396 15199 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:52.665864 11174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.663s	user 0.021s	sys 0.042s
W20260709 12:13:52.666531 11278 debug-util.cc:398] Leaking SignalData structure 0x7b08001652e0 after lost signal to thread 11206
W20260709 12:13:52.668442 11278 debug-util.cc:398] Leaking SignalData structure 0x7b080031db00 after lost signal to thread 11281
W20260709 12:13:52.668874 11278 debug-util.cc:398] Leaking SignalData structure 0x7b0800356380 after lost signal to thread 11282
I20260709 12:13:52.633710 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.567324 15202 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.634864 15220 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:52.631546 14984 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.635355 14994 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:13:52.635632 15217 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:52.635928 15159 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.619882 15205 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:13:52.638411 14952 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10.
W20260709 12:13:52.640602 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.642446 15207 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.643846 15099 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.644172 11255 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:13:52.826056 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:52.826206 15207 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.826694 15217 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:52.827689 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:52.828576 11255 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:52.828049 15217 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.656545 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:52.830700 14994 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:52.831243 14994 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.831598 14994 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:52.656627 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:52.832664 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 176190us
I20260709 12:13:52.834656 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:52.836555 14952 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:52.664891 15206 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:13:52.838346 15206 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:13:52.838544 15202 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:52.654027 14965 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:52.840356 14965 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:52.830704 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 174250us
I20260709 12:13:52.631590 15209 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:52.846342 15209 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:52.663813 15052 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:52.700420 15153 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.702615 15163 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:52.660073 15147 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.849993 14952 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:52.851123 14952 consensus_queue.cc:1243] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:13:52.852290 11175 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:52.852797 15147 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.851732 14952 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:52.853951 15153 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:52.862800 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:52.838871 15206 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:52.894863 15207 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.914624 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:52.891548 11929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.483s	user 0.003s	sys 0.033s
W20260709 12:13:52.919337 11929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.511s	user 0.004s	sys 0.033s
I20260709 12:13:52.905169 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:52.924794 11174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.925s	user 0.022s	sys 0.043s
I20260709 12:13:52.925884 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1959) took 2957 ms. Trace:
I20260709 12:13:52.896725 15068 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.898680 14952 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:52.929147 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 671) took 2535 ms. Trace:
I20260709 12:13:52.926369 11174 rpcz_store.cc:276] 0709 12:13:49.968476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.972572 (+  4096us) service_pool.cc:224] Handling call
0709 12:13:49.972872 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.975798 (+  2926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.975828 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:49.976272 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:13:52.925401 (+2949129us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:52.925456 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.925570 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.925624 (+    54us) raft_consensus.cc:1685] finished
0709 12:13:52.925657 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.925815 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":728,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11208,"thread_start_us":2925402,"threads_started":1,"wal-append.queue_time_us":153417,"wal-append.run_cpu_time_us":2925,"wal-append.run_wall_time_us":1171370}
I20260709 12:13:52.938618 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.939663 11174 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:52.928865 15220 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:52.941154 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.918653 15153 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:52.942898 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.937220 11929 rpcz_store.cc:276] 0709 12:13:50.393442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.398260 (+  4818us) service_pool.cc:224] Handling call
0709 12:13:50.398623 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.407346 (+  8723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.407377 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.407833 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:13:52.924128 (+2516295us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:52.924189 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.924349 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.924414 (+    65us) raft_consensus.cc:1685] finished
0709 12:13:52.924432 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.928650 (+  4218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":2513891,"threads_started":1,"wal-append.queue_time_us":66524,"wal-append.run_cpu_time_us":1587,"wal-append.run_wall_time_us":1002971}
I20260709 12:13:52.931430 15227 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:52.946853 11174 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:52.945858 11255 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:52.959692 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:52.959893 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:52.946499 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:52.961221 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:52.963213 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70835.8us
I20260709 12:13:52.929270 15089 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:52.945397 15153 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.936408 15081 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:52.968906 11935 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:52.920027 15207 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:52.961136 15150 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.001s
W20260709 12:13:52.963798 15194 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.000s
W20260709 12:13:52.981137 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.982863 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:52.959056 15227 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:52.987670 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:52.992007 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:52.967772 15068 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:53.002183 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 700) took 4328 ms. Trace:
I20260709 12:13:53.002610 11934 rpcz_store.cc:276] 0709 12:13:48.673924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.596331 (+922407us) service_pool.cc:224] Handling call
0709 12:13:49.596673 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.157819 (+561146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.001948 (+2844129us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540069fad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:53.002126 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276623744}
I20260709 12:13:53.001971 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.003924 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.004441 11935 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.004768 11173 raft_consensus.cc:2410] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:13:52.964493 15052 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.009158 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:52.990386 15228 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:52.996021 15227 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:52.966823 15226 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:13:53.012598 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.012718 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:53.013248 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.014137 14984 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:53.014372 15174 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.769s	user 0.007s	sys 0.006s
I20260709 12:13:53.017201 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.005204 15194 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:52.992301 15150 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.018407 11935 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:53.022428 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.024850 15233 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:53.030253 11173 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:53.034478 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:52.980134 15206 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.035391 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.035970 15194 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:53.036082 11934 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:53.036602 15131 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:53.037194 15194 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.035542 15143 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:53.037405 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:53.040246 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:53.029064 15081 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:53.008893 14952 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.046967 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:53.046934 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.048429 11173 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:53.035923 15206 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:53.048462 11935 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.017899 15205 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:13:53.048770 15143 raft_consensus.cc:2764] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:13:53.059639 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.060532 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:53.061437 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.053788 15150 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.062258 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:53.067793 11928 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:53.068360 11176 raft_consensus.cc:2410] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:13:53.072889 15207 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:53.062623 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.074739 11324 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:53.078043 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.078754 11935 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:53.079224 11928 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.080248 11928 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:53.080896 11935 pending_rounds.cc:85] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 6
I20260709 12:13:53.081204 11935 pending_rounds.cc:107] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:13:53.081354 11928 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
W20260709 12:13:53.064774 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.081442 11935 raft_consensus.cc:2889] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:53.062979 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:53.064002 11926 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.064488 11174 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:53.064595 11173 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:53.065421 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.066061 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.046124 15081 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.088757 11173 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:13:53.087225 11322 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:53.087446 11934 raft_consensus.cc:2410] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:13:53.091704 11174 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:13:53.098352 11173 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:53.098465 11174 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:13:53.098762 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:53.099581 11324 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:53.103005 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.109647 11928 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:13:53.091007 15081 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:53.123067 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.119009 11324 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:53.127089 11252 raft_consensus.cc:2410] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:53.101820 15033 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.136647 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:53.124279 15235 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.142s	user 0.000s	sys 0.002s
I20260709 12:13:53.119679 15180 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:53.139623 15033 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:53.141486 15230 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 12:13:53.142748 15179 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.669s	user 0.013s	sys 0.004s
I20260709 12:13:53.147078 11252 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:53.137863 15162 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.832s	user 0.013s	sys 0.001s
I20260709 12:13:53.152330 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:53.154920 11252 raft_consensus.cc:2376] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10. Re-sending same reply.
I20260709 12:13:53.156803 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1956) took 1804 ms. Trace:
I20260709 12:13:53.158484 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.159205 11252 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:53.150532 15162 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.157295 11256 rpcz_store.cc:276] 0709 12:13:51.352098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.354482 (+  2384us) service_pool.cc:224] Handling call
0709 12:13:51.354833 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.365551 (+ 10718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.365611 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.368864 (+  3253us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.461144 (+ 92280us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:51.461217 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.461330 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:53.069858 (+1608528us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b4800042b58. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:53.142102 (+ 72244us) spinlock_profiling.cc:243] Waited 87.3 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:53.156499 (+ 14397us) raft_consensus.cc:1685] finished
0709 12:13:53.156531 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.156721 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":924,"raft.queue_time_us":2103,"raft.run_cpu_time_us":20231,"raft.run_wall_time_us":1701074,"spinlock_wait_cycles":206204032,"thread_start_us":89831,"threads_started":1,"wal-append.queue_time_us":102235}
I20260709 12:13:53.162729 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:53.163264 11256 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:53.162861 11278 debug-util.cc:398] Leaking SignalData structure 0x7b0800367fe0 after lost signal to thread 11355
W20260709 12:13:53.164147 11278 debug-util.cc:398] Leaking SignalData structure 0x7b0800305340 after lost signal to thread 11356
I20260709 12:13:53.165354 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.165818 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2268) took 1556 ms. Trace:
I20260709 12:13:53.166512 11252 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:53.166419 11180 rpcz_store.cc:276] 0709 12:13:51.609002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.609834 (+   832us) service_pool.cc:224] Handling call
0709 12:13:51.610172 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.638267 (+ 28095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.638303 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.646610 (+  8307us) log.cc:844] Serialized 26 byte log entry
0709 12:13:52.673366 (+1026756us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b1800355ba0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:52.958899 (+285533us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:52.958958 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.959037 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:53.162243 (+203206us) raft_consensus.cc:1685] finished
0709 12:13:53.162260 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.165743 (+  3483us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14778,"raft.queue_time_us":30683,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":192876,"spinlock_wait_cycles":144767360,"thread_start_us":285252,"threads_started":1,"wal-append.queue_time_us":1363564}
I20260709 12:13:53.168067 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.168310 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:53.170535 11252 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:53.173228 15033 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:53.173444 15162 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:53.174173 15116 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:53.174587 15116 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:53.175379 15116 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.176321 15116 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:53.179287 15217 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:53.179811 15217 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.180177 15217 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:53.181980 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.183054 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:53.183403 11323 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:53.186187 11256 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.187703 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:53.188783 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.183005 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.191735 11252 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.192948 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:53.192611 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:53.192797 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.193502 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.194022 11256 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.194128 11182 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:13:53.195158 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 96337.4us
I20260709 12:13:53.189534 15068 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:53.198354 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:53.199342 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:53.201457 15145 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:53.201933 15145 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.202298 15145 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:53.202728 15201 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:53.203120 15201 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:53.203964 15201 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.204938 15201 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:53.206506 15171 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.800s	user 0.014s	sys 0.006s
I20260709 12:13:53.207818 15162 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:53.213127 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.213527 14969 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:53.214499 14969 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.214879 14969 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:53.216779 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1851) took 1267 ms. Trace:
I20260709 12:13:53.217190 11253 rpcz_store.cc:276] 0709 12:13:51.949738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.988903 (+ 39165us) service_pool.cc:224] Handling call
0709 12:13:52.141367 (+152464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.196120 (+ 54753us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:52.262105 (+ 65985us) log.cc:844] Serialized 14 byte log entry
0709 12:13:52.426677 (+164572us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:52.426757 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:52.426823 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.215987 (+789164us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b1800409960. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3eca5c058f 00007f3ec35ff571 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd
0709 12:13:53.216512 (+   525us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b540010b654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:53.216557 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.216718 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5083,"prepare.queue_time_us":125479,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":14694,"spinlock_wait_cycles":141465216,"thread_start_us":219929,"threads_started":3,"wal-append.queue_time_us":125956}
W20260709 12:13:53.218924 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.219173 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.220121 11253 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:53.213325 15145 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.166124 14994 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.223624 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:53.224818 11252 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:53.221508 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1857) took 1168 ms. Trace:
I20260709 12:13:53.199069 15052 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:53.222227 15068 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:53.226393 11253 rpcz_store.cc:276] 0709 12:13:52.052695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.220761 (+1168066us) service_pool.cc:224] Handling call
0709 12:13:53.221067 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.221234 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:53.221257 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:53.221294 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:53.221320 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.221370 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.221449 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.242187 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:53.213208 15090 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:13:53.199227 15033 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:13:53.245273 15227 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.269542 11252 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:53.236533 15052 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.257651 11253 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:53.272001 11252 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.4-10.4]
I20260709 12:13:53.285063 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1859) took 1103 ms. Trace:
I20260709 12:13:53.295220 11252 rpcz_store.cc:276] 0709 12:13:52.181689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.270900 (+1089211us) service_pool.cc:224] Handling call
0709 12:13:53.271751 (+   851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.284739 (+ 12988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.284998 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.277494 15052 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:53.296392 15162 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:13:53.299678 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49996us
I20260709 12:13:53.305830 11252 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:13:53.306542 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1861) took 1110 ms. Trace:
I20260709 12:13:53.306907 11252 rpcz_store.cc:276] 0709 12:13:52.196268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.299470 (+1103202us) service_pool.cc:224] Handling call
0709 12:13:53.305517 (+  6047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.306278 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.306482 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:53.301151 15072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.016s	sys 0.033s
I20260709 12:13:53.301731 15162 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:53.310906 15072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.017s	sys 0.033s
I20260709 12:13:53.321162 11252 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:13:53.302953 15205 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:13:53.331087 15217 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.333573 15217 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:53.334193 15205 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:53.334493 15251 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.334649 15205 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:53.335181 15251 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.335493 15205 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.336687 15205 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:53.338447 15227 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.338975 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.320788 15248 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:53.341120 11278 debug-util.cc:398] Leaking SignalData structure 0x7b0800231200 after lost signal to thread 11959
I20260709 12:13:53.339849 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.342190 15248 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.343247 15251 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:53.344677 15248 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:13:53.340111 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.339852 11180 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:53.345755 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.347587 14969 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.308017 15033 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:13:53.348219 15180 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:53.348601 11901 connection.cc:256] RPC call timeout handler was delayed by -1.7689676479999434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
W20260709 12:13:53.349041 11901 connection.cc:256] RPC call timeout handler was delayed by -1.3304917680000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
W20260709 12:13:53.349455 11901 connection.cc:256] RPC call timeout handler was delayed by -1.3078264470000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
I20260709 12:13:53.349557 15081 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.341156 15145 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.357847 15192 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:53.348855 15180 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.358475 15192 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.359006 15081 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:53.319736 15147 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:13:53.360150 15180 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.317589 15162 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:53.360685 15192 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.361117 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:53.361804 11181 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:53.349864 15237 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.362834 15237 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.364749 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.365610 15237 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.361227 15192 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.365999 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.366917 11182 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:53.366817 15192 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.348903 15033 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:53.369372 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:53.369889 15033 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:53.370167 11256 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:53.370262 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.370924 15033 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.300892 15068 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.372566 15192 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.373116 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1641) took 2687 ms. Trace:
I20260709 12:13:53.373659 11330 rpcz_store.cc:276] 0709 12:13:50.685512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.692483 (+  6971us) service_pool.cc:224] Handling call
0709 12:13:50.692850 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.118119 (+425269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.372778 (+2254659us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400429fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:53.373038 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968469888}
I20260709 12:13:53.371883 15006 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:53.371912 15090 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:53.342579 11927 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:53.377206 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:53.382157 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.384877 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:53.403988 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.405107 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.423940 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46835.6us
I20260709 12:13:53.456210 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.457310 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:53.458230 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:53.461817 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.466205 11223 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.797s (SENT) [suppressed 50 similar messages]
I20260709 12:13:53.389876 11932 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:53.426915 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:53.434659 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.470897 11180 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:53.481735 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.484972 11935 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:53.487238 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.488292 11935 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:53.372109 15033 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:53.490156 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:53.494801 11174 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:53.423532 15006 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:53.506047 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.501219 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:53.461342 15237 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:53.506058 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.507040 11174 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:53.532640 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.535370 11928 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:53.423849 15090 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.473067 15227 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:53.534447 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.475842 15154 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.373149 15251 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 12:13:53.472873 15217 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:53.475555 15073 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:53.543958 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:53.546147 11180 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:53.548848 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.462569 15145 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 12:13:53.556885 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.551554 11174 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:53.493700 15206 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
W20260709 12:13:53.538667 15178 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:13:53.539106 15090 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:53.571656 11180 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:53.572834 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:53.546446 14969 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:53.586797 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.589963 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.589874 15006 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.551811 15073 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.592286 15073 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:53.591925 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:53.555886 15116 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.590971 14829 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:53.565927 15145 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.610868 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.615345 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.552445 15154 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.556697 15217 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.621244 11928 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:53.621208 11929 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:53.651273 15217 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.652750 15052 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.641264 15145 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.653556 15073 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:53.654870 15257 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.655563 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.656854 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:53.658380 15176 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 2.186s	user 0.005s	sys 0.012s
I20260709 12:13:53.658248 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.662961 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.612609 15116 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:53.675441 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.676949 15253 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.678996 11928 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:53.664427 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:53.597368 15006 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:53.688959 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:53.695173 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:53.696822 11929 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:13:53.698621 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.780814 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:53.799494 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:53.800297 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.696604 15132 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:53.665244 15192 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:13:53.680124 15257 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:53.801579 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.800371 11929 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:53.802845 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.855602 11934 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:53.858279 11181 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:53.859521 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1955) took 2448 ms. Trace:
I20260709 12:13:53.860065 11254 rpcz_store.cc:276] 0709 12:13:51.351814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.352034 (+   220us) service_pool.cc:224] Handling call
0709 12:13:51.352333 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:51.364777 (+ 12444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.364824 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:51.369145 (+  4321us) log.cc:844] Serialized 52 byte log entry
0709 12:13:51.461614 (+ 92469us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:51.529302 (+ 67688us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.589752 (+ 60450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.589833 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.564939 (+975106us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b4800042b58. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:53.374495 (+809556us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b18003908e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:53.800210 (+425715us) raft_consensus.cc:1685] finished
0709 12:13:53.800233 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.800358 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14354,"prepare.queue_time_us":81354,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":34002,"raft.run_cpu_time_us":12260,"raft.run_wall_time_us":1198794,"spinlock_wait_cycles":13825664,"thread_start_us":159198,"threads_started":3,"wal-append.queue_time_us":92771}
I20260709 12:13:53.864148 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:53.866892 11253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.005s	sys 0.013s
W20260709 12:13:53.867906 11253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.005s	sys 0.013s
I20260709 12:13:53.869948 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2275) took 2257 ms. Trace:
I20260709 12:13:53.691489 15253 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:53.870473 11178 rpcz_store.cc:276] 0709 12:13:51.612487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.681265 (+ 68778us) service_pool.cc:224] Handling call
0709 12:13:51.897499 (+216234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.901281 (+  3782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.869623 (+1968342us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54000d9150. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:53.869864 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496493312}
I20260709 12:13:53.873005 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:53.877924 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.878898 11331 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:13:53.880313 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.874666 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:53.874779 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 265908us
I20260709 12:13:53.883478 11331 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:13:53.883862 11254 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:13:53.873203 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.885576 11322 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:53.885900 11330 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:13:53.887969 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:53.888325 11323 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:13:53.806649 15072 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:53.875345 15230 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.181s	user 0.008s	sys 0.000s
I20260709 12:13:53.884357 11254 pending_rounds.cc:85] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 6
I20260709 12:13:53.898509 11254 pending_rounds.cc:107] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 10.7
W20260709 12:13:53.880445 15162 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 12:13:53.899051 11254 raft_consensus.cc:2889] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:53.896970 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:53.905239 11330 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:13:53.914458 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1958) took 2561 ms. Trace:
I20260709 12:13:53.922824 11178 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 12:13:53.929608 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1862) took 1702 ms (client timeout 1707 ms). Trace:
W20260709 12:13:53.930192 11251 rpcz_store.cc:269] 0709 12:13:52.227268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.361158 (+133890us) service_pool.cc:224] Handling call
0709 12:13:53.929356 (+1568198us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b5400276854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:53.929527 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95361920}
I20260709 12:13:53.926757 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.889611 15132 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:53.932657 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.937135 11322 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:53.928318 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1864) took 1490 ms. Trace:
I20260709 12:13:53.939450 11254 rpcz_store.cc:276] 0709 12:13:52.437714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.873708 (+1435994us) service_pool.cc:224] Handling call
0709 12:13:53.879562 (+  5854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.927944 (+ 48382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.928234 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:53.928959 15068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.008s	sys 0.020s
I20260709 12:13:53.944984 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.915253 11250 rpcz_store.cc:276] 0709 12:13:51.352672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.352786 (+   114us) service_pool.cc:224] Handling call
0709 12:13:51.353102 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:51.353434 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.353461 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:51.354052 (+   591us) log.cc:844] Serialized 52 byte log entry
0709 12:13:51.464423 (+110371us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:51.528683 (+ 64260us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.538038 (+  9355us) log.cc:844] Serialized 14 byte log entry
0709 12:13:52.146685 (+608647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.146797 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:53.657672 (+1510875us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:53.914059 (+256387us) raft_consensus.cc:1685] finished
0709 12:13:53.914107 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.914356 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18864,"prepare.queue_time_us":94799,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":63629,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":3052,"spinlock_wait_cycles":205219840,"thread_start_us":764428,"threads_started":3,"wal-append.queue_time_us":118248}
I20260709 12:13:53.899669 15073 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:53.950996 11324 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:53.948207 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.972337 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:53.973062 11331 raft_consensus.cc:2410] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:13:53.960867 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.970196 11323 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:13:53.928857 15160 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:53.973937 11250 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
W20260709 12:13:53.947697 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 252628us
I20260709 12:13:53.975287 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:53.990615 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:53.903846 15072 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:53.943919 15070 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.636s	user 0.035s	sys 0.054s
W20260709 12:13:53.990710 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 93829.5us
W20260709 12:13:53.946382 15068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.008s	sys 0.020s
I20260709 12:13:53.992347 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1867) took 1146 ms. Trace:
I20260709 12:13:54.025899 11250 rpcz_store.cc:276] 0709 12:13:52.846171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.972127 (+1125956us) service_pool.cc:224] Handling call
0709 12:13:53.973697 (+  1570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.992051 (+ 18354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.992274 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.991544 11322 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:53.988287 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:53.945441 15264 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.947618 15266 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.925064 15162 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.023620 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:54.024824 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.990876 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:53.952720 15052 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:54.032114 11250 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:13:54.033468 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1869) took 1186 ms. Trace:
W20260709 12:13:54.033952 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 45777.9us
I20260709 12:13:54.033907 11250 rpcz_store.cc:276] 0709 12:13:52.846872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.031532 (+1184660us) service_pool.cc:224] Handling call
0709 12:13:54.031891 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.033194 (+  1303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.033413 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.037122 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:54.038187 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:54.039158 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:54.038976 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.038856 11178 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:13:54.038853 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:54.024313 15070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.728s	user 0.036s	sys 0.059s
I20260709 12:13:54.044333 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:54.046676 11324 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:54.046808 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:54.045312 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54590.3us
I20260709 12:13:54.047686 11323 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:54.021198 15226 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:54.023159 15154 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:54.060680 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:54.061816 11934 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:54.062327 11178 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:54.051985 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:54.030838 15116 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:54.080404 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.086565 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 213117us
I20260709 12:13:53.993269 15160 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.052593 15226 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.051872 15234 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:54.031090 15073 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:13:54.108373 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:54.096513 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01e75f61a948459a884630690e51c686"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
W20260709 12:13:54.113483 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.144385 11173 raft_consensus.cc:606] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.144759 11173 raft_consensus.cc:614] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:54.113742 15276 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:13:54.060047 15237 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:13:54.041549 15264 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.119942 15273 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:54.121223 15217 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:54.122401 15261 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:54.126623 14877 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.127565 15257 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:54.129025 15162 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.088551 15284 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:13:54.130177 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.130625 15072 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.062718 15227 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:13:54.042510 15266 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.131306 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:13:54.178789 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 47560.9us
I20260709 12:13:54.104962 15081 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:54.105010 15288 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:13:54.138244 15262 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:13:54.138592 15253 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.138824 15190 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:54.139182 15154 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:54.140002 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:54.148506 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:54.166345 14952 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:54.166574 15237 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:54.168306 15160 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:54.180697 15288 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.101541 15226 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:54.111865 15180 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:54.153555 15090 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.002s
W20260709 12:13:54.152900 15205 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.008s
I20260709 12:13:54.176126 11322 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:54.182905 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:54.184562 15257 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:54.186594 15266 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:54.177258 15284 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.212558 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:54.188990 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 148793us
I20260709 12:13:54.174108 15145 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.169526 15264 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.180944 15081 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:54.192847 11323 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:54.194867 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46570.3us
W20260709 12:13:54.198751 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 161649us
I20260709 12:13:54.220098 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:54.192008 11324 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:54.215462 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.212236 14982 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:54.222679 14982 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.223102 14982 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:54.223289 15081 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.224494 15081 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:54.192112 15154 consensus_queue.cc:1243] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:13:54.196847 15237 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.229199 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:54.230253 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:54.231549 15033 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.730s	user 0.013s	sys 0.000s
I20260709 12:13:54.231323 15145 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:13:54.232421 15145 consensus_queue.cc:1243] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:13:54.232749 11325 raft_consensus.cc:2410] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:54.228536 15261 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:13:54.200461 15288 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:54.197388 15283 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:54.244776 14877 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:54.245390 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:54.256136 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.210098 15257 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:54.248298 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.274051 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.217734 15284 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:54.243860 15142 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.242786 15237 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:54.285867 15142 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.210940 15090 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:54.289674 15006 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.002s
I20260709 12:13:54.277812 15257 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:54.275355 15073 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:13:54.315349 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 263425us
W20260709 12:13:54.317718 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 209356us
I20260709 12:13:54.275156 15268 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:54.331277 11178 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:54.333520 11181 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:13:54.349016 11330 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:54.329099 15234 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:54.303246 15290 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:54.340627 15291 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:54.358574 15291 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:54.359231 15142 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:54.290153 14982 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.362535 11181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fd765747f9648049b905c70696b206d"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:54.363215 11181 raft_consensus.cc:606] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.363607 11181 raft_consensus.cc:614] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:54.345813 15073 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:54.378679 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:54.393253 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:54.396502 11178 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:54.399037 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:54.354763 15154 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:54.396495 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:54.406749 11174 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:54.407835 11180 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:13:54.412470 11928 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:13:54.346220 15227 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 12:13:54.369298 15201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.011s	sys 0.006s
I20260709 12:13:54.414314 11175 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:54.414744 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:54.385903 15194 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:13:54.359558 15291 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:54.458202 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.459245 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:54.461892 15073 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.464738 14982 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.451242 15072 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:54.348456 15281 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:13:54.449187 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:54.435789 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:54.434561 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 156793us
I20260709 12:13:54.443606 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:54.467809 11181 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:54.443710 15292 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:13:54.470760 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:54.471197 11326 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:54.478186 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:54.477900 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:54.478832 11326 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:13:54.455261 15286 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 12:13:54.473541 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 217500us
I20260709 12:13:54.510988 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.511297 15263 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:54.512365 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.512290 15263 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.513378 15281 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:54.514324 15281 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:54.515604 15263 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:54.476050 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:54.521584 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:54.474726 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:54.522738 11929 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:54.523181 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:54.523849 15225 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.586s	user 0.011s	sys 0.000s
I20260709 12:13:54.524070 11930 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:54.478910 11928 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:54.473120 11935 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:13:54.450402 15201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.012s	sys 0.010s
I20260709 12:13:54.479858 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:54.475649 14982 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:54.481945 11934 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:54.482287 15142 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:54.484176 11926 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:13:54.415740 15300 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:13:54.488351 15257 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.490010 15264 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:54.490724 15033 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:54.475840 15293 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.493795 15274 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:54.451072 15162 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.503461 15070 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:13:54.504997 15112 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:54.431420 15234 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.611688 11928 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:54.507158 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:54.508792 14952 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:54.516958 15227 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.517258 15180 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:54.517732 15051 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:54.517923 15298 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:54.520120 15245 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:13:54.553838 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.570423 11926 pending_rounds.cc:85] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 8
I20260709 12:13:54.560864 11323 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:54.523738 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:54.555948 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.460769 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:54.527824 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 688) took 1347 ms. Trace:
I20260709 12:13:54.500128 15226 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.556581 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:54.811030 11325 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:54.812077 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:54.813042 11325 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:54.548549 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:54.572417 11329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1669) took 3003 ms (client timeout 1998 ms). Trace:
I20260709 12:13:54.602454 15281 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.529719 15090 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.558179 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 713) took 2053 ms. Trace:
I20260709 12:13:54.574927 11326 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:54.591224 11930 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:54.603801 11181 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
W20260709 12:13:54.535506 15254 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.889s	user 0.001s	sys 0.002s
I20260709 12:13:54.466177 15072 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:54.612551 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.825229 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 346589us
W20260709 12:13:54.533068 15255 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.877s	user 0.000s	sys 0.001s
W20260709 12:13:54.612861 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105798us
I20260709 12:13:54.557509 14877 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:54.539325 15220 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:54.528666 15189 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:54.534118 15202 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:13:54.807457 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.810433 15300 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:54.540316 15288 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:54.814304 11329 rpcz_store.cc:269] 0709 12:13:51.556355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.585413 (+ 29058us) service_pool.cc:224] Handling call
0709 12:13:54.559624 (+2974211us) spinlock_profiling.cc:243] Waited 91.5 ms on lock 0x7b5400131fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:54.559799 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210362112}
I20260709 12:13:54.807840 15245 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:54.806324 11926 pending_rounds.cc:107] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:13:54.603020 15142 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:54.914237 15142 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.917198 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 1992) took 1612 ms. Trace:
I20260709 12:13:54.853518 15281 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.917760 11179 rpcz_store.cc:276] 0709 12:13:53.304868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.307309 (+  2441us) service_pool.cc:224] Handling call
0709 12:13:53.307634 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.307982 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:53.308012 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:53.310093 (+  2081us) log.cc:844] Serialized 26 byte log entry
0709 12:13:53.507395 (+197302us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:53.507447 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.507557 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:54.534634 (+1027077us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:54.916770 (+382136us) raft_consensus.cc:1685] finished
0709 12:13:54.916801 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:54.917118 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3360,"spinlock_wait_cycles":7295360,"thread_start_us":187150,"threads_started":1,"wal-append.queue_time_us":295066}
I20260709 12:13:54.920410 15281 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.890909 15251 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.921295 15251 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.922084 15257 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.611794 15033 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.923813 11329 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:13:54.699458 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 84788.6us
I20260709 12:13:54.925953 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.806749 11932 rpcz_store.cc:276] 0709 12:13:53.180915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.181649 (+   734us) service_pool.cc:224] Handling call
0709 12:13:54.527619 (+1345970us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400667bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:54.527734 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476243840}
I20260709 12:13:54.819530 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:54.928674 11182 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:54.821434 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:54.602121 14982 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:54.823426 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:54.827527 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:54.538964 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1856) took 2546 ms. Trace:
I20260709 12:13:54.822789 11933 rpcz_store.cc:276] 0709 12:13:52.504521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.505426 (+   905us) service_pool.cc:224] Handling call
0709 12:13:52.508453 (+  3027us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:52.508853 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:52.541067 (+ 32214us) log.cc:844] Serialized 14 byte log entry
0709 12:13:52.937266 (+396199us) log.cc:844] Serialized 14 byte log entry
0709 12:13:53.006171 (+ 68905us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:53.006922 (+   751us) log.cc:844] Serialized 52 byte log entry
0709 12:13:53.007095 (+   173us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:53.007143 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.007248 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:54.523450 (+1516202us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b24000310b8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363e09d 00007f3ec627010e 00007f3ec628788c 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221
0709 12:13:54.557574 (+ 34124us) raft_consensus.cc:1685] finished
0709 12:13:54.557615 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:54.558111 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25781,"prepare.queue_time_us":19687,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":176350,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":8991,"spinlock_wait_cycles":48572800,"thread_start_us":394836,"threads_started":3,"wal-append.queue_time_us":28998}
I20260709 12:13:54.810843 15099 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:13:54.611169 15194 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:54.896732 15202 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:54.610756 15070 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:54.871160 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:54.947530 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 76491.4us
I20260709 12:13:54.948455 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:54.950096 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:54.603011 15163 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:54.832726 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:54.811755 15227 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.695634 15068 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:54.695575 15089 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:54.691318 14952 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:54.757653 14992 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:54.892748 11174 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:13:54.943898 15217 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:54.939136 15194 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:54.601955 15264 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.518755 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:54.974900 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 456197us
I20260709 12:13:54.976943 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:54.824712 15253 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:54.878731 15237 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.000s
I20260709 12:13:54.954530 15081 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:54.959568 15068 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.912773 15226 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:54.929922 11325 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:54.689765 15112 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:54.613775 15162 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.972016 15154 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:54.932636 11257 rpcz_store.cc:276] 0709 12:13:52.010423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.153209 (+142786us) service_pool.cc:224] Handling call
0709 12:13:52.153819 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.302659 (+148840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.538708 (+2236049us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001cfcd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:54.538889 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340707712}
W20260709 12:13:54.936784 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 109307us
I20260709 12:13:54.891258 15220 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:54.890628 15006 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.834131 11181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:54.603314 15268 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:54.876296 15304 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:54.939742 15145 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.936412 15281 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.957055 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:54.960165 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 346247us
I20260709 12:13:54.943703 15201 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:54.806605 11178 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:13:54.964588 11934 pending_rounds.cc:85] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 7
I20260709 12:13:54.616302 15293 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.967916 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:54.897646 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.969791 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:54.951179 15291 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:54.877737 14877 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:55.069530 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.981034 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:54.941320 15257 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:13:54.953131 15193 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:54.901275 15189 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.923391 15300 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.966529 15089 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:13:54.829111 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 273292us
I20260709 12:13:54.973695 14992 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:54.905474 11926 raft_consensus.cc:2889] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:55.115352 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:55.163825 11178 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:55.169831 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.101680 11325 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:54.957019 15163 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.165529 11929 raft_consensus.cc:2410] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:13:55.112767 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:55.174916 11176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 1972) took 2692 ms (client timeout 2028 ms). Trace:
W20260709 12:13:55.175940 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63305.9us
W20260709 12:13:55.134392 11252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.025s	sys 0.040s
W20260709 12:13:55.178820 11252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.025s	sys 0.040s
I20260709 12:13:55.118978 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1694) took 1099 ms. Trace:
I20260709 12:13:55.182121 11331 rpcz_store.cc:276] 0709 12:13:54.019204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.019432 (+   228us) service_pool.cc:224] Handling call
0709 12:13:54.027792 (+  8360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.159602 (+131810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.159660 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:54.159717 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:54.159762 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.118741 (+958979us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540042aed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:55.118787 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.118922 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302541184}
I20260709 12:13:55.122103 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:55.185179 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:55.186842 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.187552 11254 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:55.125559 11934 pending_rounds.cc:107] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 10.8
W20260709 12:13:55.126379 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 156808us
I20260709 12:13:55.188447 11934 raft_consensus.cc:2889] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:55.125984 11932 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:55.108556 11931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 715) took 2603 ms (client timeout 1520 ms). Trace:
I20260709 12:13:55.135248 11174 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:55.177372 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:55.143788 11179 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:13:55.175587 11176 rpcz_store.cc:269] 0709 12:13:52.470924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.477106 (+  6182us) service_pool.cc:224] Handling call
0709 12:13:55.127927 (+2650821us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54002e45d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:55.162862 (+ 34935us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351495424}
W20260709 12:13:55.121858 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 145034us
I20260709 12:13:55.170840 11257 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:55.122352 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1855) took 3138 ms. Trace:
I20260709 12:13:55.172154 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:55.207240 11325 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:55.209205 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:55.210048 11325 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:55.095705 15139 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:13:55.192111 11931 rpcz_store.cc:269] 0709 12:13:52.505112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.505502 (+   390us) service_pool.cc:224] Handling call
0709 12:13:55.108322 (+2602820us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b540015e0d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:55.108475 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085235584}
W20260709 12:13:55.126055 15302 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:13:55.092895 15051 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:55.123368 15227 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.203358 11248 rpcz_store.cc:276] 0709 12:13:51.984005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.150319 (+166314us) service_pool.cc:224] Handling call
0709 12:13:52.150616 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.463811 (+313195us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:53.216402 (+752591us) log.cc:844] Serialized 14 byte log entry
0709 12:13:54.694313 (+1477911us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b1800409cc0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:13:54.694601 (+   288us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.006449 (+311848us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.092123 (+ 85674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.092177 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:55.092231 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.122115 (+ 29884us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400068e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:55.122170 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.122293 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2425106,"prepare.queue_time_us":2496729,"prepare.run_cpu_time_us":10045,"prepare.run_wall_time_us":39721,"raft.queue_time_us":86025,"raft.run_cpu_time_us":513,"raft.run_wall_time_us":123394,"spinlock_wait_cycles":753164160,"thread_start_us":304839,"threads_started":2,"wal-append.queue_time_us":77979,"wal-append.run_cpu_time_us":1362,"wal-append.run_wall_time_us":1034402}
I20260709 12:13:55.153520 15154 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:55.109509 15112 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.098306 15070 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.248343 15154 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.252094 15154 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:55.252170 15070 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:55.096252 15190 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:55.254063 15190 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.255119 15257 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.255865 15257 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.118696 15306 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.258466 15306 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.261009 15306 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.262897 15189 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:55.097759 15160 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:55.264529 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2253) took 4047 ms. Trace:
I20260709 12:13:55.264101 15160 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.267985 15160 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:55.090989 15202 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.125934 15145 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.269935 15202 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:55.166985 15072 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.271628 15145 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.119923 15006 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.272825 11181 raft_consensus.cc:606] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:55.273430 11181 raft_consensus.cc:614] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:55.273833 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:55.276134 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:55.247115 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:55.117996 15244 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:13:55.102463 15068 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:55.110862 15162 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:55.172258 15274 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:13:55.264989 11328 rpcz_store.cc:276] 0709 12:13:51.217398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.366451 (+149053us) service_pool.cc:224] Handling call
0709 12:13:51.378542 (+ 12091us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:51.384951 (+  6409us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:51.399001 (+ 14050us) log.cc:844] Serialized 14 byte log entry
0709 12:13:54.169830 (+2770829us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b18001caaa0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:13:54.365171 (+195341us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b18001caaa0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:54.516565 (+151394us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:54.517223 (+   658us) log.cc:844] Serialized 52 byte log entry
0709 12:13:54.517368 (+   145us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:54.602510 (+ 85142us) log.cc:844] Serialized 14 byte log entry
0709 12:13:54.602877 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.603069 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:55.264280 (+661211us) raft_consensus.cc:1685] finished
0709 12:13:55.264306 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.264462 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97465,"prepare.queue_time_us":86945,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":14659,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":22182,"spinlock_wait_cycles":194774912,"thread_start_us":307341,"threads_started":1,"wal-append.queue_time_us":3233576}
I20260709 12:13:55.109241 15264 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.126533 15201 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.279526 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:55.190456 15305 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:55.289448 15315 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:55.090785 15245 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.257395 15190 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.125248 14952 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.235491 15251 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:55.252192 15310 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:13:55.096187 15253 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.098060 15237 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:55.291103 15280 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.095s	user 0.000s	sys 0.007s
W20260709 12:13:55.401022 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.277091 11331 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:55.310326 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.124230 15194 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.293479 14992 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.126554 15281 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.094707 14982 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.332679 15006 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.409507 15006 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:55.409922 15006 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.410022 15317 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:55.410349 15006 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:55.410622 15317 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.414815 15251 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:55.415491 15251 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.298280 15305 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.298758 15033 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.285317 15099 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:13:55.363925 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.281570 15244 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:55.427394 15311 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:13:55.427670 15244 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.428972 15244 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:55.252231 15307 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.447s
W20260709 12:13:55.155979 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1871) took 2308 ms (client timeout 1573 ms). Trace:
I20260709 12:13:55.397315 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1872) took 2549 ms. Trace:
I20260709 12:13:55.399593 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:55.433871 11328 raft_consensus.cc:2410] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:13:55.402437 11248 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:55.296782 15201 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:55.298789 15288 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:55.201555 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:55.433245 11255 rpcz_store.cc:269] 0709 12:13:52.847573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.903047 (+ 55474us) service_pool.cc:224] Handling call
0709 12:13:55.155663 (+2252616us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b54001cedd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:55.155859 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67929472}
I20260709 12:13:55.433339 11250 rpcz_store.cc:276] 0709 12:13:52.847870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.035252 (+1187382us) service_pool.cc:224] Handling call
0709 12:13:54.035712 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:54.036164 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.036203 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:54.036911 (+   708us) log.cc:844] Serialized 52 byte log entry
0709 12:13:54.350476 (+313565us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.396648 (+1046172us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.396912 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.397007 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:55.397055 (+    48us) raft_consensus.cc:1685] finished
0709 12:13:55.397069 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.397252 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":572198,"prepare.queue_time_us":741950,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":67242,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":48392,"thread_start_us":648315,"threads_started":2,"wal-append.queue_time_us":399825}
I20260709 12:13:55.371773 15245 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:55.379982 15298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.023s	sys 0.032s
W20260709 12:13:55.464257 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 262785us
I20260709 12:13:55.456003 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:55.466389 11250 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:55.467036 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1987) took 1068 ms. Trace:
I20260709 12:13:55.467450 11250 rpcz_store.cc:276] 0709 12:13:54.398750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.455490 (+1056740us) service_pool.cc:224] Handling call
0709 12:13:55.466957 (+ 11467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.468699 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.465528 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:55.462795 11325 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:13:55.470417 11253 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.000s	sys 0.016s
W20260709 12:13:55.474025 11253 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.000s	sys 0.016s
I20260709 12:13:55.464684 11257 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:55.479076 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1882) took 1234 ms. Trace:
I20260709 12:13:55.479589 11257 rpcz_store.cc:276] 0709 12:13:54.244249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.169495 (+925246us) service_pool.cc:224] Handling call
0709 12:13:55.478987 (+309492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.480887 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.460382 11254 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:55.484290 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1885) took 1208 ms. Trace:
I20260709 12:13:55.484750 11254 rpcz_store.cc:276] 0709 12:13:54.276381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.186537 (+910156us) service_pool.cc:224] Handling call
0709 12:13:55.484219 (+297682us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.498667 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:55.502285 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.503145 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.488940 11248 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:55.508796 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1986) took 1114 ms. Trace:
I20260709 12:13:55.509347 11248 rpcz_store.cc:276] 0709 12:13:54.394524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.309982 (+915458us) service_pool.cc:224] Handling call
0709 12:13:55.508700 (+198718us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.511061 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.470363 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1985) took 1081 ms. Trace:
I20260709 12:13:55.539113 11250 rpcz_store.cc:276] 0709 12:13:54.387858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.468363 (+1080505us) service_pool.cc:224] Handling call
0709 12:13:55.469570 (+  1207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.472975 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:55.540655 15252 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.887s	user 0.004s	sys 0.001s
W20260709 12:13:55.525187 15284 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.239s	user 0.019s	sys 0.000s
I20260709 12:13:55.429571 15227 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.519045 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.431003 15293 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.454612 15217 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:13:55.465122 15298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.023s	sys 0.032s
I20260709 12:13:55.244763 15268 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.431859 15300 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.517000 15274 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:13:55.512631 15033 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:13:55.532298 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.534320 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1711) took 1048 ms. Trace:
I20260709 12:13:55.482409 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:55.432987 15150 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:13:55.540673 11250 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: [10.5-12.6]
I20260709 12:13:55.539465 11248 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:55.519893 15251 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:55.543718 11255 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:55.544220 11329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.010s	sys 0.031s
W20260709 12:13:55.582835 11329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.010s	sys 0.032s
I20260709 12:13:55.511988 15081 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:55.526773 15257 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.587808 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1881) took 1343 ms. Trace:
I20260709 12:13:55.563241 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1972) took 3135 ms. Trace:
I20260709 12:13:55.588729 11252 rpcz_store.cc:276] 0709 12:13:52.427799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.322629 (+894830us) service_pool.cc:224] Handling call
0709 12:13:53.323078 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.327264 (+  4186us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:53.327310 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:53.327932 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 12:13:54.537143 (+1209211us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b18002824c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:13:55.181603 (+644460us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.181695 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.181811 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:55.541181 (+359370us) raft_consensus.cc:1685] finished
0709 12:13:55.541206 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.563166 (+ 21960us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2144,"raft.queue_time_us":2280,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":27377920,"thread_start_us":644171,"threads_started":1,"wal-append.queue_time_us":2187906}
W20260709 12:13:55.562245 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 384987us
I20260709 12:13:55.568030 15293 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:55.570690 11328 rpcz_store.cc:276] 0709 12:13:54.485586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.518944 (+1033358us) service_pool.cc:224] Handling call
0709 12:13:55.533649 (+ 14705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.533881 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:55.533921 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.533977 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.534022 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.534095 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.534243 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.580518 11257 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:55.594045 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1988) took 1123 ms. Trace:
I20260709 12:13:55.594513 11257 rpcz_store.cc:276] 0709 12:13:54.470224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.482128 (+1011904us) service_pool.cc:224] Handling call
0709 12:13:55.593960 (+111832us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.594982 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:55.588488 11248 rpcz_store.cc:276] 0709 12:13:54.243928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.510408 (+1266480us) service_pool.cc:224] Handling call
0709 12:13:55.587719 (+ 77311us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:55.561864 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.596254 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.529083 15233 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.007s	sys 0.033s
W20260709 12:13:55.584800 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 454355us
I20260709 12:13:55.585179 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1875) took 2665 ms. Trace:
I20260709 12:13:55.600652 11250 rpcz_store.cc:276] 0709 12:13:52.919821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.539947 (+2620126us) service_pool.cc:224] Handling call
0709 12:13:55.540415 (+   468us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:55.584791 (+ 44376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.585104 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.556442 14994 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.596094 11257 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:13:55.602795 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1878) took 2411 ms. Trace:
I20260709 12:13:55.603142 11257 rpcz_store.cc:276] 0709 12:13:53.191550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.595349 (+2403799us) service_pool.cc:224] Handling call
0709 12:13:55.595798 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.602525 (+  6727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.602743 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.600131 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1713) took 1093 ms. Trace:
I20260709 12:13:55.605765 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1888) took 1325 ms. Trace:
I20260709 12:13:55.607019 11257 rpcz_store.cc:276] 0709 12:13:54.279279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.603933 (+1324654us) service_pool.cc:224] Handling call
0709 12:13:55.604296 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.604510 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.604546 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.604599 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.604644 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.604723 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.604845 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.605391 11325 rpcz_store.cc:276] 0709 12:13:54.506888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.599293 (+1092405us) service_pool.cc:224] Handling call
0709 12:13:55.599608 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.599788 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.599815 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.599856 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:55.599890 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.599961 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.600079 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.609797 11257 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:13:55.610688 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1889) took 1331 ms. Trace:
I20260709 12:13:55.611865 11257 rpcz_store.cc:276] 0709 12:13:54.279629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.609028 (+1329399us) service_pool.cc:224] Handling call
0709 12:13:55.609382 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.610356 (+   974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.610608 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.612498 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1879) took 2365 ms. Trace:
W20260709 12:13:55.617767 11220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.000s	sys 0.016s
W20260709 12:13:55.619035 11220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.000s	sys 0.017s
I20260709 12:13:55.619822 11252 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:13:55.620685 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1876) took 2687 ms. Trace:
I20260709 12:13:55.621107 11252 rpcz_store.cc:276] 0709 12:13:52.933797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.592887 (+2659090us) service_pool.cc:224] Handling call
0709 12:13:55.619400 (+ 26513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.620356 (+   956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.620614 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.581727 15150 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:55.600382 15233 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.008s	sys 0.033s
W20260709 12:13:55.639112 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.598711 15293 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.608048 15220 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.637549 15190 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.644032 14994 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:55.636621 11255 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:55.645406 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1883) took 1400 ms. Trace:
I20260709 12:13:55.589987 15307 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:55.646173 11255 rpcz_store.cc:276] 0709 12:13:54.244611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.457072 (+1212461us) service_pool.cc:224] Handling call
0709 12:13:55.645288 (+188216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.561864 15227 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.609354 15298 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.644676 14994 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.656045 15068 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.546798 15284 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.648859 11932 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:55.644151 15190 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.611392 15300 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:55.694010 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.005s	sys 0.054s
W20260709 12:13:55.701270 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.005s	sys 0.054s
I20260709 12:13:55.644483 15316 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.638114 15257 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.715338 15300 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.644286 15006 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.699759 14994 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:55.699419 14992 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:55.716576 15318 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:13:55.758992 15068 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.768483 14992 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.701036 15281 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.694149 15307 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:55.773949 11146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.028s	sys 0.035s
I20260709 12:13:55.699576 15298 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.734218 15316 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:55.788885 15006 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:55.700012 15217 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:55.806320 15329 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:13:55.751339 15150 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:55.811074 15227 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.739606 15257 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:55.808086 15089 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.634s	user 0.012s	sys 0.003s
W20260709 12:13:55.840853 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.256541 11146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.258s	user 0.032s	sys 0.052s
W20260709 12:13:56.123260 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.994813 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.886696 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1686) took 1960 ms. Trace:
I20260709 12:13:55.613003 11248 rpcz_store.cc:276] 0709 12:13:53.247658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.611500 (+2363842us) service_pool.cc:224] Handling call
0709 12:13:55.611870 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.612082 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.612129 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.612184 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:55.612229 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.612298 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.612432 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.904269 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1847) took 3987 ms. Trace:
I20260709 12:13:55.885305 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1702) took 1413 ms. Trace:
W20260709 12:13:56.114885 11934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.017s	sys 0.026s
I20260709 12:13:56.263931 14539 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:56.265409 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:56.266050 11934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.019s	sys 0.027s
W20260709 12:13:56.253773 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 780800us
W20260709 12:13:56.258011 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.266695 11248 raft_consensus.cc:2368] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
I20260709 12:13:55.879328 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 747) took 1404 ms. Trace:
W20260709 12:13:56.267239 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.849746 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1866) took 3411 ms. Trace:
I20260709 12:13:55.875070 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 681) took 1404 ms. Trace:
I20260709 12:13:55.877808 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1698) took 1806 ms. Trace:
W20260709 12:13:55.955946 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.263450 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.268751 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.041446 11077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 168) took 30802 ms (client timeout 15000 ms). Trace:
I20260709 12:13:56.023430 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:56.228652 11253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.010s	sys 0.000s
I20260709 12:13:56.267601 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:55.911996 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 620) took 2345 ms. Trace:
I20260709 12:13:55.887112 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1701) took 1415 ms. Trace:
I20260709 12:13:55.872537 11928 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
W20260709 12:13:55.886502 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.297314 14539 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:13:56.140954 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.211884 11930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.006s	sys 0.025s
I20260709 12:13:56.007560 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:56.253715 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 658771us
W20260709 12:13:56.262055 11331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.013s	sys 0.000s
I20260709 12:13:56.269506 11323 rpcz_store.cc:276] 0709 12:13:54.470355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.471591 (+  1236us) service_pool.cc:224] Handling call
0709 12:13:54.560588 (+ 88997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.811918 (+251330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.874759 (+1062841us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b5400131fd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:55.874967 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124353280}
I20260709 12:13:55.963346 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1674) took 3963 ms. Trace:
I20260709 12:13:56.269152 11322 rpcz_store.cc:276] 0709 12:13:54.071785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.071947 (+   162us) service_pool.cc:224] Handling call
0709 12:13:54.072297 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.208810 (+136513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.877466 (+1668656us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:55.877727 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226211072}
W20260709 12:13:56.263803 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.885967 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1974) took 3456 ms. Trace:
I20260709 12:13:56.015909 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:51810 (request call id 23) took 1476 ms. Trace:
I20260709 12:13:56.329618 14539 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:56.337770 11181 rpcz_store.cc:276] 0709 12:13:54.539390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.833928 (+294538us) service_pool.cc:224] Handling call
0709 12:13:56.015606 (+1181678us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400364a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e6853 00007f3ecb6b460a 00007f3ec4c3a7d4 00007f3ec4c3a762 00007f3ec4c3a691 00007f3ec4c3a60c 00007f3ec4c398c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:56.015782 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357779712}
I20260709 12:13:56.267390 11929 rpcz_store.cc:276] 0709 12:13:54.474961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.523432 (+ 48471us) service_pool.cc:224] Handling call
0709 12:13:55.879064 (+1355632us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b5400690ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:55.879247 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823093632}
I20260709 12:13:56.331350 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.320847 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.331601 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:56.264108 11330 rpcz_store.cc:276] 0709 12:13:53.926079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.926385 (+   306us) service_pool.cc:224] Handling call
0709 12:13:55.886502 (+1960117us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54000e5e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:55.886640 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953416064}
I20260709 12:13:56.331386 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:56.265296 11326 rpcz_store.cc:276] 0709 12:13:54.471763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.938991 (+467228us) service_pool.cc:224] Handling call
0709 12:13:54.939339 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.112058 (+172719us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:55.121714 (+  9656us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.470843 (+349129us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.883750 (+412907us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800438460. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:13:55.883946 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.884003 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:55.884050 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.885086 (+  1036us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002f7bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:55.885132 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.885248 (+   116us) inbound_call.cc:177] Queueing success response
0709 12:13:56.012803 (+127555us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b1800438460. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
Metrics: {"mutex_wait_us":404755,"prepare.queue_time_us":15528,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":2584,"raft.queue_time_us":420030,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":6228,"spinlock_wait_cycles":503007744,"thread_start_us":343510,"threads_started":1,"wal-append.queue_time_us":573459}
W20260709 12:13:56.211676 14877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.012s	sys 0.021s
I20260709 12:13:56.314304 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:56.338644 11930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.008s	sys 0.026s
I20260709 12:13:56.365861 11178 pending_rounds.cc:85] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 7
I20260709 12:13:56.366120 11176 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:56.333747 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:55.777974 15237 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:56.334721 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.299021 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.924234 15258 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 2.276s	user 0.000s	sys 0.004s
I20260709 12:13:56.313823 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:55.790345 15332 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.335770 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69652.2us
I20260709 12:13:55.784984 15253 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:56.336547 11256 rpcz_store.cc:276] 0709 12:13:52.429662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.371034 (+941372us) service_pool.cc:224] Handling call
0709 12:13:55.155866 (+1784832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.156164 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.156204 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.156267 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.156314 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.156378 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.885800 (+729422us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54001cedd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:55.885906 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4103865088}
I20260709 12:13:56.312565 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.027240 11078 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:13:56.337253 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 313886us
I20260709 12:13:56.366317 11178 pending_rounds.cc:107] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:13:56.389669 11178 raft_consensus.cc:2889] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:56.382441 14539 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.335134 15150 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:56.331230 15068 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.373900 14877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.014s	sys 0.023s
I20260709 12:13:56.298457 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 748) took 1823 ms. Trace:
I20260709 12:13:56.023195 15293 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:56.336910 11255 raft_consensus.cc:2368] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 7. Current term is 8.
I20260709 12:13:55.826504 15333 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.922072 14992 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:56.268712 11251 rpcz_store.cc:276] 0709 12:13:52.438309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.974548 (+1536239us) service_pool.cc:224] Handling call
0709 12:13:54.022293 (+ 47745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.022540 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:55.299511 (+1276971us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.784662 (+485151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.784743 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:55.849321 (+ 64578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.849447 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.849660 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":899178,"prepare.queue_time_us":983474,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":7759,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":151,"thread_start_us":859830,"threads_started":2}
I20260709 12:13:55.990482 15226 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:56.333774 11926 rpcz_store.cc:276] 0709 12:13:53.566287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.566437 (+   150us) service_pool.cc:224] Handling call
0709 12:13:53.600701 (+ 34264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.165022 (+1564321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.911593 (+746571us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54004312d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:55.911850 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030916864}
I20260709 12:13:56.333132 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:56.418171 11322 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:56.374083 14982 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:13:56.374147 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.419260 11174 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:56.273165 15315 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.263238 15190 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:56.278168 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:56.374820 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.299253 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:56.373862 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:55.888813 14952 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:56.253795 15194 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:56.377772 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.327844 11252 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
W20260709 12:13:56.378995 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.379762 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.336373 11329 rpcz_store.cc:276] 0709 12:13:54.471511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.925012 (+453501us) service_pool.cc:224] Handling call
0709 12:13:54.925334 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.925576 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:54.942914 (+ 17338us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.886753 (+943839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.886806 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:55.886853 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.886926 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.887055 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":303439,"prepare.queue_time_us":20685,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":305315,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":46,"thread_start_us":640002,"threads_started":1,"wal-append.queue_time_us":898873}
I20260709 12:13:56.380198 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:56.381259 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:56.134960 15180 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:55.886090 15248 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:13:56.364428 11253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.012s	sys 0.002s
I20260709 12:13:55.886442 15300 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:55.818449 15234 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:56.267804 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:56.382203 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.405794 15068 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:55.989199 15201 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.005s
W20260709 12:13:56.161487 15145 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.004s	sys 0.008s
W20260709 12:13:55.939067 15154 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.686s	user 0.001s	sys 0.016s
W20260709 12:13:55.989604 15309 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.862s	user 0.004s	sys 0.002s
W20260709 12:13:56.080996 15245 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.000s
I20260709 12:13:56.389470 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:55.894959 15163 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:56.336423 14829 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:13:56.368080 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.508119 11177 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:56.456314 15248 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:13:56.334106 11331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.013s	sys 0.001s
I20260709 12:13:55.899058 15263 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.362967 15281 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:56.393326 11078 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:56.368652 11927 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:13:56.375226 15329 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:13:56.065091 15070 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.812s	user 0.014s	sys 0.007s
I20260709 12:13:56.338701 11327 rpcz_store.cc:276] 0709 12:13:51.999814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.014226 (+ 14412us) service_pool.cc:224] Handling call
0709 12:13:52.014638 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.593968 (+1579330us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:53.594292 (+   324us) log.cc:844] Serialized 14 byte log entry
0709 12:13:53.936318 (+342026us) log.cc:844] Serialized 14 byte log entry
0709 12:13:54.115480 (+179162us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b18000ebfc0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:54.358159 (+242679us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18000ebfc0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:13:54.358320 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:54.358370 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:54.358413 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.963021 (+1604608us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000e5954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:55.963086 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.963270 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":477541,"prepare.queue_time_us":12012,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":97,"raft.queue_time_us":437455,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":21423,"spinlock_wait_cycles":3643659776,"thread_start_us":339544,"threads_started":1,"wal-append.queue_time_us":639196,"wal-append.run_cpu_time_us":1416,"wal-append.run_wall_time_us":1001768}
I20260709 12:13:56.264973 11249 rpcz_store.cc:276] 0709 12:13:51.917356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.917500 (+   144us) service_pool.cc:224] Handling call
0709 12:13:51.917893 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.154086 (+2236193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.154134 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.154719 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:13:54.355584 (+200865us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:55.300105 (+944521us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.328932 (+ 28827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.903701 (+574769us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b54001cf7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:55.903779 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:55.903853 (+    74us) raft_consensus.cc:1685] finished
0709 12:13:55.903976 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.904185 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":617849,"prepare.queue_time_us":747350,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":43,"raft.queue_time_us":63940,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":1177,"spinlock_wait_cycles":5077613440,"thread_start_us":520176,"threads_started":2,"wal-append.queue_time_us":302080,"wal-append.run_cpu_time_us":3065,"wal-append.run_wall_time_us":1816113}
I20260709 12:13:56.398890 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.530912 11175 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:56.400874 11928 rpcz_store.cc:276] 0709 12:13:54.475280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.642566 (+167286us) service_pool.cc:224] Handling call
0709 12:13:55.871985 (+1229419us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b58001e0c98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:13:55.872274 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.298139 (+425865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.298377 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113684224}
I20260709 12:13:56.402537 11933 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:13:56.363977 15244 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.405032 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:55.958793 15220 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:55.886602 15033 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:56.413538 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:56.540131 11180 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:13:56.540522 11932 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:56.263418 15089 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.392398 15317 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:56.272122 15143 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.121s
I20260709 12:13:56.423121 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:56.427932 15315 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.427559 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:56.426348 11931 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 6.
I20260709 12:13:56.413865 15227 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:13:56.421492 15326 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.456864 11935 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:56.428857 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 129667us
I20260709 12:13:56.446202 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:56.445530 11179 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 12:13:56.447197 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 66005.3us
I20260709 12:13:56.451957 11176 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:13:56.072568 15081 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.467454 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:56.577324 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:56.580302 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:56.580706 11252 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:56.422012 15226 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:56.475620 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 749) took 2000 ms. Trace:
I20260709 12:13:56.391304 15332 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.477824 11248 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:56.478892 11181 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:13:56.045111 15202 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.774s	user 0.015s	sys 0.002s
I20260709 12:13:56.455900 14982 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:56.486997 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:56.488193 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:56.490828 14539 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:56.443264 14824 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:13:56.428388 15284 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:56.442866 15257 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:56.506932 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:56.513810 15263 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:56.336580 11077 rpcz_store.cc:269] 0709 12:13:25.238679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.255957 (+ 17278us) service_pool.cc:224] Handling call
0709 12:13:56.041141 (+30785184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:42.722911 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:42.723623 (+   712us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:42.723682 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:13:42.723709 (+    27us) tablet.cc:662] Decoding operations
0709 12:13:42.784696 (+ 60987us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:42.789750 (+  5054us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:42.789786 (+    36us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:13:42.790391 (+   605us) tablet.cc:701] Row locks acquired
0709 12:13:42.790414 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:13:42.790576 (+   162us) write_op.cc:270] Start()
0709 12:13:42.790696 (+   120us) write_op.cc:276] Timestamp: P: 1783599222790528 usec, L: 0
0709 12:13:42.790719 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:13:42.791356 (+   637us) log.cc:844] Serialized 2069 byte log entry
0709 12:13:43.419777 (+628421us) op_driver.cc:464] REPLICATION: finished
0709 12:13:43.515358 (+ 95581us) write_op.cc:301] APPLY: starting
0709 12:13:43.515450 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:43.515608 (+   158us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:43.515733 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:43.522201 (+  6468us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:43.522399 (+   198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:13:43.522446 (+    47us) write_op.cc:312] APPLY: finished
0709 12:13:43.522712 (+   266us) log.cc:844] Serialized 47 byte log entry
0709 12:13:43.522895 (+   183us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:43.523049 (+   154us) write_op.cc:454] Released schema lock
0709 12:13:43.523090 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:43.529974 (+  6884us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:43.530004 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2181659,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":73417,"apply.run_cpu_time_us":2656,"apply.run_wall_time_us":14603,"num_ops":5,"prepare.queue_time_us":567754,"prepare.run_cpu_time_us":31994,"prepare.run_wall_time_us":204607,"raft.queue_time_us":252946,"raft.run_cpu_time_us":18165,"raft.run_wall_time_us":53290,"replication_time_us":628901,"thread_start_us":639718,"threads_started":4,"wal-append.queue_time_us":219990,"wal-append.run_cpu_time_us":12279,"wal-append.run_wall_time_us":1512559}]]}
I20260709 12:13:56.514863 15281 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.481989 15331 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:56.508445 15154 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.519485 15288 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.520280 15139 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:56.522679 15305 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:56.454365 15328 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 12:13:56.501919 15145 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.223s	user 0.007s	sys 0.008s
I20260709 12:13:56.541054 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:56.540527 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:56.543964 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:56.544559 15155 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.539706 14829 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:56.468899 15180 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.504886 15201 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.512563 15248 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:56.547626 15316 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:56.535342 11078 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 5 to 6, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 6 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:56.552505 15307 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:56.421124 15333 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.423764 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:56.530124 15296 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.566236 11250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.033s	sys 0.063s
I20260709 12:13:56.582321 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:56.495277 15234 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.581570 11930 rpcz_store.cc:276] 0709 12:13:54.475605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.883474 (+407869us) service_pool.cc:224] Handling call
0709 12:13:54.923807 (+ 40333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.235552 (+311745us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:13:55.614058 (+378506us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.366256 (+752198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.366308 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:56.366358 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.475291 (+108933us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540041b754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:56.475352 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.475541 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":510779,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":1062,"raft.queue_time_us":10743,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":10662,"spinlock_wait_cycles":716285696,"thread_start_us":1129960,"threads_started":2,"wal-append.queue_time_us":893966}
I20260709 12:13:56.582336 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:56.650563 11326 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: [11.5-11.5]
W20260709 12:13:56.576040 11325 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.022s	sys 0.042s
I20260709 12:13:56.653234 11251 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:13:56.565892 15298 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:56.565879 15190 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:13:56.574427 15268 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.036s	sys 0.049s
I20260709 12:13:56.870854 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2286) took 1710 ms. Trace:
W20260709 12:13:56.820570 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.871941 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.871680 11326 rpcz_store.cc:276] 0709 12:13:55.160115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.649785 (+1489670us) service_pool.cc:224] Handling call
0709 12:13:56.650215 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.870498 (+220283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.870775 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:56.672520 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.721813 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 746) took 2247 ms. Trace:
W20260709 12:13:56.810196 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.873239 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 290973us
I20260709 12:13:56.834746 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:56.874118 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:13:56.865082 11250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.147s	user 0.041s	sys 0.067s
W20260709 12:13:56.740151 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.708106 11330 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:13:56.865980 11249 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:56.875116 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:56.866786 11256 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:56.723536 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1709) took 2238 ms. Trace:
W20260709 12:13:56.868242 11325 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.025s	sys 0.044s
I20260709 12:13:56.866293 11078 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 10. New cstate: current_term: 10 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:56.735628 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1710) took 2250 ms. Trace:
I20260709 12:13:56.553925 15244 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:56.719357 15344 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:13:56.796257 11322 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
W20260709 12:13:56.780913 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.869622 11255 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:56.576429 14994 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.793s	user 0.014s	sys 0.000s
W20260709 12:13:56.868652 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 286358us
I20260709 12:13:56.899693 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:56.885519 11331 rpcz_store.cc:276] 0709 12:13:54.484993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.427347 (+942354us) service_pool.cc:224] Handling call
0709 12:13:55.427796 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.432063 (+  4267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.432094 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:55.432601 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:13:56.509530 (+1076929us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.509570 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.509686 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.723292 (+213606us) raft_consensus.cc:1685] finished
0709 12:13:56.723314 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.723467 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":1076484,"threads_started":1,"wal-append.queue_time_us":1288755}
W20260709 12:13:56.877987 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.902808 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.875813 11934 rpcz_store.cc:276] 0709 12:13:54.474660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.689965 (+215305us) service_pool.cc:224] Handling call
0709 12:13:54.825330 (+135365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:55.282576 (+457246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.282608 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:55.283234 (+   626us) log.cc:844] Serialized 52 byte log entry
0709 12:13:56.266697 (+983463us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:56.266740 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.370747 (+104007us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54003b5454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:56.370816 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.721323 (+350507us) raft_consensus.cc:1685] finished
0709 12:13:56.721359 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.721724 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67039,"raft.queue_time_us":308643,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":14689,"spinlock_wait_cycles":319423360,"thread_start_us":952482,"threads_started":1,"wal-append.queue_time_us":1319611}
I20260709 12:13:56.901294 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:56.904201 11254 raft_consensus.cc:2368] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:13:56.886276 11324 rpcz_store.cc:276] 0709 12:13:54.485331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.428837 (+943506us) service_pool.cc:224] Handling call
0709 12:13:55.429193 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:55.429570 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.429600 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:55.430761 (+  1161us) log.cc:844] Serialized 52 byte log entry
0709 12:13:55.709253 (+278492us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:55.709322 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.709441 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.735383 (+1025942us) raft_consensus.cc:1685] finished
0709 12:13:56.735403 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.735563 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1931,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":92,"thread_start_us":277899,"threads_started":1,"wal-append.queue_time_us":1274510}
W20260709 12:13:56.829602 15073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.138s	user 0.032s	sys 0.054s
W20260709 12:13:56.718287 15341 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 12:13:56.908766 15073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.035s	sys 0.057s
W20260709 12:13:56.807950 15233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.021s	sys 0.027s
I20260709 12:13:56.905020 11251 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:13:56.585083 15293 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.320s
I20260709 12:13:56.909473 11182 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:13:56.583292 15338 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:56.904196 11078 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 7 to 8, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 8 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:56.796658 14824 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.912781 11257 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: []
W20260709 12:13:56.914306 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.727274 15315 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:56.831436 15237 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.871476 15268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.037s	sys 0.050s
I20260709 12:13:56.876116 14998 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:13:56.919095 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 1984) took 2531 ms (client timeout 2368 ms). Trace:
W20260709 12:13:56.928507 11254 rpcz_store.cc:269] 0709 12:13:54.387603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.498253 (+1110650us) service_pool.cc:224] Handling call
0709 12:13:56.918893 (+1420640us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400109854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:56.919033 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3228901248}
I20260709 12:13:56.882318 15068 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:56.935550 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 1983) took 2553 ms. Trace:
I20260709 12:13:56.935942 11257 rpcz_store.cc:276] 0709 12:13:54.382235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.612656 (+1230421us) service_pool.cc:224] Handling call
0709 12:13:55.612998 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.923458 (+1310460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.935326 (+ 11868us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54000870d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:56.935494 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2988803456}
I20260709 12:13:56.924997 11329 raft_consensus.cc:1217] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-10.9]   Dedup: 6.7->[8.8-10.9]
I20260709 12:13:56.937206 11329 pending_rounds.cc:85] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:13:56.937454 11329 pending_rounds.cc:107] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:13:56.937695 11329 raft_consensus.cc:2889] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:56.882688 15202 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:56.904165 15305 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:13:56.916790 15233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.022s	sys 0.034s
I20260709 12:13:56.585990 15326 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.903680 15163 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:56.906232 15335 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:56.908129 15257 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:56.877985 15139 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.877813 15155 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.934257 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2303) took 1966 ms. Trace:
W20260709 12:13:56.941728 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.031s	sys 0.051s
I20260709 12:13:56.882478 15150 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:13:56.941403 11255 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:13:56.945434 14539 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.951447 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.031s	sys 0.051s
I20260709 12:13:56.954320 14539 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:56.954418 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 573043us
I20260709 12:13:56.956910 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:56.951292 11182 rpcz_store.cc:276] 0709 12:13:54.968301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.969862 (+  1561us) service_pool.cc:224] Handling call
0709 12:13:54.970379 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.922700 (+1952321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.933949 (+ 11249us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54000d9154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:56.934187 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4458746752}
I20260709 12:13:56.878183 15190 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.882284 15281 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.907588 15307 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:13:56.968185 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:56.913139 14994 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.731375 15226 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.883539 15296 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.884132 15288 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.970912 11326 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:56.971015 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 505) took 1037 ms. Trace:
I20260709 12:13:56.974758 14539 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:56.981657 11255 rpcz_store.cc:276] 0709 12:13:55.916758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.407947 (+491189us) service_pool.cc:224] Handling call
0709 12:13:56.954040 (+546093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.969199 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:56.976322 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:56.944659 15335 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:56.977350 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:56.942262 15143 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:13:56.732503 15033 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.986181 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:56.996250 11931 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:13:56.939635 14824 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:56.875969 15331 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:56.993193 11252 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:13:56.995381 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.003756 11327 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:57.004451 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:56.944916 15163 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:56.980334 15316 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:13:56.998713 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:56.977759 15257 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:56.958415 15068 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:13:57.008109 11934 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:57.008257 11930 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:13:57.000319 15336 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.440s	user 0.001s	sys 0.000s
I20260709 12:13:57.009128 11323 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:56.998713 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:56.975575 15315 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:13:57.013502 15263 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:57.013648 15333 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:57.033197 14824 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:56.991034 15150 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:57.014493 15154 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.039091 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:56.973913 14829 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.014707 15284 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:57.016587 15281 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.017400 11327 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:13:57.008767 15226 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:57.009583 15163 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:57.018102 15190 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.006824 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:57.035938 15263 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.024932 11932 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:13:57.025028 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.001s	sys 0.017s
I20260709 12:13:56.993958 15335 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:57.026301 11220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.008s	sys 0.006s
W20260709 12:13:57.075574 11220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.008s	sys 0.006s
I20260709 12:13:56.938403 15237 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:57.024948 11221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.002s	sys 0.011s
I20260709 12:13:57.005929 15089 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.047219 11327 pending_rounds.cc:85] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:13:57.087646 11327 pending_rounds.cc:107] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:13:57.087996 11327 raft_consensus.cc:2889] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:57.043838 15332 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:57.093602 15351 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 12:13:57.047859 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.004s	sys 0.011s
W20260709 12:13:57.045526 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.032s	sys 0.066s
W20260709 12:13:57.101527 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.032s	sys 0.066s
W20260709 12:13:57.101897 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 712069us
W20260709 12:13:57.101701 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.004s	sys 0.012s
I20260709 12:13:57.103504 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:57.070084 15201 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:56.969066 11078 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 7 to 8, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 8 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:57.071003 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.002s	sys 0.017s
W20260709 12:13:57.124010 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 740095us
I20260709 12:13:57.128610 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:57.129374 11322 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:13:57.079265 11221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.002s	sys 0.011s
I20260709 12:13:57.131595 15356 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:57.132021 11148 connection.cc:256] RPC call timeout handler was delayed by -1.1332246990000385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2918s for a response
I20260709 12:13:57.132105 15356 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.132680 15089 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.134521 15356 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:57.090998 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 84262.3us
W20260709 12:13:57.064201 15217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.024s	sys 0.044s
W20260709 12:13:57.135433 15217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.024s	sys 0.044s
I20260709 12:13:57.135780 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:57.138969 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:57.139705 11176 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:57.142401 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.133168 15089 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.151768 15089 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.162019 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.162876 11174 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 9: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:57.165238 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.165958 11180 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:57.167259 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.168107 11175 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:57.174590 11175 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:13:57.177338 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:57.103869 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 724772us
I20260709 12:13:57.107010 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:57.105014 15284 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.179636 15284 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.180042 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:57.093964 15281 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:57.181865 15281 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:57.183220 11176 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:57.183127 15281 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.184893 15281 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:57.093649 15139 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.113783 15180 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.186225 15139 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:57.187125 15139 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:57.187777 11322 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:57.087070 14824 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:57.090771 15154 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:13:57.095853 15234 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.193288 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:57.115790 15288 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.195405 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.195804 11331 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:13:57.196260 11331 pending_rounds.cc:85] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 5
I20260709 12:13:57.196130 11330 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:13:57.196534 11331 pending_rounds.cc:107] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:13:57.196826 11331 raft_consensus.cc:2889] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:57.072794 15357 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 12:13:57.201990 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.202296 15326 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.202747 11323 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:57.203022 15154 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:57.203442 15154 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:57.203835 15154 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:57.204165 15305 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.204591 15305 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:57.091300 15244 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.116699 15354 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:57.206171 15354 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.207911 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 502) took 1292 ms. Trace:
I20260709 12:13:57.208515 15354 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.208318 11252 rpcz_store.cc:276] 0709 12:13:55.915896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.660840 (+744944us) service_pool.cc:224] Handling call
0709 12:13:56.718654 (+ 57814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:57.003380 (+284726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.207664 (+204284us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b5400277254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:57.207846 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630325120}
I20260709 12:13:57.118454 15307 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.211930 15307 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.077979 15263 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.214227 15307 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.099228 15190 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:57.197542 11330 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:13:57.214862 15190 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:57.215693 15190 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:57.217139 15349 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:13:57.119784 15331 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.218510 15316 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:57.219837 15316 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.120530 14994 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:57.061935 11253 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.011s	sys 0.006s
I20260709 12:13:57.187943 15139 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:57.115494 11931 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
W20260709 12:13:57.226487 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 187464us
I20260709 12:13:57.226982 11078 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 6 to 7, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:57.213863 14824 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.209209 15354 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:57.127924 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:57.122934 11178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.006s	sys 0.017s
I20260709 12:13:57.088126 15257 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.229233 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:57.047254 11248 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:13:57.224018 11253 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.012s	sys 0.007s
I20260709 12:13:57.204919 14982 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:57.237422 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.223483 15073 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:57.240340 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:57.282328 11178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.007s	sys 0.021s
I20260709 12:13:57.283021 11323 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
W20260709 12:13:57.284044 11248 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 500) took 1368 ms (client timeout 1730 ms). Trace:
W20260709 12:13:57.234599 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.247185 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.287637 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:57.288584 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1884) took 3034 ms. Trace:
I20260709 12:13:57.232638 14994 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.284729 15073 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:57.295441 15220 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:57.304126 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.309983 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:57.281667 15180 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.241204 15358 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:57.325712 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.326841 11324 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:13:57.328234 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 443807us
I20260709 12:13:57.329282 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:57.332455 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.332593 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.334062 11324 raft_consensus.cc:2410] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:13:57.316332 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1858) took 5173 ms. Trace:
I20260709 12:13:57.344122 11253 rpcz_store.cc:276] 0709 12:13:52.142925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.249795 (+1106870us) service_pool.cc:224] Handling call
0709 12:13:53.250259 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.250827 (+   568us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:53.300769 (+ 49942us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.474585 (+2173816us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.284250 (+1809665us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:57.292965 (+  8715us) log.cc:844] Serialized 26 byte log entry
0709 12:13:57.293144 (+   179us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:57.293186 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.293317 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.316091 (+ 22774us) raft_consensus.cc:1685] finished
0709 12:13:57.316112 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.316266 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":915865,"prepare.queue_time_us":974029,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":306777,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":96957,"thread_start_us":3030247,"threads_started":5,"wal-append.queue_time_us":1873902,"wal-append.run_cpu_time_us":1282,"wal-append.run_wall_time_us":1001374}
I20260709 12:13:57.317982 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:57.285617 11248 rpcz_store.cc:269] 0709 12:13:55.915314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.267505 (+352191us) service_pool.cc:224] Handling call
0709 12:13:57.283703 (+1016198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:57.337785 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:13:57.337621 11078 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 7 to 9, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 9 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:57.339475 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.353348 11323 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:57.340293 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:57.354478 11331 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:57.341871 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:57.342262 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.298097 15132 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:57.300712 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:57.352216 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:57.358031 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:57.282137 15248 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.327481 15202 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:57.374634 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 455232us
I20260709 12:13:57.292071 15234 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 12:13:57.380434 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.382165 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:13:57.389797 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.330689 15284 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.379861 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:57.327982 15358 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.395295 11248 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:13:57.396592 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:57.397275 11252 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:57.348757 15220 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.406414 11254 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:13:57.407660 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:57.408238 11254 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:57.398378 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:57.408548 11078 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:57.409271 11257 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:13:57.403324 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 296413us
I20260709 12:13:57.402132 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.370405 14982 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:57.403585 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:57.410732 11323 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:57.411165 11248 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:57.411700 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.412340 11323 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:57.404227 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:57.414027 11255 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:57.371280 15244 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:57.405653 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:13:57.417184 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.376538 15201 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:13:57.381670 15089 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:57.410351 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:57.436398 11248 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:57.438539 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:57.439596 11331 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:57.442863 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:57.421293 11257 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:57.422489 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:57.381215 15354 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:57.451160 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 271186us
I20260709 12:13:57.452001 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:57.452255 11254 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:57.453241 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:13:57.418700 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 241443us
I20260709 12:13:57.313851 14994 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.431106 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:57.459208 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 87084.2us
I20260709 12:13:57.445291 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:57.448714 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:57.460613 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:57.453298 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:57.455022 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:57.384919 15070 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:13:57.455950 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:57.403966 15365 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.405092 15081 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:57.336825 15180 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.461555 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:57.462093 11933 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:57.382112 15331 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:13:57.399663 15358 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:57.469167 11927 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:57.463385 15369 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:57.402838 14998 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:13:57.433596 15089 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.499109 15089 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:57.501888 15385 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:13:57.502146 15363 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:13:57.502560 15385 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:57.503468 15385 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.448479 15288 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:13:57.449719 15362 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:57.455113 14829 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:57.454895 14982 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:57.505945 15362 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.506757 14982 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.467945 15333 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:57.507896 15333 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.508404 15333 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:57.460224 15332 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:57.486114 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.425125 15201 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.489485 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:57.519251 11927 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:13:57.393085 15073 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.466101 15355 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:57.488790 15081 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:57.496838 15263 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.498010 14824 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.496464 15369 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.418598 15220 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.505949 14829 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:57.498340 15390 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.463850 15354 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.475031 15305 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.508502 15362 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.476603 15365 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.524299 15201 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:57.527685 15355 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.525254 15332 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:57.544015 15360 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 12:13:57.544843 15373 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:13:57.488111 15202 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.533311 15369 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:57.542179 15355 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:57.565076 15305 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.538723 15390 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.571241 15248 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.574599 15332 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.573370 15335 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.581936 15365 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:57.589227 15335 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.586460 15390 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:57.594844 15354 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.602213 15369 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.604420 15369 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.632742 11078 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 5 to 8, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 8 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:57.641374 11078 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:13:57.651242 11078 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:13:57.425088 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0532113670001308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 12:13:57.423559 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 43822.8us
W20260709 12:13:57.656029 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.749172 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.658001 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 215161us
I20260709 12:13:57.750705 11933 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:13:57.752233 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:57.658213 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 202648us
I20260709 12:13:57.754721 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:57.758015 14829 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:57.763108 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:57.658313 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 589) took 1338 ms. Trace:
I20260709 12:13:57.765134 11178 rpcz_store.cc:276] 0709 12:13:56.319822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.320013 (+   191us) service_pool.cc:224] Handling call
0709 12:13:56.365603 (+ 45590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.390266 (+ 24663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.390296 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:56.390752 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:13:57.283296 (+892544us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:57.283333 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.283398 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.657621 (+374223us) raft_consensus.cc:1685] finished
0709 12:13:57.657692 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.658104 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":865945,"threads_started":1,"wal-append.queue_time_us":905578}
I20260709 12:13:57.767797 15155 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.660136 11248 raft_consensus.cc:2410] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:13:57.473901 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:57.491883 11327 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:57.790201 11254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:55516
W20260709 12:13:57.790853 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 316914us
I20260709 12:13:57.791374 11254 raft_consensus.cc:606] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:13:57.791894 11254 raft_consensus.cc:614] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:57.792200 15237 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:57.792258 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:57.497226 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:57.495261 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:57.795977 15398 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:57.796607 15398 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.797061 15398 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:57.586943 15226 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.002s
I20260709 12:13:57.800321 15253 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:57.801105 15253 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:57.805969 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54066.1us
I20260709 12:13:57.705732 11927 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:13:57.810415 15355 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.712882 15081 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:57.714775 15399 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:13:57.812255 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49606.8us
I20260709 12:13:57.716135 11324 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:13:57.814122 15369 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:57.814867 15369 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:57.715939 15354 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:57.717008 15073 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:57.816468 15369 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.711683 15244 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:57.711902 11078 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 7 to 9, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 9 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:57.711934 14994 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.738073 15220 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:57.738651 11328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2289) took 1776 ms (client timeout 1571 ms). Trace:
W20260709 12:13:57.768671 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.767791 15376 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:57.549018 15163 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.658836 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.761089 15132 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:13:57.785863 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.826157 15163 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:57.787187 15180 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.810901 15298 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.832470 15298 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.834522 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:57.835372 11324 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:57.837486 15253 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.720635 15335 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:57.805168 15268 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:57.842697 15268 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.813488 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:57.830511 11323 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:57.838150 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2284) took 2678 ms. Trace:
I20260709 12:13:57.810791 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:57.820765 11328 rpcz_store.cc:269] 0709 12:13:55.961833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.000036 (+ 38203us) service_pool.cc:224] Handling call
0709 12:13:57.737914 (+1737878us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540042a9d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:13:57.738230 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454655232}
I20260709 12:13:57.849891 15296 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.840080 11078 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:13:57.827399 15376 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:57.853679 15376 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.845557 15268 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:57.862568 15244 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.870172 15358 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:57.872112 11935 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:13:57.872351 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.873468 11322 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:13:57.880203 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.880584 15155 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:57.876196 15358 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.882135 11182 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:57.886071 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.289040 11250 rpcz_store.cc:276] 0709 12:13:54.254478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.601387 (+1346909us) service_pool.cc:224] Handling call
0709 12:13:55.601889 (+   502us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:55.611214 (+  9325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.611247 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:13:55.645464 (+ 34217us) log.cc:844] Serialized 78 byte log entry
0709 12:13:56.877732 (+1232268us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:57.288053 (+410321us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.288296 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.288363 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.288399 (+    36us) raft_consensus.cc:1685] finished
0709 12:13:57.288409 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.288521 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14996,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":26990,"thread_start_us":1540182,"threads_started":2,"wal-append.queue_time_us":1310250}
I20260709 12:13:57.888402 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:57.888797 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 75554.6us
I20260709 12:13:57.888943 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.889350 11181 raft_consensus.cc:2368] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11.
I20260709 12:13:57.890748 11174 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:13:57.848027 11329 rpcz_store.cc:276] 0709 12:13:55.159406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.578521 (+1419115us) service_pool.cc:224] Handling call
0709 12:13:56.578959 (+   438us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:56.943000 (+364041us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:56.968450 (+ 25450us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.306946 (+338496us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:57.319077 (+ 12131us) log.cc:844] Serialized 52 byte log entry
0709 12:13:57.319296 (+   219us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:57.404447 (+ 85151us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.404668 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.808651 (+403983us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b5400428bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:13:57.808734 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.821794 (+ 13060us) raft_consensus.cc:1685] finished
0709 12:13:57.821814 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.837956 (+ 16142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49991,"prepare.run_cpu_time_us":161,"prepare.run_wall_time_us":9659,"raft.queue_time_us":189097,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":20250,"spinlock_wait_cycles":795279104,"thread_start_us":393804,"threads_started":2,"wal-append.queue_time_us":336602}
I20260709 12:13:57.886994 11178 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:57.909066 11329 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [7.7-9.8]
W20260709 12:13:57.910825 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.912864 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.916373 15245 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:57.918057 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.918416 15298 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:57.919335 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 173896us
I20260709 12:13:57.920687 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:57.921970 15068 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:57.922375 15068 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.922713 15068 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:57.926141 15257 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:13:57.926702 15257 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:57.927124 15257 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:57.931228 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:57.929270 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49060.8us
I20260709 12:13:57.936117 15194 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:13:57.932924 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 86457.1us
I20260709 12:13:57.944399 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:57.948597 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:57.966439 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:57.962095 15298 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:13:57.968801 15298 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:57.969332 15298 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:57.971968 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:57.965143 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:57.981765 15233 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:13:57.982402 15233 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:57.983785 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:13:57.984162 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.977320 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:57.992686 15154 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.727s	user 0.004s	sys 0.027s
I20260709 12:13:57.993348 15154 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:57.996809 15154 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:57.997888 15194 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:57.998472 15194 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.998898 15194 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:57.999670 15089 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.009328 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:58.010602 11178 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 12:13:57.952152 15180 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:13:58.018042 11253 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 7.
I20260709 12:13:58.018827 11323 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:58.020968 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.041508 11324 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:13:58.022627 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.018654 11078 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 9, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 9 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:13:58.048364 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.059624 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.060463 11327 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:13:58.111678 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 150077us
W20260709 12:13:58.066990 15201 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.515s	user 0.005s	sys 0.010s
I20260709 12:13:58.120903 15253 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:58.121338 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.121798 11296 leader_election.cc:400] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:13:58.122177 11296 leader_election.cc:403] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:58.124446 15248 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:13:58.121735 15253 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:58.126830 15253 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:58.121622 11322 raft_consensus.cc:2410] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:13:58.145821 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:58.150496 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 176611us
I20260709 12:13:58.150647 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:58.170279 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.171232 15154 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:58.171674 15154 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.172026 15154 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:58.174165 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:58.174989 11934 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:13:58.174816 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:58.176211 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:58.179126 15190 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.963s	user 0.009s	sys 0.010s
I20260709 12:13:58.151724 15281 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:13:58.152257 15070 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:13:58.118044 15355 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:58.180065 15070 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.144194 15201 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.182426 11179 raft_consensus.cc:2368] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:13:58.182540 15070 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:58.153656 14994 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:58.158742 15089 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:58.035180 15298 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.183378 14994 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:58.183887 14994 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:58.184319 15298 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:13:58.058881 15398 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:58.151401 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 229750us
I20260709 12:13:58.193953 15362 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.194033 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:58.194998 11929 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:58.194792 15362 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.195885 15245 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:58.196841 15245 consensus_queue.cc:1243] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 8.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:13:58.197345 15163 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:58.197746 15281 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:58.197907 15163 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:58.199110 15281 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.199246 15163 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.200516 15163 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:58.204097 11323 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:58.190824 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2283) took 2804 ms. Trace:
I20260709 12:13:58.179811 15190 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.207294 15398 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:58.208127 11329 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
W20260709 12:13:58.210207 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.223598 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:58.197388 15362 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:58.203231 15201 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:58.226491 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:58.226629 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:58.232448 11178 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:58.229276 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.234567 11179 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:13:58.235100 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:58.237233 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.189981 14829 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.204941 11325 rpcz_store.cc:276] 0709 12:13:55.159071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.930288 (+771217us) service_pool.cc:224] Handling call
0709 12:13:55.930835 (+   547us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:55.931408 (+   573us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:56.885397 (+953989us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.371215 (+485818us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.449129 (+ 77914us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:57.449993 (+   864us) log.cc:844] Serialized 52 byte log entry
0709 12:13:57.450152 (+   159us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:57.465486 (+ 15334us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.755856 (+290370us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800374ec0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:57.807997 (+ 52141us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b1800374ec0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:13:57.881969 (+ 73972us) spinlock_profiling.cc:243] Waited 55.4 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:13:57.904517 (+ 22548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.911633 (+  7116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.945878 (+ 34245us) raft_consensus.cc:1685] finished
0709 12:13:57.945901 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.963286 (+ 17385us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92576,"prepare.queue_time_us":96026,"prepare.run_cpu_time_us":149,"prepare.run_wall_time_us":3655,"raft.queue_time_us":193709,"raft.run_cpu_time_us":2199,"raft.run_wall_time_us":112537,"spinlock_wait_cycles":158199040,"thread_start_us":1735845,"threads_started":4,"wal-append.queue_time_us":612057}
I20260709 12:13:58.226594 15355 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:13:58.255275 11182 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:13:58.256532 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.191896 15233 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.243652 15362 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.241556 15180 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:13:58.269884 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.271837 15317 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:58.273191 15154 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.281795 15154 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:58.282409 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60790.4us
I20260709 12:13:58.283452 15317 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:58.285269 14539 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:58.286238 14539 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:58.286576 14539 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:58.287508 14539 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.288714 14539 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:58.291929 15409 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 12:13:58.284945 15317 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.257840 15089 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:13:58.283679 15406 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:13:58.281908 11331 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:58.294231 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:58.284082 15362 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:58.301386 15333 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.774s	user 0.019s	sys 0.013s
I20260709 12:13:58.303011 14829 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:58.312695 11178 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:58.313742 11222 leader_election.cc:400] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:13:58.322067 11222 leader_election.cc:403] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:58.314822 15333 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.331710 15194 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.335880 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:58.337044 11178 raft_consensus.cc:2368] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
W20260709 12:13:58.331734 15150 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.289s	user 0.014s	sys 0.013s
I20260709 12:13:58.381953 15150 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:13:58.344352 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 98430.4us
I20260709 12:13:58.349856 11331 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:13:58.339785 11182 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:58.349983 14982 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:58.350821 15245 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:13:58.351567 15190 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:13:58.354013 15413 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 12:13:58.358526 15410 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:13:58.359473 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.359603 14994 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.363142 15089 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:58.344619 15361 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.365587 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:58.369866 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.370234 15350 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
W20260709 12:13:58.370901 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76894.4us
I20260709 12:13:58.373749 15253 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.374159 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:58.374707 15362 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:58.375468 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.375905 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.376828 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.378911 15333 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:58.379489 11935 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:13:58.342881 11900 leader_election.cc:336] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.572s (SENT)
W20260709 12:13:58.388446 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 118368us
I20260709 12:13:58.305186 15068 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:58.388160 15415 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 12:13:58.404445 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.405449 11327 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:58.400034 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:58.412879 11179 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:13:58.412981 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.413769 11326 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 7: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:58.414667 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.414980 11180 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:58.415232 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:58.411087 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:58.418064 15355 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:58.419674 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 81410.3us
I20260709 12:13:58.421077 14952 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:58.421530 14952 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:58.422885 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:58.422652 14952 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.423832 15335 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:13:58.424117 15416 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:13:58.424304 15335 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:58.424731 15335 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:58.426023 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.426867 11330 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
W20260709 12:13:58.432165 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.396253 15150 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:13:58.466658 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.467890 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 59266.4us
I20260709 12:13:58.468269 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.469522 11324 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 12:13:58.469405 11182 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
W20260709 12:13:58.476936 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 79007.7us
I20260709 12:13:58.486575 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:58.483465 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:58.508612 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:58.514449 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:58.526809 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:58.531980 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.535512 11934 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:13:58.537815 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:13:58.523397 15257 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.588s	user 0.015s	sys 0.006s
I20260709 12:13:58.470408 15361 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.541060 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.554668 11327 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:13:58.555855 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:58.557780 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 71340.3us
I20260709 12:13:58.542912 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.542599 11180 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:58.550374 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:58.543736 11929 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:58.557987 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54631.5us
I20260709 12:13:58.559681 11178 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:58.481441 14829 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:13:58.561156 11935 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:58.559360 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:58.526024 15244 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:13:58.568995 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.547032 15073 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:58.568007 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:58.543776 15201 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:13:58.549945 15257 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.537882 15089 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:58.631712 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.621316 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:58.609596 11179 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:58.619545 11324 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
W20260709 12:13:58.682626 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.691083 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 133378us
I20260709 12:13:58.640681 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:58.615911 11173 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:13:58.637970 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 217554us
W20260709 12:13:58.708159 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.586905 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:58.566953 14994 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:58.677500 15248 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.553s	user 0.013s	sys 0.000s
I20260709 12:13:58.665573 15362 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.722218 11178 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:58.594291 15068 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:58.571995 15194 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:58.569558 15253 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:58.614130 15145 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:58.605206 15190 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.583657 15268 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:58.568257 15369 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.153s
W20260709 12:13:58.738519 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.682s (SENT) [suppressed 9 similar messages]
W20260709 12:13:58.750635 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 155725us
W20260709 12:13:58.777884 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.641373 15244 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:58.781920 15420 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:13:58.782434 15338 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:58.783034 15390 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:13:58.744570 15392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.019s	sys 0.025s
I20260709 12:13:58.571332 15281 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.169s
I20260709 12:13:58.696187 15335 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.795257 11929 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 7.
I20260709 12:13:58.731002 15248 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:13:58.740108 15073 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:58.796968 11935 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:58.802826 11177 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:13:58.803617 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:58.804883 11928 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:13:58.833241 11928 pending_rounds.cc:85] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 8
I20260709 12:13:58.809103 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:13:58.831454 11900 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.730s (SENT)
I20260709 12:13:58.822808 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:58.834132 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:58.729041 15155 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
W20260709 12:13:58.819967 15392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.020s	sys 0.026s
I20260709 12:13:58.836144 11175 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:13:58.836412 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:58.838392 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:58.839038 11930 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:13:58.796193 14994 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:58.833539 11928 pending_rounds.cc:107] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:13:58.848281 11928 raft_consensus.cc:2889] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:58.847862 15422 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:13:58.752164 15362 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:58.852026 15428 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:13:58.853055 11326 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:58.856796 11934 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:13:58.852696 15268 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:58.859762 15268 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:58.812234 15190 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.863428 15190 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:58.871089 15335 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:58.839509 15361 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:58.887449 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 67651.2us
I20260709 12:13:58.908715 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:58.909581 11248 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:58.909762 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:58.910377 11932 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:13:58.878754 15317 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:13:58.918135 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:58.919204 11322 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:58.860970 15268 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:58.820677 15338 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.884378 11329 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:58.841871 14994 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:58.892455 15390 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:13:58.932713 11326 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
W20260709 12:13:58.934456 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.942601 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 120687us
W20260709 12:13:58.933756 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 346937us
I20260709 12:13:58.934734 15244 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
W20260709 12:13:58.935127 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.938359 15430 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:13:58.933274 11930 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:58.878269 15073 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.947193 11222 leader_election.cc:400] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:58.951630 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:58.959438 11256 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:58.960215 15073 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:58.953071 11181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:13:58.956525 14994 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:58.957746 15257 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:58.958582 11222 leader_election.cc:403] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:13:58.961596 11181 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:58.967888 11181 raft_consensus.cc:614] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:58.973301 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:58.961241 15390 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:13:59.003122 11256 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:13:59.008064 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:58.998723 15073 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:13:59.026613 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.026604 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:13:59.019578 15150 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.005s
I20260709 12:13:59.034029 15434 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.020123 15281 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:59.016912 14994 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.037168 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:59.038089 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:59.038642 15431 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 12:13:59.040177 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.036693 15335 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.027976 15435 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:59.042179 15362 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.042930 15362 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:13:59.043871 15421 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.322s	user 0.003s	sys 0.001s
I20260709 12:13:59.024978 15433 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:13:59.037685 15281 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.045441 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:59.046172 15281 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:59.045426 15154 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:59.047115 15154 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:59.048436 15154 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.051198 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.051903 11250 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:13:59.038255 15190 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:13:59.064651 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 86481us
I20260709 12:13:59.072098 11932 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:13:59.033165 15150 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
W20260709 12:13:59.080188 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.089905 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 66930.7us
I20260709 12:13:59.091214 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:59.093274 11900 leader_election.cc:400] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11.
I20260709 12:13:59.093750 11900 leader_election.cc:403] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:59.041383 15335 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:13:59.103484 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69165us
I20260709 12:13:59.106884 11250 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:13:59.107108 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:59.113770 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.098507 15398 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:59.118366 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:59.061286 15434 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:59.121812 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 102320us
I20260709 12:13:59.124238 11254 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:13:59.095651 15190 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.126863 11250 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:13:59.127836 15446 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:59.128283 15201 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:59.128662 15446 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11.
I20260709 12:13:59.129951 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:59.134032 14982 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:59.134490 14982 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:59.136358 11256 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:13:59.136596 15281 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.128715 15201 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.142262 15434 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.144243 15281 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.126480 15338 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:13:59.155616 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.122742 15361 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:59.168473 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.170388 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:59.156744 15201 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.174516 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:59.176327 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.177954 15233 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:59.135468 14982 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.104316 15390 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.191121 11182 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:13:59.186975 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 93938.6us
I20260709 12:13:59.188648 11330 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:13:59.182441 14982 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.194561 11331 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:59.194906 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.195323 11930 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:13:59.195928 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:59.196051 11322 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:13:59.196460 11181 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:13:59.196652 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.200199 11326 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:59.196835 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.170105 15361 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:59.188462 15435 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:59.203615 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:13:59.190940 15338 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:13:59.206185 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.208009 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.210060 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.212026 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.212435 11926 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:59.214594 15154 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.215222 15163 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:13:59.216176 11932 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:13:59.185235 15281 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:59.222513 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.219451 11180 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:59.226305 11325 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:59.229285 15281 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:59.231441 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.231688 15281 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.232194 15434 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.232663 15434 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.234684 15434 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:59.237026 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.237870 11935 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:59.238945 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.239598 11255 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
W20260709 12:13:59.240890 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.240991 15163 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:13:59.229277 15418 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.616s	user 0.000s	sys 0.003s
I20260709 12:13:59.248939 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.252379 11326 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:13:59.246330 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 79497.3us
I20260709 12:13:59.255234 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.256880 15361 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.258023 15163 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.259042 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 701) took 1367 ms. Trace:
I20260709 12:13:59.261034 15163 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.263352 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:59.264730 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.259459 11328 rpcz_store.cc:276] 0709 12:13:57.891385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.891578 (+   193us) service_pool.cc:224] Handling call
0709 12:13:57.891994 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:57.949265 (+ 57271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:57.949318 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:57.950165 (+   847us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.266688 (+316523us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:58.266731 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.266808 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.228363 (+961555us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:59.258735 (+ 30372us) raft_consensus.cc:1685] finished
0709 12:13:59.258757 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.258976 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98796,"raft.queue_time_us":59222,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":156,"spinlock_wait_cycles":23186688,"thread_start_us":315970,"threads_started":1,"wal-append.queue_time_us":566402}
I20260709 12:13:59.266788 11930 raft_consensus.cc:2368] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:13:59.265211 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.216730 15435 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.275894 14952 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.278270 15335 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:13:59.285172 11323 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:13:59.285604 11323 pending_rounds.cc:85] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:13:59.285851 11323 pending_rounds.cc:107] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:13:59.286091 11323 raft_consensus.cc:2889] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:59.286916 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:59.287614 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.287932 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:59.288331 11926 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:13:59.289683 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.289824 15449 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:13:59.291000 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.292261 15268 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:13:59.292840 15268 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:59.294335 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.295637 11331 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:59.297596 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.254748 15387 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:59.317325 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.289426 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:13:59.319484 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.315863 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:59.328106 11255 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:59.328949 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.328671 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.329909 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:59.330691 11181 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:59.259241 15338 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:59.313938 15434 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:59.328039 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.251528 15281 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:13:59.337270 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:13:59.343149 11173 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:13:59.335151 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.337497 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.335235 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:59.347818 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.350458 15268 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.353790 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.358853 11177 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
W20260709 12:13:59.361215 15437 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.172s	user 0.002s	sys 0.000s
I20260709 12:13:59.363662 15433 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:13:59.375147 11181 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:13:59.379318 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.380496 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.418911 11255 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:13:59.382040 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 138857us
I20260709 12:13:59.382136 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.383356 15190 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.383396 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.316890 15387 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:59.334019 15448 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:59.406656 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.426928 11174 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:59.407063 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:59.314535 15150 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:13:59.422364 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.335033 15335 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:59.427361 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.428246 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.429417 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.436580 11930 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:59.420238 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:59.407495 15155 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:13:59.430796 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95668.4us
I20260709 12:13:59.432679 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.430378 15398 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.433748 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.434533 11256 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:59.402676 11325 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:59.413727 15296 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.437397 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.418112 15433 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.450213 11926 raft_consensus.cc:2368] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:13:59.451303 11175 raft_consensus.cc:2368] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:13:59.451884 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.426523 15387 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.334647 15455 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 12:13:59.435672 15268 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:59.438855 15155 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.455473 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:59.472486 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.465204 11174 raft_consensus.cc:2410] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:13:59.433427 15201 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.474541 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:13:59.476236 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 128294us
I20260709 12:13:59.455307 15433 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:59.478490 11934 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 12:13:59.491310 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.479864 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 190504us
I20260709 12:13:59.454556 15398 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.500999 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.481827 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:13:59.502384 11253 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:13:59.470201 15073 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:59.509985 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:59.450874 15449 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:59.511435 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.511870 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.512156 11252 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:13:59.463732 15296 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.513152 11933 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:13:59.514933 15296 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.477546 15155 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:59.498364 15333 raft_consensus.cc:2764] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:13:59.520110 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:59.520933 15201 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.520957 15462 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:13:59.521690 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:59.521773 15338 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.523197 15457 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:59.524760 11324 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:13:59.525169 11324 pending_rounds.cc:85] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:13:59.525020 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:59.525408 11324 pending_rounds.cc:107] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:13:59.525967 15457 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:13:59.525907 11324 raft_consensus.cc:2889] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:59.525902 15398 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:59.456646 15335 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.526739 15457 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:13:59.528470 15335 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.437831 15448 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.530459 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.531316 15448 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.527901 15457 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:59.526050 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.531860 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:13:59.533537 11930 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:59.522442 15462 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:59.553592 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.481446 15431 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:59.556221 11178 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 12:13:59.539119 11331 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:13:59.453966 15150 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:13:59.548889 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.551640 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.551994 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.481050 15081 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:59.566296 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.566488 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:59.571184 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.572898 15458 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:13:59.575524 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.576443 11182 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:13:59.536149 15194 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:59.578063 14994 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:59.578953 14994 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:59.578159 15081 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.580305 14994 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.581743 15081 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.583936 15449 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.585917 15431 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:13:59.587388 15431 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:13:59.592232 15155 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.595595 15150 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:13:59.596477 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:13:59.598897 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.601653 11256 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:59.560631 15073 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:59.602896 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.605129 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.581804 15398 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:13:59.606769 11250 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:13:59.607573 15398 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:13:59.608743 11257 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:13:59.609886 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.610407 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.611209 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.611589 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.611899 11250 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:13:59.612752 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.612870 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.615852 11174 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:59.616765 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:59.623013 11254 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:59.617449 15424 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.764s	user 0.001s	sys 0.001s
I20260709 12:13:59.624926 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.631883 11325 raft_consensus.cc:2368] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:13:59.629354 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.634651 11174 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:13:59.635982 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.636893 11257 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:13:59.638236 11257 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:13:59.553221 15462 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.650596 11329 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
W20260709 12:13:59.625768 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 126790us
I20260709 12:13:59.628615 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.658041 11255 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:59.659438 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.660466 11250 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:13:59.662828 11324 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:13:59.665491 11251 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 8.
I20260709 12:13:59.649907 15154 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:13:59.555148 15450 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 12:13:59.672415 11254 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:13:59.674799 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.675532 11328 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:13:59.676980 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.677222 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.677736 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.712736 11175 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:59.694301 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.678144 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.673552 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.678496 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.678831 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.680522 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.716349 11325 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:13:59.680809 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.681241 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.688822 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.718865 11329 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:59.673267 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.698757 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.646834 15462 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.706132 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.708334 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:13:59.723565 11330 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
W20260709 12:13:59.728084 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.723764 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:13:59.730158 11324 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:59.731314 11926 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:13:59.693893 15089 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:59.675712 15398 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:13:59.727129 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:13:59.673164 15460 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:13:59.740824 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:13:59.646127 15317 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:13:59.745666 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.752136 11322 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:13:59.747375 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:59.757754 11252 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:13:59.758519 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:13:59.774609 11178 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:13:59.777980 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.778774 11328 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:13:59.779721 11254 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:59.754514 15150 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:59.769078 15458 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:59.796782 11322 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:13:59.748405 14952 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:13:59.796361 11325 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:13:59.808300 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.797541 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:13:59.776767 15452 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:59.812240 11329 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
W20260709 12:13:59.813362 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 84460.2us
I20260709 12:13:59.817754 11328 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:13:59.819235 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:13:59.819813 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 345345us
I20260709 12:13:59.820963 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2334) took 1514 ms. Trace:
I20260709 12:13:59.788195 15150 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:13:59.797145 15458 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.821362 11176 rpcz_store.cc:276] 0709 12:13:58.269949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.313573 (+ 43624us) service_pool.cc:224] Handling call
0709 12:13:58.314026 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.351311 (+ 37285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.351347 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:58.351847 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.569663 (+217816us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:58.569712 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.569796 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.616821 (+1047025us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:13:59.776378 (+159557us) raft_consensus.cc:1685] finished
0709 12:13:59.776407 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.784293 (+  7886us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36702,"raft.queue_time_us":47307,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"spinlock_wait_cycles":4107520,"thread_start_us":217262,"threads_started":1,"wal-append.queue_time_us":401018}
I20260709 12:13:59.849454 11932 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:59.842219 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.788025 15154 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:13:59.852942 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:13:59.858227 11252 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:59.859833 11254 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:13:59.861534 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.866379 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.786115 15467 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:13:59.833662 15150 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:59.921751 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.882053 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:13:59.882877 11322 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 3. (term mismatch)
I20260709 12:13:59.883620 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.884377 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.884451 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:13:59.885241 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:13:59.834597 15296 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:13:59.886165 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:13:59.929177 11328 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:59.882037 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:13:59.890950 15471 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:13:59.918962 15423 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:59.856953 15435 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:13:59.852494 15458 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:59.926250 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 226697us
W20260709 12:13:59.930766 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 220048us
I20260709 12:13:59.923367 11322 pending_rounds.cc:85] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 2
I20260709 12:13:59.935817 11322 pending_rounds.cc:107] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 7.3
I20260709 12:13:59.936115 11322 raft_consensus.cc:2889] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:59.925473 11174 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:13:59.932021 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 108590us
I20260709 12:13:59.933936 15435 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:13:59.947664 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63331.7us
I20260709 12:13:59.952430 11179 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:13:59.937166 15296 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:13:59.940730 15201 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:13:59.971545 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:00.009353 11930 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:00.010869 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:00.019928 11255 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:13:59.964643 15335 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.035562 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:00.043120 11248 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:00.038424 15387 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 12:14:00.031971 15154 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.061971 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:00.063046 11930 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.031919 15333 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:00.052567 15268 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.005s
I20260709 12:14:00.069778 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:00.039130 15434 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.079424 11257 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 13 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
I20260709 12:14:00.067662 15333 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.029850 15472 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:14:00.042697 15335 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.110807 11250 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 12:14:00.106555 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.139261 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:00.146039 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.147032 11930 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:00.054628 15480 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:14:00.095780 15194 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:00.066972 15387 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.150260 11927 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:14:00.151729 11255 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.103724 15333 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.047518 15435 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.080384 15154 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:00.160270 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 140322us
I20260709 12:14:00.162665 11250 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:00.152859 11927 pending_rounds.cc:85] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 7
I20260709 12:14:00.165800 11927 pending_rounds.cc:107] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:14:00.166177 11927 raft_consensus.cc:2889] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:00.085068 15434 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:00.166132 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 115407us
W20260709 12:14:00.176461 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.179677 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 227588us
I20260709 12:14:00.090858 15296 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:00.183795 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77364.3us
W20260709 12:14:00.197417 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.198283 15473 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 12:14:00.208262 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.194655 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:00.209800 11250 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.212252 11253 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:00.215899 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:00.216652 11251 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.216813 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.217743 15362 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:00.219861 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.222165 11248 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:00.222971 15443 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.441s	user 0.003s	sys 0.000s
I20260709 12:14:00.223410 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:00.224588 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:00.222132 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.230214 11251 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:00.230932 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1913) took 1316 ms. Trace:
I20260709 12:14:00.231130 15362 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:00.231371 11249 rpcz_store.cc:276] 0709 12:13:58.908659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.915037 (+  6378us) service_pool.cc:224] Handling call
0709 12:13:58.915561 (+   524us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:58.916171 (+   610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.916217 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:58.917057 (+   840us) log.cc:844] Serialized 52 byte log entry
0709 12:13:59.035802 (+118745us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:59.051152 (+ 15350us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.060250 (+  9098us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.060833 (+   583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.060948 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.738656 (+677708us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18003644e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:00.222382 (+483726us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:00.224689 (+  2307us) raft_consensus.cc:1685] finished
0709 12:14:00.224711 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.224840 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3517,"prepare.queue_time_us":17705,"prepare.run_cpu_time_us":186,"prepare.run_wall_time_us":2652,"raft.queue_time_us":100761,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":3446,"spinlock_wait_cycles":14011904,"thread_start_us":118232,"threads_started":1,"wal-append.queue_time_us":191711}
I20260709 12:14:00.239665 15268 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:00.239952 15461 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:00.240070 15486 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:14:00.240497 11931 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:00.241742 15483 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:14:00.250874 11255 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:00.252282 11251 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:14:00.226969 11900 connection.cc:256] RPC call timeout handler was delayed by -1.9871211929998935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2823s for a response
W20260709 12:14:00.254221 11900 connection.cc:256] RPC call timeout handler was delayed by -1.1017918449999797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 12:14:00.255403 15154 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:00.255414 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:00.259173 11253 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:00.259531 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.260816 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:00.224768 15150 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.285190 15387 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:00.286616 11935 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:00.291337 15150 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:00.297533 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.282673 15433 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.002s
I20260709 12:14:00.306906 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:00.301891 15335 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:00.316057 15163 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:00.316552 15163 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.316931 15163 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:00.320564 15434 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:00.306667 15433 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.323861 15433 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:00.324654 11256 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:00.329828 15333 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.332363 15333 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:00.335350 15476 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
I20260709 12:14:00.340912 15163 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.351014 15155 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:00.351133 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 91689.9us
W20260709 12:14:00.348035 15462 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.598s	user 0.006s	sys 0.006s
I20260709 12:14:00.355293 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:00.356695 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:00.364158 15482 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 12:14:00.353041 15150 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:00.386183 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:00.391669 15464 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.392997 15154 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:00.387319 15494 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:00.414832 15494 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:00.415136 15488 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 12:14:00.415412 15494 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:00.395927 15390 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:00.418148 15390 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.418551 15390 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:00.419997 15435 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:00.400588 15398 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:00.409891 15464 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.374347 15458 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.427685 15458 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:00.409979 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53232.9us
I20260709 12:14:00.355175 15462 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.432080 15462 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:00.433595 11249 raft_consensus.cc:2368] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
I20260709 12:14:00.437561 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:00.438325 11325 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:00.438820 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.440848 15163 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:00.443076 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:00.455750 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:00.456971 15338 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:14:00.457609 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 290604us
I20260709 12:14:00.458196 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:00.458832 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:00.460225 11931 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:00.462154 15491 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:14:00.463373 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 277865us
I20260709 12:14:00.464167 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:00.466316 14994 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:00.466900 14994 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:00.457494 15338 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:00.467522 15464 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:00.467968 14994 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:00.470525 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.468201 15338 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:00.483601 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53945.3us
I20260709 12:14:00.489720 11223 leader_election.cc:400] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 7 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8.
I20260709 12:14:00.489883 11325 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
W20260709 12:14:00.490826 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.497475 15458 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.504890 11175 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:00.501271 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.503397 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:00.505321 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.515609 11927 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:00.505622 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.506897 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.504309 11181 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:00.511353 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.511943 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:00.499191 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:00.522537 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:00.523228 11930 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:14:00.527331 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.528285 11174 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:00.528988 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 50067.7us
I20260709 12:14:00.530871 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.520468 11931 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:00.501543 11077 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 4, term changed from 5 to 9, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193), VOTER fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194), VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195), VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 9 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:00.536330 15434 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.528713 15433 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:14:00.538041 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.536969 11077 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:00.546196 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:00.546231 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45090us
I20260709 12:14:00.547396 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.510169 15472 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.546705 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:00.550926 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.553454 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.570715 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.543654 15338 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.564620 11077 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 2 to 6, term changed from 6 to 11, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to <none>, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 11 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:00.567204 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:00.578392 11175 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:00.503526 15150 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:14:00.584232 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:00.581192 11177 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:14:00.555296 15317 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:00.504165 15154 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:14:00.596786 11927 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:00.525774 15362 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:14:00.588186 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.511310 15458 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.552731 15462 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:00.525578 15492 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:14:00.600508 11250 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
W20260709 12:14:00.602993 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.536995 15434 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.607945 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.605314 11077 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:00.610865 11175 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:14:00.621107 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:00.623189 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.594394 15155 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:00.623677 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.587706 15398 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.562650 15449 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:14:00.626952 11182 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:00.567283 15472 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.601578 15464 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:00.609935 15480 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:00.631403 11175 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:00.631971 15398 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.648180 15398 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:00.648814 15398 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.611917 15317 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:00.653612 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 86515.4us
I20260709 12:14:00.655930 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.627792 15433 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:00.629463 15335 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:00.627905 15500 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:00.665448 11177 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:00.643661 15501 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:00.649312 15398 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:00.689409 11078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 238) took 19983 ms (client timeout 15000 ms). Trace:
W20260709 12:14:00.670845 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 100210us
I20260709 12:14:00.622474 15493 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:14:00.679613 15163 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:00.680408 15390 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.683193 15472 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:00.686076 15362 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.686337 11077 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:00.703711 11077 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:00.693178 11173 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:00.657738 15491 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.692714 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.659891 15433 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.670328 15155 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:00.667929 15500 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.709910 11077 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:00.688076 15501 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.726702 11077 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:00.744583 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.693267 15163 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:00.745504 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 89664us
I20260709 12:14:00.748253 11077 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:00.714233 15492 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:00.754746 15492 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.755151 15492 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:00.755257 11077 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 1 to 10, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 10 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:00.758575 15434 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:00.758759 11077 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 5 to 8, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 8 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:00.726735 15500 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.762908 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.764453 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.765799 15458 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:00.765915 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.766710 15506 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:00.767630 11180 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:14:00.767546 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.722343 15431 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:14:00.763069 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.770023 15163 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.771335 15458 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:00.770560 11175 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:00.746951 15501 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:00.712903 15362 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.779680 15362 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:00.751729 15317 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.771561 15163 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:00.785516 15333 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.786147 15333 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.786772 15452 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.787384 15452 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.788039 15390 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:00.788244 15333 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:00.789065 15155 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:00.786765 15398 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:00.714982 15433 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:00.744670 15507 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:14:00.801433 11176 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:00.802803 11174 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:14:00.796525 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45905.6us
I20260709 12:14:00.812126 11180 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:14:00.812639 11180 pending_rounds.cc:85] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 8
I20260709 12:14:00.812911 11180 pending_rounds.cc:107] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:14:00.813277 11180 raft_consensus.cc:2889] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:00.814692 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:00.815604 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.809298 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:00.841609 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.839916 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:00.714461 15491 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:00.690165 11078 rpcz_store.cc:269] 0709 12:13:40.702092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.702355 (+   263us) service_pool.cc:224] Handling call
0709 12:14:00.685484 (+19983129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:58.315545 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:58.354406 (+ 38861us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:58.354455 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:13:58.354540 (+    85us) tablet.cc:662] Decoding operations
0709 12:13:58.356128 (+  1588us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:58.356239 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:58.356277 (+    38us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:13:58.430139 (+ 73862us) tablet.cc:701] Row locks acquired
0709 12:13:58.430207 (+    68us) write_op.cc:260] PREPARE: finished
0709 12:13:58.517233 (+ 87026us) write_op.cc:270] Start()
0709 12:13:58.553244 (+ 36011us) write_op.cc:276] Timestamp: P: 1783599238483094 usec, L: 0
0709 12:13:58.562195 (+  8951us) op_driver.cc:348] REPLICATION: starting
0709 12:13:58.721845 (+159650us) log.cc:844] Serialized 5929 byte log entry
0709 12:13:59.433362 (+711517us) op_driver.cc:464] REPLICATION: finished
0709 12:13:59.770475 (+337113us) write_op.cc:301] APPLY: starting
0709 12:13:59.783697 (+ 13222us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:59.878339 (+ 94642us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:59.878529 (+   190us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:59.880384 (+  1855us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:59.880896 (+   512us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=30
0709 12:13:59.880983 (+    87us) write_op.cc:312] APPLY: finished
0709 12:13:59.881571 (+   588us) log.cc:844] Serialized 107 byte log entry
0709 12:13:59.935053 (+ 53482us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:59.946474 (+ 11421us) write_op.cc:454] Released schema lock
0709 12:13:59.964905 (+ 18431us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:00.038194 (+ 73289us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:00.040588 (+  2394us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5536623,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":277064,"apply.run_cpu_time_us":7024,"apply.run_wall_time_us":277592,"num_ops":15,"prepare.queue_time_us":119810,"prepare.run_cpu_time_us":85946,"prepare.run_wall_time_us":659487,"raft.queue_time_us":263299,"raft.run_cpu_time_us":27199,"raft.run_wall_time_us":247586,"replication_time_us":783318,"thread_start_us":468284,"threads_started":4,"wal-append.queue_time_us":257652}]]}
I20260709 12:14:00.805152 15452 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:00.809926 15467 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:00.849450 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.853228 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 160599us
I20260709 12:14:00.869913 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.871181 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:00.871788 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.872030 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.872852 11177 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:00.873965 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.858392 15435 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:00.875439 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:00.877473 11181 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:00.877988 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:00.878072 11181 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:00.878741 11324 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:00.879396 11181 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.880875 11181 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:00.881255 11327 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:00.883342 15435 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:00.884413 15435 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:00.875650 15268 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:00.877378 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:00.887904 11322 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:14:00.894021 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:00.895009 11322 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:14:00.899689 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:00.900432 11175 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:00.904441 11179 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:00.866704 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:00.916637 11175 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:14:00.924045 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57400.4us
I20260709 12:14:00.926124 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:00.926710 11331 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:00.926863 11323 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:00.844165 15480 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:00.951759 11323 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:00.951874 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:00.953159 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:00.953601 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:00.954454 11331 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:14:00.955439 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:00.861716 15449 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:00.951874 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:00.936955 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:00.963559 11931 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:00.961977 11323 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:00.923211 15268 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:00.966504 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:00.966650 11325 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:00.971204 11325 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:00.967155 11329 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:00.971772 11329 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:00.979506 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.983812 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 80765us
I20260709 12:14:00.931965 15506 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:01.027053 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.028007 11181 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:14:01.028517 11181 pending_rounds.cc:85] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 9
I20260709 12:14:01.028867 11181 pending_rounds.cc:107] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:14:01.029184 11181 raft_consensus.cc:2889] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:01.027766 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.032884 11930 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:14:00.924332 15073 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:14:01.042227 11325 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.052109 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:01.052722 11325 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:01.020290 15296 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:01.021302 15517 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:01.060503 11329 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.098129 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:01.098935 11323 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:14:01.063295 15268 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.042057 15154 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:14:01.031811 15317 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:14:01.107607 11329 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:01.068084 15296 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:01.083551 15435 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:01.018054 15449 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:01.110411 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 124900us
I20260709 12:14:01.115477 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:01.117929 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.142441 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.158265 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.120355 15296 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:01.176991 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 126203us
I20260709 12:14:01.120834 11325 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:01.180819 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:01.184896 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:01.189864 11325 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
W20260709 12:14:01.190819 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.100934 15462 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:14:01.195264 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:00.906186 15467 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.206177 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:01.207402 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.208222 11249 raft_consensus.cc:2376] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12. Re-sending same reply.
I20260709 12:14:01.209144 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.206254 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:01.171821 15494 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.753s	user 0.011s	sys 0.001s
I20260709 12:14:01.126368 15435 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:01.169324 15333 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:01.205276 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.205603 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.210222 11254 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:01.199429 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:01.110854 15268 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.214622 11329 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:01.215183 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:01.214898 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:01.214653 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:01.113132 15492 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.218995 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:01.193048 15524 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.247566 11323 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:01.248476 15333 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:01.251673 15512 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
W20260709 12:14:01.253074 15521 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:14:01.220218 15435 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:01.258855 15519 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:01.260452 15491 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:01.247980 11257 raft_consensus.cc:2376] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11. Re-sending same reply.
I20260709 12:14:01.248571 15532 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.112475 15480 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 4, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:01.360106 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.189472 15458 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:14:01.363080 11322 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:01.359107 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:01.374006 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.374459 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.471346 11253 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:01.377533 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.005s	sys 0.030s
W20260709 12:14:01.474486 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.006s	sys 0.030s
W20260709 12:14:01.474850 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 995521us
W20260709 12:14:01.379046 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 570244us
I20260709 12:14:01.381244 11248 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:14:01.381000 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.398138 11323 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 12:14:01.483122 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.399049 11331 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:14:01.483664 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.360607 15508 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:14:01.488740 11077 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.017s	sys 0.017s
I20260709 12:14:01.381865 15515 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.218592 15494 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:01.411852 15501 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.634s	user 0.012s	sys 0.012s
I20260709 12:14:01.429384 15460 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:01.359174 11325 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:01.492267 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.492548 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:01.490190 11077 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.018s	sys 0.017s
I20260709 12:14:01.378954 15081 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 12:14:01.497018 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.495141 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 558242us
I20260709 12:14:01.379323 15514 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:14:01.522522 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:01.508008 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:01.430361 15520 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.463223 14994 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:01.509979 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:01.512966 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.490177 15398 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.539278 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:01.539597 15398 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.508395 15501 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.489521 15508 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:01.543560 11326 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:01.543846 15508 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.544839 15501 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:01.544973 15398 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:01.523607 11248 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:01.542389 15489 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:01.558482 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 360404us
I20260709 12:14:01.561538 11174 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:01.562156 11174 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:01.548025 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:01.562959 11174 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.571928 11174 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:01.531021 15497 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:14:01.504401 15201 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.613358 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:01.519997 15150 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:01.545380 15508 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:01.582458 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.510203 15194 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 12:14:01.608203 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.419417 15524 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.529922 15073 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:14:01.535969 14994 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:01.620380 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72420.4us
W20260709 12:14:01.623579 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.582775 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.685091 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.686782 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:01.689030 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.581593 14982 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:14:01.634809 15464 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:01.611452 15390 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:14:01.699230 11323 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:01.642683 15154 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:14:01.642458 15489 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:14:01.700862 15522 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:14:01.701125 15489 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:01.701742 11252 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:01.702135 15489 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:01.703291 15528 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.444s	user 0.000s	sys 0.007s
I20260709 12:14:01.703536 15489 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:01.603686 15081 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:14:01.569542 15491 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:01.572827 15452 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:01.573561 15460 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:01.653735 15527 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.233s	user 0.004s	sys 0.001s
I20260709 12:14:01.655435 15544 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:01.661147 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 672) took 1004 ms. Trace:
I20260709 12:14:01.609932 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:01.731233 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.559749 15296 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.698283 11324 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:01.503327 15515 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.489240 15532 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.736142 11324 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:01.665162 15458 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:01.709379 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 215590us
I20260709 12:14:01.666054 15150 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:01.678982 14994 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:01.764858 11323 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:01.765306 11323 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:01.766537 11255 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:01.730212 11927 rpcz_store.cc:276] 0709 12:14:00.656443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.656630 (+   187us) service_pool.cc:224] Handling call
0709 12:14:00.657046 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:00.659479 (+  2433us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:00.659519 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:01.267338 (+607819us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b2c000a0068. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:01.374833 (+107495us) log.cc:844] Serialized 26 byte log entry
0709 12:14:01.464260 (+ 89427us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:01.464312 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.464395 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:01.660786 (+196391us) raft_consensus.cc:1685] finished
0709 12:14:01.660808 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:01.661062 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":237,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7942,"spinlock_wait_cycles":15604608,"thread_start_us":88988,"threads_started":1,"wal-append.queue_time_us":183023}
I20260709 12:14:01.558986 15520 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.774322 11933 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:01.779644 11927 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:01.632423 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:01.744024 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 440638us
W20260709 12:14:01.782330 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.747248 11174 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 12:14:01.782749 11250 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:01.710775 15491 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.727624 15452 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.770308 11324 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.782819 11174 pending_rounds.cc:85] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 11
I20260709 12:14:01.709856 15511 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:01.755514 15458 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.786303 11935 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:01.786477 11323 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.786718 11174 pending_rounds.cc:107] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 12:14:01.787114 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:01.787221 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 262896us
I20260709 12:14:01.787492 11931 raft_consensus.cc:1217] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Deduplicated request from leader. Original: 8.8->[10.9-10.9]   Dedup: 10.9->[]
W20260709 12:14:01.784060 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 151768us
I20260709 12:14:01.691174 15201 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.793746 11324 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:01.789853 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47039.1us
I20260709 12:14:01.795233 11174 raft_consensus.cc:2889] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:01.753322 15333 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:01.782043 15163 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.989s	user 0.013s	sys 0.004s
I20260709 12:14:01.799975 11323 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:01.800429 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 307957us
W20260709 12:14:01.789278 11147 connection.cc:256] RPC call timeout handler was delayed by -1.7798337499998524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 12:14:01.795720 11931 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 12:14:01.790539 15433 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.981s	user 0.010s	sys 0.007s
I20260709 12:14:01.770926 15530 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:01.795717 15500 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.034s	user 0.015s	sys 0.004s
I20260709 12:14:01.790154 15491 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:01.789736 15537 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:01.822777 11330 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:01.790293 15452 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:01.792657 15511 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:01.797737 15458 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:01.794145 15545 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:01.801294 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.801354 15506 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.837549 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:01.806349 11147 connection.cc:256] RPC call timeout handler was delayed by -1.7780337499998495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 12:14:01.838428 11147 connection.cc:256] RPC call timeout handler was delayed by -1.5089337499998692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 12:14:01.838809 11147 connection.cc:256] RPC call timeout handler was delayed by -1.1885337499998059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
I20260709 12:14:01.838924 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:01.793646 15544 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:01.830322 11251 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:01.836026 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:01.797528 15268 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:01.842712 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.859454 11322 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:01.788331 15514 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:14:01.850626 15549 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:14:01.850858 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.861011 11251 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:01.851313 15457 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:01.851874 11935 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:01.852352 15543 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:14:01.853085 15541 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:14:01.853394 15464 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:01.859290 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.742131 15460 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.846850 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:01.862355 11323 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:01.817675 15492 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:01.857990 15073 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:01.867022 11324 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:01.834403 15494 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:01.836941 15545 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:01.797271 15431 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:14:01.823160 15433 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.868856 15163 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:01.835021 15500 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.903043 15505 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:01.916977 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:01.902771 15517 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:01.941998 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49193.7us
I20260709 12:14:01.952859 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:01.893143 15073 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.830624 15530 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.902505 15398 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:01.961097 11327 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:01.966176 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58070.4us
I20260709 12:14:01.966506 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:01.959501 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139601us
I20260709 12:14:01.967442 11180 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:01.968416 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.969286 15493 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:01.969202 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.977556 11935 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:14:01.914510 15163 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:01.951880 15505 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.865000 15150 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:01.905452 15461 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:14:01.906258 15501 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:14:01.862553 15506 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:01.905913 15537 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:14:02.023903 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:02.010778 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 163980us
I20260709 12:14:01.962990 15073 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:02.019003 11251 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:01.969616 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:01.957973 15494 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:14:01.958721 15434 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:14:02.051782 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:02.024713 11935 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:14:02.031280 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.031286 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:02.019337 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 193813us
I20260709 12:14:02.059819 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:02.047454 11253 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.000s
W20260709 12:14:02.049527 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2028) took 1532 ms (client timeout 1579 ms). Trace:
I20260709 12:14:01.991043 15163 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:02.064725 11179 rpcz_store.cc:269] 0709 12:14:00.496120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.497139 (+  1019us) service_pool.cc:224] Handling call
0709 12:14:02.027917 (+1530778us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:02.028046 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416251776}
I20260709 12:14:02.065448 11253 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
W20260709 12:14:02.089841 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 120269us
I20260709 12:14:02.096683 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:02.097381 11249 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:02.043848 15296 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:02.075589 15512 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:02.025431 15268 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:02.115127 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.125952 11253 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:01.958098 15511 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.119042 11296 leader_election.cc:400] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
I20260709 12:14:02.131415 11296 leader_election.cc:403] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:02.134220 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.962956 15517 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.121047 15268 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.146211 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.168592 11253 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:02.111404 15532 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.104803 15574 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:02.163494 15268 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.131500 15553 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:02.142290 15201 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.211597 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.227044 11253 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:02.117112 15545 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.214886 15570 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:02.215593 15551 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 12:14:02.216161 15576 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:02.218369 15556 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.260s	user 0.000s	sys 0.000s
W20260709 12:14:02.219388 15560 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:14:02.220924 15562 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:02.222314 15536 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.050s	user 0.003s	sys 0.000s
I20260709 12:14:02.136984 15296 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:02.222184 15467 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.224023 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:01.995553 15544 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.225020 15433 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.225809 11251 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:02.236826 15574 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:02.294905 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.295718 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.243040 15553 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:02.297595 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 178534us
I20260709 12:14:02.302747 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.257149 15296 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.133654 15520 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:02.235795 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.202217 15504 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.204s	user 0.003s	sys 0.003s
I20260709 12:14:02.238166 15545 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:02.238233 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.308380 11253 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:02.239024 15564 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:02.239566 15568 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:14:02.240024 15570 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:02.321549 15570 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.324437 15570 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.168022 15398 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:02.206524 15511 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:02.325899 15398 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.247694 15081 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:14:02.248726 15557 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:14:02.328783 15081 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:02.329595 15564 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.329912 15081 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.331110 15081 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:02.332507 15524 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.250051 15460 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.332965 15564 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.255266 11256 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:02.304347 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:02.333822 15553 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.316345 11254 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:02.240947 15501 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:14:02.337881 15501 consensus_queue.cc:1243] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:14:02.339639 15501 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:02.340278 15501 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:02.236267 15494 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:14:02.255295 15562 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.258122 15500 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:02.344225 15562 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.348006 15163 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.349951 15515 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.303853 15549 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:02.350696 15163 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:02.351732 15452 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.352873 15574 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:02.353879 15491 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.005s
I20260709 12:14:02.311808 15517 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.313519 15296 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.306814 15505 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.364977 11296 leader_election.cc:400] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:02.245217 15576 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.365669 15582 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:02.366333 15582 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.367983 15576 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.314217 15530 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:02.316345 15489 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.003s
I20260709 12:14:02.368834 15582 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.369840 11182 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:02.369947 11296 leader_election.cc:403] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:14:02.330029 15506 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.311503 15449 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.372313 15583 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:02.374253 15073 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.374886 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.384208 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.385967 11251 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:02.386202 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.359337 15491 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.324257 15520 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11.
W20260709 12:14:02.228729 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 197567us
I20260709 12:14:02.354153 15452 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.355967 15458 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.386885 11331 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:02.398483 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.398802 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.388649 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:02.399432 11328 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:02.399863 11324 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.388899 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.389096 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:02.404455 11322 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:02.390326 11256 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:02.397083 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94420.4us
W20260709 12:14:02.393910 15435 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.926s	user 0.014s	sys 0.000s
I20260709 12:14:02.393425 15073 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:02.497507 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.471671 15435 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.535013 15491 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.535626 11257 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:02.536309 15435 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:02.529969 15508 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.007s
W20260709 12:14:02.529891 14994 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.748s	user 0.011s	sys 0.004s
I20260709 12:14:02.534606 11324 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:02.480142 15585 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:14:02.546947 15585 consensus_queue.cc:1243] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:02.530968 15458 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.531594 11322 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:02.532168 11296 leader_election.cc:400] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:02.532492 15452 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:02.530573 15268 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.341398 15501 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.390326 15530 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:14:02.545084 14994 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.395891 15570 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:14:02.556155 15587 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.559785 15452 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.590296 15452 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.559641 11296 leader_election.cc:403] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:02.591765 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59751.6us
I20260709 12:14:02.593538 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:02.594508 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:02.596436 15491 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:02.573343 14994 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.594749 15584 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:02.562049 15268 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.598372 15584 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:02.598867 15584 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.599323 15584 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:02.599462 15587 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.603945 14994 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.604995 14994 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.597245 15491 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:02.610615 15491 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.596602 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.615217 11322 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:02.617401 14994 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.617802 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.618767 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:02.626109 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 219679us
I20260709 12:14:02.627722 15548 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:02.628621 15548 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:02.628983 15587 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.633023 11322 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:14:02.630803 15548 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.635571 15548 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:02.635438 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:02.636243 11931 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:02.637645 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:02.638417 11935 raft_consensus.cc:2410] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:02.621887 15501 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:02.640799 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:02.645929 11927 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:14:02.646237 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:02.643092 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.648193 11324 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:02.648949 11175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:02.649593 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:02.649705 11175 raft_consensus.cc:606] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:02.650488 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2333) took 1681 ms (client timeout 2029 ms). Trace:
W20260709 12:14:02.650872 11330 rpcz_store.cc:269] 0709 12:14:00.969232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.115042 (+145810us) service_pool.cc:224] Handling call
0709 12:14:02.650343 (+1535301us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b5400285fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:02.650430 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495484288}
I20260709 12:14:02.652591 11331 raft_consensus.cc:1217] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[6.6-12.8]   Dedup: 6.6->[10.7-12.8]
I20260709 12:14:02.653080 11331 pending_rounds.cc:85] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:14:02.653290 11331 pending_rounds.cc:107] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:14:02.653501 11331 raft_consensus.cc:2889] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:02.655447 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 534) took 1461 ms (client timeout 1882 ms). Trace:
I20260709 12:14:02.655524 11253 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:02.656060 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:02.656044 11252 rpcz_store.cc:269] 0709 12:14:01.193782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.193900 (+   118us) service_pool.cc:224] Handling call
0709 12:14:02.646290 (+1452390us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b540010c054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:02.655352 (+  9062us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750588288}
I20260709 12:14:02.656792 11330 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.643191 15583 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:14:02.662830 11322 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:02.669251 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:02.661890 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:02.677471 11933 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:02.678719 11253 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:02.679313 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.682281 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:02.677529 15452 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:02.692301 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61627.1us
W20260709 12:14:02.693509 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48673.7us
I20260709 12:14:02.630851 14994 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:14:02.689432 15489 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:02.700623 11252 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:02.688650 15081 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:02.704169 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.704957 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.639186 15584 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.711257 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:02.709164 15081 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:02.712628 15081 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.714128 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:02.714512 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.715910 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:02.716960 15458 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:02.717783 15458 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:02.718070 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.718274 15458 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:02.719285 15458 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.726693 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:02.729959 11330 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:02.731063 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:02.695106 15452 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.713392 15584 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.705956 15511 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:02.735595 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:02.736588 11330 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 7.
I20260709 12:14:02.736536 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:02.728562 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:02.739602 15333 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:02.739785 11322 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:02.741415 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.745992 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.735812 15530 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:02.752492 11331 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:02.737841 15089 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 12:14:02.748128 15596 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.123s	user 0.000s	sys 0.002s
I20260709 12:14:02.729725 15457 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:14:02.733395 15338 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:14:02.744411 15317 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:02.743637 15333 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:02.760622 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.763784 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:02.749202 15576 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:02.744594 15532 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:02.771132 15489 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:02.771258 15317 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:02.749404 15568 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:02.743896 15081 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:14:02.788045 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 529) took 1596 ms. Trace:
I20260709 12:14:02.788555 11250 rpcz_store.cc:276] 0709 12:14:01.191436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.191631 (+   195us) service_pool.cc:224] Handling call
0709 12:14:02.787820 (+1596189us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:02.787972 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974248832}
W20260709 12:14:02.794723 15589 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.445s	user 0.002s	sys 0.000s
I20260709 12:14:02.795342 15452 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.797735 15333 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.800256 15547 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:14:02.800804 15606 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:14:02.755662 15530 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.776345 15089 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:02.779485 15584 raft_consensus.cc:2764] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:14:02.784780 15317 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.800798 15547 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:02.805035 15089 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:02.805678 15593 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.445s	user 0.006s	sys 0.006s
I20260709 12:14:02.783004 15532 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.808780 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1928) took 1041 ms. Trace:
W20260709 12:14:02.809861 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49509.3us
I20260709 12:14:02.812393 15435 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.825996 11255 rpcz_store.cc:276] 0709 12:14:01.766869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.768256 (+  1387us) service_pool.cc:224] Handling call
0709 12:14:01.768513 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:01.768888 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:01.899183 (+130295us) log.cc:844] Serialized 14 byte log entry
0709 12:14:02.721295 (+822112us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b18003f77e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:02.804782 (+ 83487us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:02.806913 (+  2131us) log.cc:844] Serialized 26 byte log entry
0709 12:14:02.808281 (+  1368us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:02.808327 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.808405 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:02.808465 (+    60us) raft_consensus.cc:1685] finished
0709 12:14:02.808576 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:02.808714 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32952,"prepare.queue_time_us":215622,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":729644,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":15604,"spinlock_wait_cycles":10158976,"thread_start_us":222544,"threads_started":3,"wal-append.queue_time_us":284815}
I20260709 12:14:02.816190 15530 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:02.833776 15435 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.834422 15089 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.835836 15089 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.836755 15435 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.842891 15578 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.867925 15532 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:02.869364 15554 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.514s	user 0.000s	sys 0.003s
W20260709 12:14:02.871273 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 107574us
I20260709 12:14:02.872260 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.874040 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:02.876012 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.878512 15296 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:02.883324 15296 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.847849 15578 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.890200 15296 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.892102 15578 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.878572 15576 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:02.899376 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.900174 15594 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.883747 15568 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:02.904582 15559 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.545s	user 0.007s	sys 0.002s
I20260709 12:14:02.906052 15617 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.906694 15617 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.908861 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:02.910228 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:02.913583 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:02.924711 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:02.932097 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.934499 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.935591 11255 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:02.944758 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:02.942248 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:02.950395 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.951124 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:02.954185 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.955123 11255 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:02.962113 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:02.963107 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:02.915792 15493 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:02.965821 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55678.2us
I20260709 12:14:02.973217 11296 leader_election.cc:400] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:02.942145 15587 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:02.942845 15296 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.943805 15508 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.983726 15508 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.984702 15296 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.973788 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:02.968237 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.989388 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:02.974882 15493 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.990051 15491 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:02.990707 15491 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.991925 15493 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:02.992820 15491 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:02.990085 11252 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.944054 15524 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.948786 15333 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:02.979054 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:02.999444 11255 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:14:02.979813 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:02.981098 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.970129 15530 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:02.942413 15568 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:02.974745 15514 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:14:02.994583 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1938) took 1132 ms. Trace:
I20260709 12:14:02.960371 15567 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:03.003512 11254 rpcz_store.cc:276] 0709 12:14:01.861956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.336936 (+474980us) service_pool.cc:224] Handling call
0709 12:14:02.337340 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:02.772336 (+434996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.772371 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:02.801993 (+ 29622us) log.cc:844] Serialized 26 byte log entry
0709 12:14:02.855451 (+ 53458us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:02.855507 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.855588 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:02.994331 (+138743us) raft_consensus.cc:1685] finished
0709 12:14:02.994352 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:02.994519 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":464105,"raft.queue_time_us":437249,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":21522,"thread_start_us":53031,"threads_started":1,"wal-append.queue_time_us":179406}
I20260709 12:14:03.008445 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.976058 15587 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.886994 15547 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:02.964805 15584 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:02.947597 15573 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:14:03.005944 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:02.983651 15594 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:03.011761 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:02.970467 15548 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:02.971109 15561 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.970687 15155 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:03.015410 11254 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:03.004735 15390 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:03.022908 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:03.025105 11067 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.728s	user 0.000s	sys 0.000s
I20260709 12:14:03.014470 15567 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:03.028266 11255 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:03.038394 15617 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.064046 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:03.011682 15508 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:03.044819 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:03.067855 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.068678 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.070017 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1766) took 2179 ms. Trace:
W20260709 12:14:03.052162 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1930) took 1281 ms (client timeout 1660 ms). Trace:
I20260709 12:14:03.015556 15583 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:03.058816 15568 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.061905 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:03.032590 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.013116 15587 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:03.004137 15524 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.075820 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.089419 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:03.070634 11327 rpcz_store.cc:276] 0709 12:14:00.882469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.924141 (+ 41672us) service_pool.cc:224] Handling call
0709 12:14:00.957942 (+ 33801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.983747 (+1025805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.057511 (+1073764us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54000c7bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:03.061319 (+  3808us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2306542976}
W20260709 12:14:03.071820 11251 rpcz_store.cc:269] 0709 12:14:01.771077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.051202 (+280125us) service_pool.cc:224] Handling call
0709 12:14:03.051887 (+1000685us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540010a754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:03.052072 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394269952}
I20260709 12:14:03.075536 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:03.077067 15591 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.726s	user 0.005s	sys 0.000s
I20260709 12:14:03.093760 11254 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:03.095911 11252 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:03.096940 11325 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:14:03.098088 11326 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:03.100122 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:03.102735 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.107539 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 62805us
I20260709 12:14:03.089955 15458 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:03.115267 11926 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:03.113531 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:03.114531 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.116111 11933 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:03.116415 11251 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:03.111852 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.098052 15489 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:03.026958 15561 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.118736 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.120369 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:03.132174 11327 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:03.122414 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.132961 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:03.137902 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.139415 11325 raft_consensus.cc:2368] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12.
I20260709 12:14:03.137907 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:03.138231 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:03.139622 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.143587 11330 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 8.
I20260709 12:14:03.144711 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.143688 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.146775 11325 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:03.146659 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.114558 15583 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:03.097805 15491 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:14:03.117901 15547 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:03.150188 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 129066us
I20260709 12:14:03.096722 15480 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:03.166419 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.115234 15458 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:03.142097 15317 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:03.130486 15508 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:03.170001 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:03.170776 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.148126 15609 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:14:03.172923 11251 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:03.167574 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.148622 15501 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:03.182928 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:03.230217 11330 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:14:03.231132 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48295us
I20260709 12:14:03.231398 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:03.232605 11223 leader_election.cc:400] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12.
I20260709 12:14:03.233062 11223 leader_election.cc:403] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:03.224081 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:03.247040 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:03.168303 15489 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:03.249862 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.183672 15458 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:03.259877 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.262238 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:03.268036 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44044.7us
I20260709 12:14:03.224421 14982 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.186717 15524 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:03.270133 11251 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:14:03.277323 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:03.275271 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 105399us
I20260709 12:14:03.254953 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:03.272430 15578 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:03.256053 15532 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12.
I20260709 12:14:03.260207 15155 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:03.249222 15618 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:03.155820 15567 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.287910 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:03.266498 15520 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:03.289693 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.270430 15553 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:03.295337 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:03.317909 11174 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:03.289139 15155 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:03.234006 15508 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:03.297407 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45968.8us
I20260709 12:14:03.284351 15578 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.323510 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.258752 15514 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:14:03.307498 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.307777 11182 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:03.286921 15333 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:03.311508 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 545) took 2114 ms. Trace:
I20260709 12:14:03.312176 11255 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:03.271207 15492 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:14:03.286294 15524 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:03.284574 15462 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:03.271643 15568 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:03.274776 14994 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:14:03.279067 14982 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:03.313390 15622 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:14:03.281368 15317 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:14:03.307380 15564 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:03.326433 11257 rpcz_store.cc:276] 0709 12:14:01.197245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.560997 (+363752us) service_pool.cc:224] Handling call
0709 12:14:03.311266 (+1750269us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b540007ec54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:03.311440 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849902976}
I20260709 12:14:03.329916 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.333362 15508 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:03.344606 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2338) took 2074 ms. Trace:
I20260709 12:14:03.344889 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:03.322757 15619 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:14:03.305859 15501 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:14:03.310937 15561 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.322954 15155 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:03.344967 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:03.345057 11329 rpcz_store.cc:276] 0709 12:14:01.270030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.490136 (+220106us) service_pool.cc:224] Handling call
0709 12:14:01.507660 (+ 17524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:01.562876 (+ 55216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:02.657927 (+1095051us) log.cc:844] Serialized 14 byte log entry
0709 12:14:02.689838 (+ 31911us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:02.694015 (+  4177us) log.cc:844] Serialized 26 byte log entry
0709 12:14:02.694144 (+   129us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:02.703555 (+  9411us) log.cc:844] Serialized 14 byte log entry
0709 12:14:02.703779 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.703900 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:02.993036 (+289136us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b18001cb880. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:03.344362 (+351326us) raft_consensus.cc:1685] finished
0709 12:14:03.344390 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.344529 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1129383,"prepare.queue_time_us":1096595,"prepare.run_wall_time_us":49,"raft.queue_time_us":24571,"raft.run_cpu_time_us":491,"raft.run_wall_time_us":13605,"spinlock_wait_cycles":14144768,"thread_start_us":30217,"threads_started":1,"wal-append.queue_time_us":51305}
W20260709 12:14:03.376507 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53076.3us
I20260709 12:14:03.377597 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.380079 11327 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:03.381168 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.382295 11326 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:03.353148 15584 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:03.326241 15333 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.310688 15489 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.411257 15467 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:14:03.411895 15089 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.000s
I20260709 12:14:03.414312 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.335827 15398 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:03.382850 11327 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13. Re-sending same reply.
I20260709 12:14:03.416366 11329 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:03.417209 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:03.428228 11252 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:03.383948 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:03.384910 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.390715 11181 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:03.432132 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:03.341719 15317 consensus_queue.cc:1243] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:14:03.403293 15333 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:03.405557 14982 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:03.407680 15581 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.070s	user 0.003s	sys 0.013s
I20260709 12:14:03.334901 15524 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.352243 11182 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:03.383129 15511 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:14:03.418637 15194 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:03.365783 15583 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:14:03.394407 15631 raft_consensus.cc:2764] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:14:03.419764 15578 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.394852 15584 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:03.419720 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:03.422943 15390 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:03.424224 15629 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:14:03.426136 15627 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 12:14:03.426448 15630 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 12:14:03.425400 15633 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:14:03.439060 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54243.5us
I20260709 12:14:03.444764 11329 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:14:03.372990 15568 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:03.454298 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:03.338366 15462 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:03.400290 15625 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 12:14:03.461007 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:03.462867 11181 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:03.463275 11252 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:03.372824 15637 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:03.419147 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:03.411890 15467 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:03.357553 15564 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.503196 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:03.447010 15460 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.674750 15460 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.464494 15617 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:03.681068 11177 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2373) took 2832 ms (client timeout 2247 ms). Trace:
W20260709 12:14:03.681686 11177 rpcz_store.cc:269] 0709 12:14:00.848124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.875098 (+ 26974us) service_pool.cc:224] Handling call
0709 12:14:03.680743 (+2805645us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54000d7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:03.680943 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4430357760}
I20260709 12:14:03.682960 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:03.430295 14994 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.689340 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:03.457201 15623 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 12:14:03.551980 11926 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:14:03.439075 15338 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:03.693426 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:03.706091 11932 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:03.709789 15338 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.711736 15568 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:03.689591 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.715487 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:03.716821 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:03.717195 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:03.720950 11252 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:03.591261 15635 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:14:03.464139 15089 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:03.564934 15626 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:03.589385 15587 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.503s	user 0.004s	sys 0.004s
I20260709 12:14:03.718142 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:03.724928 15641 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:03.688731 15584 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:03.679571 15480 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:03.670729 15524 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:03.723822 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:03.736105 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.757126 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.759356 11174 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:03.770041 11174 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:03.749250 11179 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:03.770890 11252 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
W20260709 12:14:03.726027 15590 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.375s	user 0.000s	sys 0.005s
I20260709 12:14:03.754913 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.774896 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:03.747435 15506 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.779434 11327 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:03.784349 11179 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:03.784809 11179 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:03.780448 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:03.786240 11182 raft_consensus.cc:2410] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:14:03.774194 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:03.776921 15642 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:03.744575 11176 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:03.785794 11179 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.798656 11179 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:03.714107 14994 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.679544 15617 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.755470 15480 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:03.804661 11327 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:03.781672 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:03.765004 15641 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:03.752764 15089 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.754362 15587 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.844515 11177 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:03.845681 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.850674 11252 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:03.852731 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:03.853407 11252 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:03.803510 15642 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:03.839849 15622 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:14:03.861511 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.866523 11177 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:14:03.867033 11179 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
W20260709 12:14:03.863508 15458 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.003s
I20260709 12:14:03.804262 15506 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.855893 15460 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.850883 15578 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:03.846490 15645 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:03.893792 11179 pending_rounds.cc:85] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 8
I20260709 12:14:03.881700 11252 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:03.796680 15467 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.900565 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.900645 15506 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:03.900830 15640 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:14:03.902331 15607 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.125s	user 0.002s	sys 0.000s
I20260709 12:14:03.903151 11252 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:03.905205 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1796) took 1252 ms. Trace:
I20260709 12:14:03.905493 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:03.905711 11323 rpcz_store.cc:276] 0709 12:14:02.652461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.652564 (+   103us) service_pool.cc:224] Handling call
0709 12:14:02.652856 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:02.653177 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.653206 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:02.653663 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:14:02.694383 (+ 40720us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:02.694422 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.694473 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.901744 (+1207271us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:03.904893 (+  3149us) raft_consensus.cc:1685] finished
0709 12:14:03.904913 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.905137 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":986,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":92,"spinlock_wait_cycles":40435456,"thread_start_us":40385,"threads_started":1,"wal-append.queue_time_us":84119}
I20260709 12:14:03.898831 11179 pending_rounds.cc:107] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:14:03.908182 11179 raft_consensus.cc:2889] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:03.910743 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:03.852317 15398 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:14:03.862656 15338 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:03.893296 15632 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:14:03.913265 11252 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:03.922307 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.921139 15517 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:03.925175 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 539) took 2729 ms. Trace:
I20260709 12:14:03.930864 15089 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.865051 15567 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:14:03.945278 15517 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.954074 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.954564 15564 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.955096 11330 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:03.937107 11248 rpcz_store.cc:276] 0709 12:14:01.195703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.546216 (+350513us) service_pool.cc:224] Handling call
0709 12:14:01.546782 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:01.547774 (+   992us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:01.811587 (+263813us) log.cc:844] Serialized 14 byte log entry
0709 12:14:01.952168 (+140581us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:01.962999 (+ 10831us) log.cc:844] Serialized 26 byte log entry
0709 12:14:01.963146 (+   147us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:03.713939 (+1750793us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b18002e8100. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:14:03.722204 (+  8265us) log.cc:844] Serialized 14 byte log entry
0709 12:14:03.920995 (+198791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.921667 (+   672us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.924868 (+  3201us) raft_consensus.cc:1685] finished
0709 12:14:03.924897 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.925081 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67718,"prepare.queue_time_us":2101802,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":95,"raft.queue_time_us":26260,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":7080,"spinlock_wait_cycles":3479424,"thread_start_us":545230,"threads_started":3,"wal-append.queue_time_us":308088,"wal-append.run_cpu_time_us":2662,"wal-append.run_wall_time_us":1173681}
I20260709 12:14:03.957290 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.950240 11252 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:03.967459 11927 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:03.932586 15641 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.970544 15642 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:03.958917 15398 consensus_queue.cc:1243] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:03.974769 15587 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.973093 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.978294 15617 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.979668 14994 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.958189 15462 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:03.987355 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1942) took 2028 ms. Trace:
I20260709 12:14:03.987802 11253 rpcz_store.cc:276] 0709 12:14:01.961492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.679992 (+718500us) service_pool.cc:224] Handling call
0709 12:14:02.694520 (+ 14528us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:02.815049 (+120529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.815082 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:02.815684 (+   602us) log.cc:844] Serialized 52 byte log entry
0709 12:14:03.009450 (+193766us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:03.967594 (+958144us) log.cc:844] Serialized 14 byte log entry
0709 12:14:03.986950 (+ 19356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.987061 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.987110 (+    49us) raft_consensus.cc:1685] finished
0709 12:14:03.987127 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.987292 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":906326,"raft.queue_time_us":121202,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6286,"thread_start_us":378401,"threads_started":2,"wal-append.queue_time_us":279426}
I20260709 12:14:03.987111 15073 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:03.993129 15073 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:03.997028 15568 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:03.998041 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.991326 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:03.998720 11331 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:03.998832 11927 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:03.997370 15480 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.000880 15480 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:04.008104 11175 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:03.915947 15458 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:04.011260 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:03.962010 15645 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.014110 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:04.018076 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:04.014825 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:03.918339 15578 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:03.992681 15155 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.001s
I20260709 12:14:04.019650 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:03.981475 15194 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:03.985481 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.030557 11330 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:14:03.989548 15333 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:04.033269 11934 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:04.034401 15333 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:04.035624 11323 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:04.036026 11323 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:04.036105 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.041214 11330 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:14:04.050285 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:04.050369 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:04.050247 11331 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:04.050484 11323 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:04.053279 15587 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:04.053682 11323 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:04.053853 15587 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.056252 15587 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:04.057605 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1924) took 2292 ms. Trace:
I20260709 12:14:04.058024 11249 rpcz_store.cc:276] 0709 12:14:01.765616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.774598 (+  8982us) service_pool.cc:224] Handling call
0709 12:14:02.068670 (+294072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:02.119934 (+ 51264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.057401 (+1937467us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400068950. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:04.057545 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632110464}
I20260709 12:14:04.059417 11249 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:04.062088 15458 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:04.062225 11323 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:14:04.062847 11323 pending_rounds.cc:85] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:14:04.063110 11323 pending_rounds.cc:107] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:14:04.063377 11323 raft_consensus.cc:2889] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:04.063485 15530 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:04.065713 11253 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:14:04.072224 11253 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [10.5-13.7]
I20260709 12:14:04.073101 11257 pending_rounds.cc:85] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 4
I20260709 12:14:04.074965 11257 pending_rounds.cc:107] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:14:04.075423 11257 raft_consensus.cc:2889] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:04.078387 11253 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:04.019892 15571 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:14:04.097828 11253 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:04.099290 11249 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:14:04.029422 15155 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:04.113108 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 305304us
I20260709 12:14:03.929184 11067 maintenance_manager.cc:643] P bbfb3415e272477dac9ba599cc40a89d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.668s	user 0.203s	sys 0.104s Metrics: {"bytes_written":30638,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":298247,"dirs.run_cpu_time_us":2661,"dirs.run_wall_time_us":75644,"drs_written":1,"lbm_read_time_us":19161,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":2,"lbm_write_time_us":440118,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":12,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":19,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":212430,"threads_started":1,"update_count":15}
I20260709 12:14:04.124728 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:04.128088 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:04.128814 11935 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:04.142758 11253 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:14:04.145004 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:04.146566 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:04.147027 11249 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:04.147562 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1956) took 1153 ms. Trace:
I20260709 12:14:04.147909 11251 rpcz_store.cc:276] 0709 12:14:02.994025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.276760 (+282735us) service_pool.cc:224] Handling call
0709 12:14:03.290896 (+ 14136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.722604 (+431708us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:03.722905 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 12:14:03.723159 (+   254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:03.723214 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:03.723267 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.147332 (+424065us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54002a6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:04.147389 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.147507 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1922,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":33,"raft.queue_time_us":4414,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":22756,"spinlock_wait_cycles":992168832}
W20260709 12:14:04.150274 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.154026 11249 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:04.128595 15431 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:04.158723 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:04.170279 11935 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
W20260709 12:14:04.171546 15655 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:14:04.174032 11068 maintenance_manager.cc:419] P bbfb3415e272477dac9ba599cc40a89d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9343 bytes on disk
I20260709 12:14:04.174932 15646 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:04.175400 15646 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.176292 15646 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.179404 11175 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:14:04.179997 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 1949) took 1243 ms. Trace:
I20260709 12:14:04.180414 11254 rpcz_store.cc:276] 0709 12:14:02.936848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.032213 (+ 95365us) service_pool.cc:224] Handling call
0709 12:14:04.179828 (+1147615us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b540010b654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:04.179938 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37310848}
W20260709 12:14:04.181439 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 57221.2us
I20260709 12:14:04.181597 11181 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:04.182541 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:04.182862 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:04.184623 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:04.187198 11180 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:04.187237 11254 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:04.189069 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:04.190294 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:04.193950 11254 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:14:04.202183 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:04.208714 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:04.207669 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:04.206244 11178 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:04.212162 11067 maintenance_manager.cc:643] P bbfb3415e272477dac9ba599cc40a89d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.032s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":375,"lbm_reads_lt_1ms":4}
I20260709 12:14:04.215240 11248 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:04.236588 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:04.192561 15398 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:04.192840 15089 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:04.207885 15652 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:14:04.245220 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 62772.1us
W20260709 12:14:04.247319 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.218760 15650 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:04.249794 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:04.208110 15574 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.852s	user 0.011s	sys 0.018s
I20260709 12:14:04.207608 15520 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:04.145871 15645 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.170710 15431 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.221361 15646 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:04.256603 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:04.255916 15574 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.899s	user 0.011s	sys 0.018s
I20260709 12:14:04.211457 15511 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:14:04.294344 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44708.7us
I20260709 12:14:04.287678 11935 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:04.236207 15154 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:04.256836 15571 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:04.324208 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 65572.9us
I20260709 12:14:04.403538 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:04.287087 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 79420.6us
W20260709 12:14:04.405706 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3730038060000425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:14:04.406423 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 2.247s (SENT) [suppressed 12 similar messages]
I20260709 12:14:04.264148 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:04.252040 15089 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.327895 15650 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:04.412617 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:04.418005 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 153905us
I20260709 12:14:04.259671 15520 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:04.330859 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:04.329722 15578 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:04.403254 15654 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 12:14:04.257118 15390 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.158s
I20260709 12:14:04.372085 15154 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.598199 11935 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
W20260709 12:14:04.408334 15656 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 12:14:04.257426 15644 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.170s
I20260709 12:14:04.421988 14994 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:04.422883 15089 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:04.434135 15520 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:04.552950 15511 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.261027 15398 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:04.432410 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 101696us
I20260709 12:14:04.628376 11935 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:04.608309 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 175777us
W20260709 12:14:04.603344 15657 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.486s	user 0.001s	sys 0.000s
I20260709 12:14:04.613595 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:04.635321 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:04.617084 15659 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 12:14:04.612303 15506 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:04.663096 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49629.2us
W20260709 12:14:04.616612 15660 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.446s	user 0.000s	sys 0.001s
I20260709 12:14:04.615962 15644 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:04.610033 15154 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:04.667650 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:04.670480 11173 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:04.690625 11173 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:04.616140 15663 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 12:14:04.648277 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:04.615238 15390 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:04.609807 15574 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:04.633505 15155 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:04.676081 15669 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:14:04.693506 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 287097us
I20260709 12:14:04.677309 15506 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.627754 15652 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.595547 15489 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:04.699234 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:04.691397 11173 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:04.709972 11173 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:04.682199 15644 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.701049 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:04.661134 14994 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:14:04.677441 15460 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:04.696581 15390 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.722318 15390 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.723440 15390 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:04.714933 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:04.702746 15491 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:04.714614 15672 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:04.285748 15647 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.446s
I20260709 12:14:04.728590 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:04.706203 15506 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:04.728881 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:04.734974 11935 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:04.725975 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1808) took 1436 ms. Trace:
I20260709 12:14:04.736006 11326 rpcz_store.cc:276] 0709 12:14:03.289908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.464753 (+174845us) service_pool.cc:224] Handling call
0709 12:14:03.566166 (+101413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.805099 (+238933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:03.805126 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:03.841196 (+ 36070us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.020577 (+179381us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:04.020633 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.020723 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.725718 (+704995us) raft_consensus.cc:1685] finished
0709 12:14:04.725745 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.725907 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13273,"raft.queue_time_us":230520,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":23029,"thread_start_us":387414,"threads_started":2,"wal-append.queue_time_us":259480}
I20260709 12:14:04.717082 15567 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:04.705359 15514 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:14:04.745510 11173 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:14:04.748462 11173 pending_rounds.cc:85] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 8
I20260709 12:14:04.750102 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:04.739305 15480 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.003s
I20260709 12:14:04.732261 15571 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:04.754163 15661 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:04.739745 15460 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:04.758668 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 110491us
I20260709 12:14:04.734186 15671 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:04.771621 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:04.746925 15431 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:04.770224 11173 pending_rounds.cc:107] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:14:04.742947 15567 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.778424 11173 raft_consensus.cc:2889] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:04.717195 15511 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.748036 14994 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:04.739461 15650 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:14:04.781518 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 649) took 1012 ms. Trace:
I20260709 12:14:04.801543 11173 rpcz_store.cc:276] 0709 12:14:03.769020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.784580 (+ 15560us) service_pool.cc:224] Handling call
0709 12:14:03.784885 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.670359 (+885474us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54002ae520. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6663dc0 00007f3ec66ee1c0 00007f3ec66ef231 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c
0709 12:14:04.781263 (+110904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.781455 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541497600}
I20260709 12:14:04.737980 15670 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:14:04.770450 15571 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.755748 15089 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:04.777486 15672 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:04.816118 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2035) took 1365 ms. Trace:
I20260709 12:14:04.797472 15567 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:04.826110 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50597us
W20260709 12:14:04.827214 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 146641us
I20260709 12:14:04.816989 11257 rpcz_store.cc:276] 0709 12:14:03.450364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.072555 (+622191us) service_pool.cc:224] Handling call
0709 12:14:04.072894 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:04.092915 (+ 20021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.092991 (+    76us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:04.097026 (+  4035us) log.cc:844] Serialized 52 byte log entry
0709 12:14:04.246131 (+149105us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:04.815452 (+569321us) log.cc:844] Serialized 14 byte log entry
0709 12:14:04.815702 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.815796 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.815851 (+    55us) raft_consensus.cc:1685] finished
0709 12:14:04.815868 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.816046 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":554022,"raft.queue_time_us":1015,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3005,"thread_start_us":148682,"threads_started":1,"wal-append.queue_time_us":496128}
I20260709 12:14:04.821511 14994 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:04.813980 15571 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:04.840021 15569 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.066s	user 0.003s	sys 0.010s
I20260709 12:14:04.751878 15644 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.774705 15661 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:04.848548 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 120101us
I20260709 12:14:04.821244 15574 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.852650 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:04.851812 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72664.1us
W20260709 12:14:04.843364 15569 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.003s	sys 0.010s
I20260709 12:14:04.808552 15671 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:14:04.772924 15652 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.857194 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:04.861357 15155 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:04.870703 15574 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:04.873070 15664 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.251s	user 0.000s	sys 0.000s
W20260709 12:14:04.868963 15646 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.001s
W20260709 12:14:04.874399 15604 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.118s	user 0.006s	sys 0.000s
W20260709 12:14:04.877027 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.778219 15489 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:14:04.880743 15524 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.104s	user 0.013s	sys 0.000s
I20260709 12:14:04.849264 15677 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:04.856761 15460 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:04.892745 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55768.3us
I20260709 12:14:04.887960 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 717) took 1424 ms. Trace:
I20260709 12:14:04.982807 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:04.976212 15578 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:04.984264 15578 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:04.945555 15584 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.180s	user 0.010s	sys 0.003s
W20260709 12:14:04.935207 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80216.2us
I20260709 12:14:04.892365 15567 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:04.893905 15644 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:04.887151 15661 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:04.891549 15553 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:04.957182 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 127927us
I20260709 12:14:04.885586 15073 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:04.943255 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1939) took 3071 ms. Trace:
I20260709 12:14:04.980020 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:04.977439 15398 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:04.988689 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.027113 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:14:04.835356 14994 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:04.890149 15684 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:14:04.938256 15680 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:04.947767 15517 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:04.986083 15581 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.262s	user 0.011s	sys 0.005s
I20260709 12:14:04.981971 11934 rpcz_store.cc:276] 0709 12:14:03.463521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.463720 (+   199us) service_pool.cc:224] Handling call
0709 12:14:03.551541 (+ 87821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.033693 (+482152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.887642 (+853949us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54001bf154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:04.887852 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1107402624}
I20260709 12:14:05.004858 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.038166 11324 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:04.921257 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64141.1us
W20260709 12:14:04.942484 15687 consensus_queue.cc:1175] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 11 status { last_received { term: 10 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:05.025940 11256 rpcz_store.cc:276] 0709 12:14:01.871489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.406754 (+535265us) service_pool.cc:224] Handling call
0709 12:14:02.407287 (+   533us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:02.837964 (+430677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.838000 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:02.838662 (+   662us) log.cc:844] Serialized 52 byte log entry
0709 12:14:02.861302 (+ 22640us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:03.756716 (+895414us) log.cc:844] Serialized 14 byte log entry
0709 12:14:04.749972 (+993256us) spinlock_profiling.cc:243] Waited 6.53 ms on lock 0x7b1800064660. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:14:04.750190 (+   218us) log.cc:844] Serialized 14 byte log entry
0709 12:14:04.942470 (+192280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.942583 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.942838 (+   255us) raft_consensus.cc:1685] finished
0709 12:14:04.942860 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.943177 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2250798,"prepare.queue_time_us":1906910,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":88,"raft.queue_time_us":471220,"raft.run_cpu_time_us":445,"raft.run_wall_time_us":37315,"spinlock_wait_cycles":15022464,"thread_start_us":214094,"threads_started":2,"wal-append.queue_time_us":187394}
I20260709 12:14:05.010165 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 554) took 1726 ms. Trace:
I20260709 12:14:04.994217 15672 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:04.988999 15524 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.025933 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:04.975070 15679 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:14:05.046586 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1810) took 1611 ms. Trace:
I20260709 12:14:05.041078 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:05.029449 15661 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:05.044776 11252 rpcz_store.cc:276] 0709 12:14:03.283550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.965247 (+681697us) service_pool.cc:224] Handling call
0709 12:14:03.965578 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.989616 (+ 24038us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:14:04.757044 (+767428us) log.cc:844] Serialized 14 byte log entry
0709 12:14:04.831473 (+ 74429us) log.cc:844] Serialized 14 byte log entry
0709 12:14:05.009674 (+178201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.009756 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:05.009827 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.009921 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.010088 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":801974,"prepare.queue_time_us":808506,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":73,"raft.queue_time_us":14866,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":7108,"spinlock_wait_cycles":9651328,"thread_start_us":226427,"threads_started":2,"wal-append.queue_time_us":151099}
I20260709 12:14:04.994125 15652 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:05.061458 11322 rpcz_store.cc:276] 0709 12:14:03.434753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.865578 (+430825us) service_pool.cc:224] Handling call
0709 12:14:03.866026 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.870466 (+  4440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:03.870509 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:03.894821 (+ 24312us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.188148 (+293327us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:04.188187 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.188268 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.872409 (+684141us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:05.037521 (+165112us) raft_consensus.cc:1685] finished
0709 12:14:05.037547 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.046507 (+  8960us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6803,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":12476,"spinlock_wait_cycles":17363840,"thread_start_us":229268,"threads_started":1,"wal-append.queue_time_us":392759}
I20260709 12:14:04.994066 15584 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:05.073161 15683 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:14:04.991560 15677 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.079927 11329 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:05.049485 15398 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:04.995251 15689 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:14:05.047776 15511 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:05.089761 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 677) took 3278 ms. Trace:
I20260709 12:14:05.038887 15553 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.097551 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2022) took 1657 ms. Trace:
I20260709 12:14:05.129868 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.053334 15680 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:05.094399 11173 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 12:14:05.066359 15089 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.027946 15685 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:14:05.098604 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1948) took 2162 ms. Trace:
I20260709 12:14:05.143891 11255 rpcz_store.cc:276] 0709 12:14:02.936470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.131442 (+194972us) service_pool.cc:224] Handling call
0709 12:14:03.311953 (+180511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.327245 (+ 15292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.098412 (+1771167us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b540007ec50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:05.098539 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361703808}
I20260709 12:14:05.047324 15155 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:14:05.130358 11248 rpcz_store.cc:276] 0709 12:14:03.440062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.052757 (+612695us) service_pool.cc:224] Handling call
0709 12:14:04.099029 (+ 46272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.249463 (+150434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.097265 (+847802us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540010ac54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:05.097476 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266614272}
I20260709 12:14:05.107795 11930 rpcz_store.cc:276] 0709 12:14:01.803580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.805592 (+  2012us) service_pool.cc:224] Handling call
0709 12:14:01.810140 (+  4548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:01.917117 (+106977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:01.917154 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:01.957098 (+ 39944us) log.cc:844] Serialized 26 byte log entry
0709 12:14:02.213470 (+256372us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:02.299208 (+ 85738us) log.cc:844] Serialized 14 byte log entry
0709 12:14:02.299792 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.299945 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.406819 (+1106874us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:04.949611 (+1542792us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b24000311d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363e09d 00007f3ec627010e 00007f3ec628788c 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221
0709 12:14:05.081339 (+131728us) raft_consensus.cc:1685] finished
0709 12:14:05.081363 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.081540 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112282,"prepare.queue_time_us":129320,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":292864,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":20816,"spinlock_wait_cycles":177638528,"thread_start_us":335843,"threads_started":3,"wal-append.queue_time_us":322429}
W20260709 12:14:05.166785 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.132175 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.141985 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.142378 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.077852 15661 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.170908 15661 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.052259 15480 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:14:05.174518 15697 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:05.175297 15697 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.176776 15677 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.178848 15553 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.109552 15398 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:05.141183 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 149952us
I20260709 12:14:05.096680 15491 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:05.087123 15584 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:05.097843 15585 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:05.163867 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.193647 11178 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:05.100167 15524 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.083608 15687 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:05.082583 15693 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:05.207343 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.081182 15652 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.216636 11181 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:05.197134 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:05.196331 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 155337us
I20260709 12:14:05.221776 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:05.204039 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.208117 11178 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:05.203281 15390 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:05.206060 15645 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 12:14:05.210502 15638 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.783s	user 0.008s	sys 0.005s
I20260709 12:14:05.181183 15680 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.206197 15317 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:14:05.173064 15672 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:05.206115 15587 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.141s	user 0.018s	sys 0.028s
I20260709 12:14:05.215476 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.216054 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.225309 11175 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:05.226253 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.238545 11178 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:05.223170 11322 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:05.177460 15697 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.229915 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:05.230788 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.194316 15553 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.191900 15661 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.192479 15491 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:05.129961 15511 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.205324 15687 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:05.243957 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:05.237068 15587 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.019s	sys 0.028s
I20260709 12:14:05.247208 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:05.207764 15693 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:05.234900 15480 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:05.231817 15696 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:14:05.254245 11181 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:05.203099 15677 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:14:05.249378 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.228675 15671 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:05.235607 15567 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.230990 15390 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.236959 15585 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:14:05.251372 15697 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:05.295189 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.296162 11178 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:05.297492 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2346) took 2177 ms. Trace:
I20260709 12:14:05.298393 11328 rpcz_store.cc:276] 0709 12:14:03.120309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.120480 (+   171us) service_pool.cc:224] Handling call
0709 12:14:03.120785 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.184523 (+ 63738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:03.184557 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:03.248908 (+ 64351us) log.cc:844] Serialized 26 byte log entry
0709 12:14:03.317150 (+ 68242us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:03.317197 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.317276 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.209937 (+1892661us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:05.297135 (+ 87198us) raft_consensus.cc:1685] finished
0709 12:14:05.297166 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.297418 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":907,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"spinlock_wait_cycles":109799296,"thread_start_us":102374,"threads_started":2,"wal-append.queue_time_us":112593}
I20260709 12:14:05.275713 15571 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.276160 15514 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:05.271521 15645 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:05.252557 15089 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:14:05.249042 15506 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.254207 15553 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.254982 15661 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.334733 11173 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:05.336987 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70743.3us
I20260709 12:14:05.309100 15514 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:05.288376 15671 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.309828 15567 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:14:05.306880 15480 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:14:05.305274 15697 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.323591 15645 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.350273 15514 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:05.370702 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.307070 15491 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.381196 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.382082 11179 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:05.383772 15687 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.384622 15697 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.385190 15687 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:05.386646 15697 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:05.387423 15697 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.389843 15677 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:05.390086 15697 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:05.390477 15677 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.390849 15697 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:05.391561 15697 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.392990 15677 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:05.394979 15693 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.398065 15697 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.405998 15467 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:05.411659 15665 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.791s	user 0.001s	sys 0.000s
I20260709 12:14:05.412444 15553 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.426761 15578 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.430421 15561 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.430945 15571 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.431205 15561 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.433773 15661 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:05.438560 15506 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.443435 15390 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:05.377341 15684 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:05.419185 15709 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:14:05.397548 15693 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:05.458097 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.459322 11178 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:05.462630 15511 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.444361 15491 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:05.467870 11248 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:05.468436 11255 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [10.5-13.7]
I20260709 12:14:05.469691 15561 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.469843 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.470938 15645 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.461910 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.471479 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.472788 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:05.480947 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.482231 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:05.485059 15154 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.787s	user 0.009s	sys 0.005s
W20260709 12:14:05.448875 15520 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.804s	user 0.014s	sys 0.001s
I20260709 12:14:05.487700 15688 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:05.488428 15688 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.493264 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.498188 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.498682 11178 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:05.499110 11176 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:05.500238 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.501053 11182 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:05.502071 15073 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.502740 15073 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:05.505910 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 480051us
I20260709 12:14:05.510313 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.511194 11180 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:05.497401 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.515965 15694 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:05.516731 15694 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.517220 15694 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:05.523860 11927 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:05.495998 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.527275 11175 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:05.505354 15073 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.553130 15073 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.553892 15073 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.562489 15398 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.499780 15520 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.526371 15715 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.461827 15684 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.529838 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.530683 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 547) took 2714 ms. Trace:
I20260709 12:14:05.531385 15462 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:05.534082 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1960) took 1567 ms. Trace:
I20260709 12:14:05.493863 15154 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.537142 15688 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:05.538116 15573 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:14:05.530699 15670 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.595796 15398 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.652639 15398 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.653723 15154 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.655449 15398 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.655833 15462 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.610982 15715 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.595374 15689 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:05.658051 15670 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.658480 15689 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:05.544741 15644 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 12:14:05.528417 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.671424 15684 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:05.569653 11249 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:05.673478 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.675148 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.675976 11249 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:14:05.571507 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.678325 11252 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:05.680017 11252 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:05.571616 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:05.681195 11257 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:05.561141 15652 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.573303 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:05.687577 15626 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.654645 11251 rpcz_store.cc:276] 0709 12:14:03.966388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.151127 (+184739us) service_pool.cc:224] Handling call
0709 12:14:04.162241 (+ 11114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.162652 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:04.800419 (+637767us) log.cc:844] Serialized 14 byte log entry
0709 12:14:05.465804 (+665385us) spinlock_profiling.cc:243] Waited 9.04 ms on lock 0x7b4800042b58. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:14:05.492729 (+ 26925us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:05.493395 (+   666us) log.cc:844] Serialized 26 byte log entry
0709 12:14:05.493590 (+   195us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:05.493689 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.493790 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.510858 (+ 17068us) raft_consensus.cc:1685] finished
0709 12:14:05.533782 (+ 22924us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.533997 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1081129,"prepare.queue_time_us":655803,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":478215,"raft.run_cpu_time_us":14745,"raft.run_wall_time_us":1296042,"spinlock_wait_cycles":20795264,"thread_start_us":179487,"threads_started":1,"wal-append.queue_time_us":260716}
I20260709 12:14:05.574172 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.576272 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.576777 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.708586 11248 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:05.590703 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:05.709555 11256 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:14:05.593848 11250 rpcz_store.cc:276] 0709 12:14:02.816868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.817057 (+   189us) service_pool.cc:224] Handling call
0709 12:14:02.819548 (+  2491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:02.857421 (+ 37873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.857484 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:05.094840 (+2237356us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b2c00000e98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:05.095315 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:14:05.164493 (+ 69178us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:05.196023 (+ 31530us) log.cc:844] Serialized 14 byte log entry
0709 12:14:05.530254 (+334231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.530346 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.530393 (+    47us) raft_consensus.cc:1685] finished
0709 12:14:05.530489 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.530623 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":378594,"prepare.queue_time_us":38763,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":372804,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":5336,"spinlock_wait_cycles":11050880,"thread_start_us":68357,"threads_started":1,"wal-append.queue_time_us":153260}
W20260709 12:14:05.771529 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 180956us
I20260709 12:14:05.688344 15626 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.772806 15661 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:05.776311 15661 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.776800 15661 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:05.772549 11257 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:14:05.784379 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.762554 15688 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.786525 15688 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.789652 15688 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.773811 15561 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 12:14:05.790383 15666 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.965s	user 0.003s	sys 0.000s
I20260709 12:14:05.790817 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.777482 15626 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.804824 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.808171 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.810684 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.814960 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.817144 11175 raft_consensus.cc:1217] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[7.7-11.9]   Dedup: 7.7->[9.8-11.9]
I20260709 12:14:05.817955 11175 pending_rounds.cc:85] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 7
I20260709 12:14:05.818260 11175 pending_rounds.cc:107] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:14:05.818567 11175 raft_consensus.cc:2889] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:05.823177 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.824280 11177 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:05.661424 15670 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.826236 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.826707 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.838770 11249 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:05.769647 15493 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:05.557623 15671 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:05.573189 15073 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.825328 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.826851 15573 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:05.851008 11179 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:05.852015 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:05.852841 15671 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:05.852982 15573 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.858242 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.860095 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.860850 11257 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:05.834908 15652 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.880880 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:05.887514 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:05.888352 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.824857 15688 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:05.896215 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:05.896821 11926 raft_consensus.cc:2376] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12. Re-sending same reply.
I20260709 12:14:05.898578 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.849543 15398 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 12:14:05.848331 15680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.029s	sys 0.033s
I20260709 12:14:05.870868 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:05.923684 11250 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:05.923966 11249 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:05.871873 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.925536 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.926059 11173 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:05.877746 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:05.878049 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.842998 15493 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:05.885687 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:05.864285 15689 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.891198 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:05.891685 15573 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.864810 15631 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:05.891839 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:05.893743 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:05.957931 11935 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13. Re-sending same reply.
I20260709 12:14:05.891198 11180 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:14:05.914661 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.915530 11257 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:05.959825 11322 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:05.862774 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.916566 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:05.926771 11934 raft_consensus.cc:2376] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12. Re-sending same reply.
W20260709 12:14:05.935941 11145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.017s	sys 0.009s
W20260709 12:14:05.890362 15501 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:14:05.900069 15491 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.965291 11331 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
W20260709 12:14:05.962797 11145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.017s	sys 0.009s
W20260709 12:14:05.927994 15680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.030s	sys 0.033s
W20260709 12:14:05.966188 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:05.928229 14994 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.009s	sys 0.011s
I20260709 12:14:05.977923 11181 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:14:05.978060 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.978937 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:05.979436 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:05.980269 11178 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:05.974982 15582 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.981868 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.982360 15582 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.983383 11176 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:14:05.984532 15582 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.984671 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:05.971131 14994 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.010s	sys 0.011s
I20260709 12:14:05.985483 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:05.896543 15652 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:05.956617 15631 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:05.995988 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.009s	sys 0.024s
W20260709 12:14:05.993387 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.915393 15688 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:05.998664 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:05.999600 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.010s	sys 0.025s
I20260709 12:14:05.999639 11179 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
W20260709 12:14:06.007267 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.016743 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:06.027921 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.012691 15501 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.030427 15501 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.032557 15501 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:06.012744 15493 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.012807 15631 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:05.870559 15661 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.036264 15661 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:05.970726 15491 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.044533 15491 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:06.044965 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:05.972759 15626 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:06.045681 11326 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:06.046171 15626 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.047531 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:06.049513 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.050299 15520 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:06.050683 11324 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
W20260709 12:14:05.983415 15714 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:14:06.054548 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:06.055918 11181 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:06.044767 15514 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.664s	user 0.006s	sys 0.002s
I20260709 12:14:06.046868 15652 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:05.981860 15689 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:06.061793 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:06.068727 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:06.049300 15626 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:06.096271 11933 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:06.101217 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:06.102027 11179 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:06.056411 15491 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.112407 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:06.066704 15460 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.015s	sys 0.027s
I20260709 12:14:06.117355 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:06.118853 11927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 685) took 1956 ms (client timeout 1549 ms). Trace:
W20260709 12:14:06.119262 11927 rpcz_store.cc:269] 0709 12:14:04.158311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.158440 (+   129us) service_pool.cc:224] Handling call
0709 12:14:06.114464 (+1956024us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540041aad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:06.114597 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3134603648}
I20260709 12:14:06.061848 15689 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:06.121326 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:06.118182 11173 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:06.106349 15652 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.127303 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2361) took 2799 ms. Trace:
I20260709 12:14:06.127835 11329 rpcz_store.cc:276] 0709 12:14:03.327635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.731739 (+404104us) service_pool.cc:224] Handling call
0709 12:14:03.732041 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.081097 (+1349056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.126968 (+1045871us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003318d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:06.127228 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3099275136}
W20260709 12:14:06.113781 15716 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:14:06.116042 15460 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.016s	sys 0.028s
I20260709 12:14:06.090206 15514 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.134097 11933 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 12. Ops: []
I20260709 12:14:06.150576 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.155682 11329 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:06.155745 11927 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:06.137684 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.012s	sys 0.040s
W20260709 12:14:06.120365 15472 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.062s	user 0.006s	sys 0.038s
W20260709 12:14:06.157804 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.012s	sys 0.041s
W20260709 12:14:06.158293 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 817421us
I20260709 12:14:06.160300 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:06.161063 11257 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:06.166483 11250 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:14:06.166478 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.170482 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.173497 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:06.175312 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.176781 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.179279 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.169063 11329 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:06.125263 15689 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:06.180841 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.177560 11931 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:06.181864 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:06.132215 15489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.006s	sys 0.011s
I20260709 12:14:06.182914 11249 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
W20260709 12:14:06.132747 15667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.381s	user 0.018s	sys 0.020s
W20260709 12:14:06.157063 15452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.002s	sys 0.016s
I20260709 12:14:06.184957 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.189342 11933 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 8.
I20260709 12:14:06.187628 11927 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 8: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:06.186728 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:06.165891 15472 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.108s	user 0.006s	sys 0.038s
I20260709 12:14:06.193116 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:06.194093 11250 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:06.195636 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.196763 11326 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:14:06.200495 11250 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 12:14:06.188321 15667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.439s	user 0.019s	sys 0.021s
I20260709 12:14:06.202821 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:06.188705 15452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.002s	sys 0.017s
I20260709 12:14:06.117393 15491 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:06.207307 15672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.021s	sys 0.016s
I20260709 12:14:06.206380 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:06.221151 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.222481 15573 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:06.224745 15573 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:06.225788 15338 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:06.226876 15338 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:06.228019 15694 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.001s
I20260709 12:14:06.228545 15694 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.229946 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.230124 15514 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:06.220945 15672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.022s	sys 0.016s
W20260709 12:14:06.212078 15693 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.004s
I20260709 12:14:06.219974 15680 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.233651 15680 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.235723 15680 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:06.144801 15652 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.237069 11329 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:06.235735 15688 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:06.213167 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:06.213465 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.219377 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.252069 11254 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:14:06.237910 15652 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:06.234436 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:06.241066 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:06.242051 11173 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:06.243655 15491 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:06.245846 15646 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:06.256095 15646 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:06.185492 15489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.007s	sys 0.011s
I20260709 12:14:06.247323 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:06.257750 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 753) took 1253 ms. Trace:
I20260709 12:14:06.258121 11327 rpcz_store.cc:276] 0709 12:14:05.004110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.004261 (+   151us) service_pool.cc:224] Handling call
0709 12:14:06.247143 (+1242882us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:14:06.257690 (+ 10547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14039424}
W20260709 12:14:06.260716 11253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.009s	sys 0.012s
W20260709 12:14:06.261102 11253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.009s	sys 0.012s
I20260709 12:14:06.225232 15631 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.251302 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.227713 15338 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.253393 11255 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:06.205067 11257 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:14:06.264273 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.205024 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:06.261766 15089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.003s	sys 0.014s
I20260709 12:14:06.272063 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:06.276111 11174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.000s	sys 0.023s
W20260709 12:14:06.268839 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 495319us
I20260709 12:14:06.276288 11250 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:06.280273 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.283722 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.284404 11327 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:06.289866 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:06.275970 15089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.004s	sys 0.014s
I20260709 12:14:06.269878 14994 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:14:06.276592 11174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.000s	sys 0.023s
I20260709 12:14:06.303218 11255 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:06.308307 14994 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.309527 14994 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:06.290671 15573 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.286301 15467 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:06.325415 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47617.5us
I20260709 12:14:06.321260 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:06.331555 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:06.333900 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:06.340754 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:06.342075 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:06.333351 15573 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.346748 15489 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:06.347640 15573 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:06.347808 15489 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:06.348400 15489 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.348817 15489 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:06.349125 15573 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:06.334079 15467 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.352773 15467 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:06.294086 15680 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:14:06.354199 15573 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:06.355541 15573 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.350497 15694 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:06.367347 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:06.372679 11181 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:06.374562 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.380055 11176 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:06.392840 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:06.395576 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.397765 11254 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:06.399072 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.399801 11254 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:06.402148 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.286938 15646 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.410113 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:06.411264 11898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.014s	sys 0.018s
W20260709 12:14:06.413213 11898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.014s	sys 0.018s
W20260709 12:14:06.436185 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 466592us
I20260709 12:14:06.436831 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.437523 11327 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:06.372118 15492 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:06.442119 11329 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:06.342849 15661 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:14:06.448230 11928 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:06.369891 15642 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:14:06.450651 11329 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:14:06.441828 15646 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:06.458607 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.441712 15492 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:06.446522 15561 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:06.461515 15631 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:06.463469 15715 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:06.411952 15689 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.477907 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.484436 11927 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:06.453351 15452 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:06.492915 15492 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.495257 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2048) took 1028 ms. Trace:
I20260709 12:14:06.495676 11253 rpcz_store.cc:276] 0709 12:14:05.466853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.483697 (+ 16844us) service_pool.cc:224] Handling call
0709 12:14:05.484073 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:05.484351 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.484389 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:05.485013 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.261430 (+776417us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:06.261473 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.261543 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.495006 (+233463us) raft_consensus.cc:1685] finished
0709 12:14:06.495030 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.495195 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":776014,"threads_started":1,"wal-append.queue_time_us":965677}
I20260709 12:14:06.497588 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.500700 11253 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:06.516992 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:06.517939 11933 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:06.518476 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:06.519096 11927 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:06.520923 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:06.525084 11931 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:06.521206 11253 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:06.534792 11934 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:06.539016 15689 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:06.540390 15492 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:06.512367 15631 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:06.482299 15561 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.539774 15689 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:14:06.557900 11927 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:14:06.558909 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:06.559583 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.560077 15670 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.565579 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:06.567358 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.596141 11329 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
W20260709 12:14:06.480330 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.569557 15685 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:06.570050 11935 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:06.611711 15646 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.612229 15652 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:06.612425 15650 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:06.612780 15652 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.512948 15452 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.518633 15626 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:14:06.561468 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:06.621824 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.002s	sys 0.022s
W20260709 12:14:06.622211 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.002s	sys 0.022s
I20260709 12:14:06.614907 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:06.619962 11927 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:14:06.622859 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 737209us
I20260709 12:14:06.548816 11255 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:06.605888 15460 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:06.636572 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:06.599691 15679 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:06.613924 11323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.020s	sys 0.007s
W20260709 12:14:06.570575 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49723.4us
W20260709 12:14:06.650375 11323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.020s	sys 0.007s
I20260709 12:14:06.651113 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:06.616933 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.610483 15685 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:06.639258 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:06.636572 15672 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:06.671011 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.636029 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:06.554734 15631 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.653146 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.653331 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:06.612497 15646 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.654295 11182 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:06.658828 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.666935 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:06.616493 15715 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:06.681697 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:06.625545 15667 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.682618 11934 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:06.636873 15693 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.609167 15670 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.690747 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.674796 15685 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:06.676990 15661 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:06.613401 15650 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.699281 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.675428 15672 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:06.678676 15582 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:06.642163 15520 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:14:06.669342 15679 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.696496 15317 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:06.715659 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:06.711683 11256 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.008s	sys 0.015s
W20260709 12:14:06.717168 11249 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.020s	sys 0.018s
I20260709 12:14:06.718304 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:06.704988 15672 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:06.727471 11329 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:06.705253 15689 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:06.726336 11249 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.021s	sys 0.019s
I20260709 12:14:06.687644 15715 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.694218 15667 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:06.716321 15472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.007s	sys 0.015s
W20260709 12:14:06.738143 11326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.004s	sys 0.020s
W20260709 12:14:06.741915 11326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.005s	sys 0.020s
I20260709 12:14:06.716578 15661 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:06.721177 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:06.751060 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 464096us
I20260709 12:14:06.714550 15583 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:06.695637 15693 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:06.761546 15735 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:06.766156 11255 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:06.717363 15737 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:06.769830 15661 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.716691 14994 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.767258 15452 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:06.711141 15582 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:06.740293 11256 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.009s	sys 0.015s
W20260709 12:14:06.772761 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 430658us
W20260709 12:14:06.756970 15727 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 12:14:06.784689 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:06.786000 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:06.786645 11255 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:06.754665 15652 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:06.741751 15689 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.803194 11181 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
W20260709 12:14:06.803537 11250 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.002s	sys 0.018s
W20260709 12:14:06.805253 11250 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.002s	sys 0.018s
W20260709 12:14:06.800956 15732 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:14:06.847043 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2399) took 1630 ms. Trace:
I20260709 12:14:06.854959 11255 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:06.847913 11174 rpcz_store.cc:276] 0709 12:14:05.216265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.216504 (+   239us) service_pool.cc:224] Handling call
0709 12:14:05.237016 (+ 20512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.237230 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:05.306253 (+ 69023us) log.cc:844] Serialized 14 byte log entry
0709 12:14:06.781157 (+1474904us) log.cc:844] Serialized 14 byte log entry
0709 12:14:06.846501 (+ 65344us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.846590 (+    89us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:06.846656 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.846762 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.846951 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84257,"prepare.queue_time_us":102556,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":1243,"raft.queue_time_us":38117,"spinlock_wait_cycles":33324416,"thread_start_us":1508062,"threads_started":3,"wal-append.queue_time_us":1464961}
I20260709 12:14:06.876101 15670 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:06.883316 15720 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:14:06.890782 11255 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:14:06.896445 11255 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [10.9-11.10]
I20260709 12:14:06.899880 11255 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:14:06.905618 11255 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 12:14:06.907308 15667 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:06.909096 15734 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:14:06.910157 15648 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:06.910890 15648 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:06.914886 15582 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:06.924085 15489 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.575s	user 0.003s	sys 0.013s
I20260709 12:14:06.924695 15489 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:06.858460 15729 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:06.930850 15729 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:06.931432 15729 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:06.960071 15689 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:06.962699 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 91603.8us
I20260709 12:14:06.963778 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:06.971032 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:06.913483 15648 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:06.966809 15647 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.995671 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:06.974479 15726 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:07.002971 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 757) took 1123 ms. Trace:
I20260709 12:14:07.003634 11328 rpcz_store.cc:276] 0709 12:14:05.879154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.879332 (+   178us) service_pool.cc:224] Handling call
0709 12:14:05.879668 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:05.880135 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.880178 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:05.897811 (+ 17633us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.415211 (+517400us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:06.415268 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.415359 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.002686 (+587327us) raft_consensus.cc:1685] finished
0709 12:14:07.002714 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.002892 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5893,"raft.queue_time_us":110,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9735,"thread_start_us":449511,"threads_started":1,"wal-append.queue_time_us":807190}
I20260709 12:14:06.986654 15718 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:06.996583 15583 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:07.039005 11222 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.029s
W20260709 12:14:07.044191 11222 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.005s	sys 0.029s
W20260709 12:14:07.044754 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 1.54906e+06us
W20260709 12:14:07.047919 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 411938us
I20260709 12:14:07.049336 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:07.040884 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2370) took 1009 ms. Trace:
I20260709 12:14:07.054579 11322 rpcz_store.cc:276] 0709 12:14:06.032017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.032214 (+   197us) service_pool.cc:224] Handling call
0709 12:14:06.032507 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.032720 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:06.577720 (+545000us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.040337 (+462617us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.040420 (+    83us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:07.040505 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.040603 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.040802 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":286129,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":577,"thread_start_us":953469,"threads_started":2}
I20260709 12:14:06.999001 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:07.064172 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.072912 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 651) took 1256 ms. Trace:
I20260709 12:14:07.000849 15648 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:14:07.073415 11180 rpcz_store.cc:276] 0709 12:14:05.816705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.833522 (+ 16817us) service_pool.cc:224] Handling call
0709 12:14:05.834532 (+  1010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.959217 (+124685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.069712 (+1110495us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b540016f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:07.072835 (+  3123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129560960}
I20260709 12:14:07.039593 15713 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:07.002169 15647 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:07.079262 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 80357.3us
W20260709 12:14:07.081261 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.082509 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:07.084398 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:07.085605 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:07.088883 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:07.090240 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:07.091650 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:07.093111 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:07.094512 11328 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:14:07.094815 11325 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:14:07.095052 11328 pending_rounds.cc:85] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:14:07.095350 11325 pending_rounds.cc:85] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:14:07.095504 11328 pending_rounds.cc:107] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:14:07.095803 11325 pending_rounds.cc:107] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:14:07.095984 11328 raft_consensus.cc:2889] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:07.096194 11325 raft_consensus.cc:2889] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:07.097868 11330 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:07.098302 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:07.102883 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:07.114348 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 461060us
I20260709 12:14:07.115615 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:07.080381 15713 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.118134 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:07.138427 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:07.142267 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.003s	sys 0.011s
W20260709 12:14:07.142724 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.003s	sys 0.011s
W20260709 12:14:07.143091 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 541554us
I20260709 12:14:07.047689 15583 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:07.119115 15317 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:07.119374 15700 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:07.141546 15713 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:07.171940 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:07.140028 15520 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:07.181825 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 99428.1us
I20260709 12:14:07.184389 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:07.160467 15652 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:07.186254 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94837.4us
I20260709 12:14:07.146795 15753 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:07.168169 15751 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:07.190016 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:07.219209 11328 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:07.216116 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1983) took 1005 ms. Trace:
I20260709 12:14:07.221143 11252 rpcz_store.cc:276] 0709 12:14:06.210363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.210500 (+   137us) service_pool.cc:224] Handling call
0709 12:14:06.210860 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.211072 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:06.621729 (+410657us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.215681 (+593952us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.215749 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:07.215808 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.215895 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.216029 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":769749,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":66307,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":995623,"threads_started":3}
I20260709 12:14:07.186911 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:07.226446 11323 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:14:07.226962 11323 pending_rounds.cc:85] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:14:07.227265 11323 pending_rounds.cc:107] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:14:07.227509 11323 raft_consensus.cc:2889] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:07.181392 15694 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:07.228850 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:07.246198 11323 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:07.187672 15748 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:07.187628 15520 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:07.243908 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59633.6us
I20260709 12:14:07.272337 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:07.270529 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:07.245893 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55953.6us
W20260709 12:14:07.244990 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 101302us
I20260709 12:14:07.227845 15751 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:07.275754 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:07.226521 15489 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:07.287360 15489 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:07.262429 15718 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:07.264519 15726 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:07.316180 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 650) took 1499 ms. Trace:
I20260709 12:14:07.314064 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:07.212759 15648 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:07.320007 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 204513us
I20260709 12:14:07.320457 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:07.330298 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:07.330520 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:07.331938 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:07.332777 11177 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:14:07.283473 15746 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 12:14:07.218246 15753 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:07.275059 15520 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:07.269366 15748 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:07.340808 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:07.342432 11177 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:07.317703 11175 rpcz_store.cc:276] 0709 12:14:05.816322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.816465 (+   143us) service_pool.cc:224] Handling call
0709 12:14:05.816866 (+   401us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:05.823486 (+  6620us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:05.843528 (+ 20042us) log.cc:844] Serialized 14 byte log entry
0709 12:14:06.774497 (+930969us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:06.781603 (+  7106us) log.cc:844] Serialized 52 byte log entry
0709 12:14:06.781819 (+   216us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:07.194379 (+412560us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.246750 (+ 52371us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.308901 (+ 62151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.315470 (+  6569us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.315597 (+   127us) raft_consensus.cc:1685] finished
0709 12:14:07.315613 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.316039 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92212,"prepare.queue_time_us":54760,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":83,"raft.queue_time_us":135812,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":3537,"spinlock_wait_cycles":31734528,"thread_start_us":1367276,"threads_started":4,"wal-append.queue_time_us":706746}
I20260709 12:14:07.321540 15763 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:07.322830 15726 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:07.347144 15672 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.002s
I20260709 12:14:07.366073 15763 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:07.352864 15652 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:07.348105 15761 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:07.347366 15753 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:07.474680 15785 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:07.476153 15785 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:07.476754 15785 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:07.500046 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 212058us
I20260709 12:14:07.502084 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:07.533917 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1991) took 1241 ms. Trace:
I20260709 12:14:07.534675 11253 rpcz_store.cc:276] 0709 12:14:06.292518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.520570 (+228052us) service_pool.cc:224] Handling call
0709 12:14:06.520855 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.521088 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:06.604927 (+ 83839us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.075009 (+470082us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.285805 (+210796us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.533392 (+247587us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.533476 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:07.533526 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.533619 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.533832 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1079,"prepare.queue_time_us":28517,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":89,"raft.queue_time_us":124205,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":188,"thread_start_us":898390,"threads_started":3,"wal-append.queue_time_us":575977}
I20260709 12:14:07.348129 15748 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:07.366938 15648 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:07.337347 15489 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:14:07.378830 15726 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:07.391177 15742 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 12:14:07.558533 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56729.9us
I20260709 12:14:07.586970 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:07.557030 15755 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
W20260709 12:14:07.603399 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 270673us
I20260709 12:14:07.395081 15763 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:07.401376 15652 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:07.555048 15685 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.847s	user 0.008s	sys 0.005s
W20260709 12:14:07.562274 15767 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 12:14:07.607478 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:07.549986 15757 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:07.400944 15761 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:07.377842 15460 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.198s
I20260709 12:14:07.616317 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1981) took 1408 ms. Trace:
I20260709 12:14:07.617153 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:07.584633 15493 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:07.605543 15735 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:14:07.626389 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:07.576335 15647 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:07.625315 11251 rpcz_store.cc:276] 0709 12:14:06.208312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.208431 (+   119us) service_pool.cc:224] Handling call
0709 12:14:06.230721 (+ 22290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:06.231115 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.231143 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:06.231689 (+   546us) log.cc:844] Serialized 52 byte log entry
0709 12:14:06.731956 (+500267us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:07.078672 (+346716us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.322436 (+243764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.322870 (+   434us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.616073 (+293203us) raft_consensus.cc:1685] finished
0709 12:14:07.616095 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.616252 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":249301,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":45876,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":175,"thread_start_us":1071548,"threads_started":3,"wal-append.queue_time_us":734863}
I20260709 12:14:07.550478 15661 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:14:07.552035 15749 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:14:07.390046 15672 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:07.607615 15338 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:07.618763 15492 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.065s	user 0.013s	sys 0.001s
I20260709 12:14:07.618425 15652 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:07.667582 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2072) took 1523 ms. Trace:
I20260709 12:14:07.604867 15583 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:07.630141 15761 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:07.686933 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60633us
I20260709 12:14:07.689018 15764 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:07.598134 15700 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 12:14:07.707118 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 99431us
I20260709 12:14:07.710269 11325 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:07.624956 15757 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:07.624068 15780 raft_consensus.cc:2764] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:14:07.697105 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1989) took 1427 ms. Trace:
I20260709 12:14:07.724314 11254 rpcz_store.cc:276] 0709 12:14:06.269653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.400601 (+130948us) service_pool.cc:224] Handling call
0709 12:14:06.400986 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.403397 (+  2411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.403444 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:06.404034 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.789085 (+385051us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:06.789141 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.789232 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.680508 (+891276us) raft_consensus.cc:1685] finished
0709 12:14:07.680537 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.697023 (+ 16486us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4562,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":384640,"threads_started":1,"wal-append.queue_time_us":674707}
I20260709 12:14:07.698511 11173 rpcz_store.cc:276] 0709 12:14:06.147614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.147764 (+   150us) service_pool.cc:224] Handling call
0709 12:14:06.154758 (+  6994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.253329 (+ 98571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.667183 (+1413854us) spinlock_profiling.cc:243] Waited 86.9 ms on lock 0x7b54003480d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:07.667486 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199956864}
I20260709 12:14:07.582130 15452 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 12:14:07.734890 11324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 760) took 1836 ms (client timeout 1833 ms). Trace:
I20260709 12:14:07.705258 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1820) took 1508 ms. Trace:
W20260709 12:14:07.692997 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 371330us
I20260709 12:14:07.720651 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:07.393054 15751 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:07.695988 15760 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.435s	user 0.000s	sys 0.001s
I20260709 12:14:07.736542 11326 rpcz_store.cc:276] 0709 12:14:06.200686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.201863 (+  1177us) service_pool.cc:224] Handling call
0709 12:14:06.202184 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.202590 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.202629 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:06.203166 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.746354 (+543188us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:06.746413 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.746734 (+   321us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.705008 (+958274us) raft_consensus.cc:1685] finished
0709 12:14:07.705031 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.705187 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1801,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4147,"thread_start_us":542752,"threads_started":1,"wal-append.queue_time_us":842532}
I20260709 12:14:07.729691 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2061) took 1279 ms. Trace:
W20260709 12:14:07.708599 15758 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.445s	user 0.000s	sys 0.001s
I20260709 12:14:07.744493 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:07.632493 15785 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:07.735571 11324 rpcz_store.cc:269] 0709 12:14:05.898132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.898319 (+   187us) service_pool.cc:224] Handling call
0709 12:14:07.734583 (+1836264us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400285fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:07.734790 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348210560}
I20260709 12:14:07.743878 11248 rpcz_store.cc:276] 0709 12:14:06.450320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.728092 (+277772us) service_pool.cc:224] Handling call
0709 12:14:06.732879 (+  4787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.733324 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.733366 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:06.733960 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.093924 (+359964us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:07.093968 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.094033 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.729295 (+635262us) raft_consensus.cc:1685] finished
0709 12:14:07.729330 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.729568 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":388956,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"thread_start_us":340329,"threads_started":1,"wal-append.queue_time_us":525903}
W20260709 12:14:07.746281 11331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 759) took 1850 ms (client timeout 1923 ms). Trace:
I20260709 12:14:07.629542 15685 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:07.690411 15626 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
W20260709 12:14:07.763052 11331 rpcz_store.cc:269] 0709 12:14:05.895460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.895622 (+   162us) service_pool.cc:224] Handling call
0709 12:14:07.746022 (+1850400us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b54000c71d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:07.746199 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89240960}
I20260709 12:14:07.679654 15647 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:14:07.723922 15757 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:07.791712 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 184293us
I20260709 12:14:07.792928 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:07.755030 15718 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.823966 15631 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:07.788750 15751 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:07.846405 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.019s	sys 0.021s
W20260709 12:14:07.849665 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.019s	sys 0.021s
I20260709 12:14:07.708389 15648 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:07.818840 15647 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:07.825217 15631 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:07.852481 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:07.851883 15766 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.461s	user 0.000s	sys 0.001s
W20260709 12:14:07.852272 15754 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.461s	user 0.001s	sys 0.000s
I20260709 12:14:07.852156 15796 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:14:07.904538 15783 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 12:14:07.852699 15797 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:14:07.916481 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 64090.7us
I20260709 12:14:07.853344 15762 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:07.852504 15790 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:07.905189 11323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.016s	sys 0.033s
I20260709 12:14:07.839179 15718 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:08.003520 11323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.017s	sys 0.034s
I20260709 12:14:07.913767 15765 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:07.852260 15792 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:07.875277 15452 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.966948 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:07.852380 15779 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:07.859143 15647 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:07.854246 15764 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
W20260709 12:14:07.970098 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 263033us
W20260709 12:14:07.850636 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 575716us
I20260709 12:14:07.955823 15796 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:07.966400 15797 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:07.968464 15762 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:08.010783 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:08.033679 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:08.034859 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:08.016963 11174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.000s	sys 0.012s
W20260709 12:14:08.009702 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 272617us
W20260709 12:14:08.018939 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.003s	sys 0.010s
W20260709 12:14:08.036082 11174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.000s	sys 0.013s
W20260709 12:14:08.036280 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.003s	sys 0.010s
I20260709 12:14:08.003422 15790 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:07.957213 15631 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.036610 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:08.036573 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 760701us
I20260709 12:14:08.048609 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:08.044883 11928 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:08.049883 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 794) took 1589 ms. Trace:
W20260709 12:14:08.018642 15775 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
I20260709 12:14:08.007892 15779 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:08.245287 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.004s	sys 0.019s
W20260709 12:14:08.367955 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.005s	sys 0.019s
W20260709 12:14:08.368433 15493 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.734s	user 0.013s	sys 0.013s
W20260709 12:14:08.372354 15726 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.766s	user 0.017s	sys 0.004s
I20260709 12:14:08.372913 15726 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.265928 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:08.061002 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:08.050359 11928 rpcz_store.cc:276] 0709 12:14:06.460116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.460518 (+   402us) service_pool.cc:224] Handling call
0709 12:14:06.460832 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:08.043544 (+1582712us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b540034fba0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6663dc0 00007f3ec66ee1c0 00007f3ec66ef231 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c
0709 12:14:08.049521 (+  5977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.049799 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23489280}
I20260709 12:14:08.011305 15452 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.074216 15460 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:08.386081 15801 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:08.386737 15801 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.387225 15801 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:08.388986 15452 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:08.024551 15642 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.003s	sys 0.012s
I20260709 12:14:08.097052 15631 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:08.389926 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 330870us
I20260709 12:14:08.390009 15823 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:08.390146 15642 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.004s	sys 0.012s
I20260709 12:14:08.391139 15823 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.391930 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:08.114897 15652 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.037726 15672 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.124102 15685 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:08.295609 15713 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.082s	user 0.011s	sys 0.004s
I20260709 12:14:08.400164 15713 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.142890 15718 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:08.113559 15729 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.182s	user 0.015s	sys 0.000s
W20260709 12:14:08.202682 15748 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.616s	user 0.012s	sys 0.000s
I20260709 12:14:08.400892 15823 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.405767 15729 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.328791 15821 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.407649 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:08.330691 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 663) took 1003 ms. Trace:
W20260709 12:14:08.332841 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.334911 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 762) took 1237 ms. Trace:
I20260709 12:14:08.329380 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2073) took 1891 ms. Trace:
W20260709 12:14:08.188794 15753 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.003s
I20260709 12:14:08.209748 15761 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.408852 11182 rpcz_store.cc:276] 0709 12:14:07.327663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.329231 (+  1568us) service_pool.cc:224] Handling call
0709 12:14:07.369365 (+ 40134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.475737 (+106372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.475788 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.475855 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:07.475907 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.330216 (+854309us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002a0154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:08.330273 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.330526 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243970944}
I20260709 12:14:08.037319 15762 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:08.209076 15763 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.582s	user 0.013s	sys 0.000s
I20260709 12:14:08.106881 15764 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:08.415333 15761 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.415777 15764 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:08.197492 15765 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:08.112558 15774 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:14:08.415351 15763 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.425302 15763 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.120357 15785 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.409225 11181 rpcz_store.cc:276] 0709 12:14:06.437499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.438673 (+  1174us) service_pool.cc:224] Handling call
0709 12:14:06.440454 (+  1781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.805926 (+365472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.328751 (+1522825us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54000cd854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:08.329132 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":833605376}
I20260709 12:14:08.037070 15790 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:08.059767 15794 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 12:14:08.103111 15795 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 12:14:08.446285 15818 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:08.446609 15646 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.447346 15818 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.449877 15826 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:08.407174 15821 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.450920 15818 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.453166 15821 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.454493 15652 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.413267 15753 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.459506 15753 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.461056 15726 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.462087 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 771) took 1219 ms. Trace:
I20260709 12:14:08.462499 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 664) took 1134 ms. Trace:
I20260709 12:14:08.087659 15796 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.463550 15670 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:08.464046 15670 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:08.465148 15796 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:08.469453 15694 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:08.449437 15650 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.470217 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 661) took 1143 ms. Trace:
I20260709 12:14:08.059043 15797 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.018431 15798 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:08.051403 15799 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:08.126856 15802 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:14:08.470681 11180 rpcz_store.cc:276] 0709 12:14:07.326901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.329075 (+  2174us) service_pool.cc:224] Handling call
0709 12:14:07.367747 (+ 38672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.368172 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.368210 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.368766 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.599034 (+230268us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:07.599084 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.599167 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.469755 (+870588us) raft_consensus.cc:1685] finished
0709 12:14:08.469796 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.470035 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13000,"raft.queue_time_us":22389,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":157730,"thread_start_us":221803,"threads_started":1,"wal-append.queue_time_us":390490}
I20260709 12:14:08.103353 15803 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.475634 15715 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.463021 11178 rpcz_store.cc:276] 0709 12:14:07.327957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.329307 (+  1350us) service_pool.cc:224] Handling call
0709 12:14:07.329682 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.336171 (+  6489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.336233 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.336771 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.612171 (+275400us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:07.612217 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.612292 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.462026 (+849734us) raft_consensus.cc:1685] finished
0709 12:14:08.462068 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.462294 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6895,"raft.queue_time_us":26010,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":11882,"thread_start_us":275072,"threads_started":1,"wal-append.queue_time_us":501554}
I20260709 12:14:08.475797 15803 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.105897 15804 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:08.108917 15810 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:14:08.351689 15816 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:08.478909 15816 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:08.479429 15816 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:08.353233 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.353747 15825 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:08.366123 15779 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:08.211366 15757 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.250037 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 1985) took 2036 ms. Trace:
W20260709 12:14:08.368409 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.01904e+06us
W20260709 12:14:08.380867 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 115469us
W20260709 12:14:08.380712 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 345752us
W20260709 12:14:08.382483 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 321439us
I20260709 12:14:08.383718 15807 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:08.381600 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 111208us
I20260709 12:14:08.075455 15647 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.404033 15713 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.405906 15748 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.498230 11257 rpcz_store.cc:276] 0709 12:14:06.213590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.275440 (+ 61850us) service_pool.cc:224] Handling call
0709 12:14:06.275819 (+   379us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:06.288871 (+ 13052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.288922 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:06.298231 (+  9309us) log.cc:844] Serialized 78 byte log entry
0709 12:14:06.845308 (+547077us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:07.262379 (+417071us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.265995 (+  3616us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.267367 (+  1372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.267465 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.249157 (+981692us) raft_consensus.cc:1685] finished
0709 12:14:08.249233 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.249747 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26672,"prepare.queue_time_us":136242,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":129400,"raft.run_cpu_time_us":544,"raft.run_wall_time_us":38818,"thread_start_us":856861,"threads_started":2,"wal-append.queue_time_us":879115}
I20260709 12:14:08.519057 15748 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:08.336356 15520 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.967s	user 0.011s	sys 0.013s
I20260709 12:14:08.520269 15757 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:08.411584 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:08.521330 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.522440 15790 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.524887 15790 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.408583 15729 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.409049 11325 rpcz_store.cc:276] 0709 12:14:07.097754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.142523 (+ 44769us) service_pool.cc:224] Handling call
0709 12:14:07.143594 (+  1071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.710902 (+567308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.334347 (+623445us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b54001c52d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:08.334682 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302619904}
I20260709 12:14:08.526317 15713 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.527308 15729 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.527027 15713 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.527997 15729 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.029731 15780 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:08.529992 11324 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:08.530594 15780 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.532152 15821 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:08.534139 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.537106 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.538106 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:08.541462 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:08.542423 11927 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:08.436998 15827 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:08.439237 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57185.3us
W20260709 12:14:08.440379 15317 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.260s	user 0.016s	sys 0.012s
I20260709 12:14:08.480618 15803 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.462554 11328 rpcz_store.cc:276] 0709 12:14:07.224432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.224653 (+   221us) service_pool.cc:224] Handling call
0709 12:14:07.225093 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.225376 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:07.244528 (+ 19152us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.615239 (+370711us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.615600 (+   361us) log.cc:844] Serialized 14 byte log entry
0709 12:14:08.337967 (+722367us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b18003ca900. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:08.377217 (+ 39250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.384445 (+  7228us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:08.409842 (+ 25397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.412519 (+  2677us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.443783 (+ 31264us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81466,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":8361,"raft.queue_time_us":443289,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":11514,"spinlock_wait_cycles":121002752,"thread_start_us":392267,"threads_started":3,"wal-append.queue_time_us":577294}
I20260709 12:14:08.492071 11331 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:08.470248 15694 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:08.494436 11326 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:08.493187 15801 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.515829 15722 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:08.416421 15764 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:08.520318 15520 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.542256 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:08.513120 15762 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.556428 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:08.558895 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2384) took 2007 ms. Trace:
I20260709 12:14:08.559470 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:08.560176 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:08.561537 15722 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.570463 11181 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:08.572904 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.563318 11327 rpcz_store.cc:276] 0709 12:14:06.551851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.551996 (+   145us) service_pool.cc:224] Handling call
0709 12:14:07.734773 (+1182777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.969110 (+234337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.969149 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.969213 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:07.969260 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.969351 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.558469 (+589118us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400285fd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:08.558696 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11955,"raft.queue_time_us":89836,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":114,"spinlock_wait_cycles":2714816896,"thread_start_us":221830,"threads_started":1}
I20260709 12:14:08.573789 11181 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:08.574777 11173 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:14:08.574941 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.575372 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.576066 11250 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:08.576159 11257 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:08.576355 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.571411 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.573804 11251 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:08.578240 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.579144 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.571121 15780 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.571388 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.574952 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.577416 11173 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:08.583000 15317 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:08.587102 11180 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:08.587257 11928 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 13. Ops: []
I20260709 12:14:08.587702 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:08.586988 11174 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:14:08.588912 15729 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.591100 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.593053 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.595041 11249 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:08.597851 14994 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.599418 15762 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.602265 15472 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:08.602922 15472 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:08.604297 15472 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.604853 15317 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:08.605351 15317 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.606168 15317 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:08.605356 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63277.2us
W20260709 12:14:08.588960 15772 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.625s	user 0.000s	sys 0.002s
I20260709 12:14:08.614709 11331 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:08.615576 15574 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.616146 15574 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.616532 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:08.617517 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:14:08.618322 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.619004 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:08.619189 11252 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:08.619673 11328 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:08.620137 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.616137 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:08.620852 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74042.2us
I20260709 12:14:08.621104 11327 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:08.614773 15561 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.622527 15561 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:08.622890 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:08.620862 11250 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:08.623992 11927 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:08.628041 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.465404 15670 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.629660 15670 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:08.629936 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.631472 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:08.633364 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 100980us
I20260709 12:14:08.636366 15722 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.634290 15713 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.646420 15154 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:08.647075 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:08.647707 15154 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:08.637679 15693 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.640880 11249 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:08.650154 15778 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:08.650731 15778 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:08.652069 15574 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.652241 15778 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.641491 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.654703 11252 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:08.648291 11326 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:08.642004 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.655838 15520 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.656279 11174 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:08.656370 15825 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:08.636734 11253 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:08.658962 15778 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:08.661243 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:08.662731 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:08.666692 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.654718 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:08.668326 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.669507 15467 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:08.671005 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.652940 15574 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.671842 11248 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:08.672009 15574 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.674057 15561 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:14:08.672029 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49317.7us
I20260709 12:14:08.682387 11328 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:08.675146 15807 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:14:08.691901 11250 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:08.693001 15517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.607s	user 0.005s	sys 0.013s
W20260709 12:14:08.693416 15517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.615s	user 0.005s	sys 0.013s
I20260709 12:14:08.668792 15679 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.686076 15472 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:08.696823 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.686841 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:08.688812 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.694198 11257 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:08.670866 15779 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.703511 11251 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:08.702394 15749 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:14:08.705593 11254 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:08.703015 15338 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.781s	user 0.017s	sys 0.009s
W20260709 12:14:08.709224 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.711071 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.716565 15786 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:08.735015 15786 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:08.725837 15759 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.361s	user 0.006s	sys 0.008s
I20260709 12:14:08.728585 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:08.729873 15713 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.730494 15089 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.730890 15821 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:08.731256 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.718305 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:08.745110 11248 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:08.738694 11327 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:08.741686 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.747462 11250 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:08.728503 15694 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.749187 11253 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:08.728468 15751 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:08.760968 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 353523us
I20260709 12:14:08.746960 15807 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:14:08.763723 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45476.4us
I20260709 12:14:08.766528 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:08.769098 11325 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:08.769583 15517 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:08.770102 15517 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:08.771039 11323 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:08.758356 15764 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.772236 15680 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:08.773149 15680 consensus_queue.cc:1243] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:08.771052 15517 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.774585 15460 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:08.774761 15517 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:08.775257 15460 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:08.736480 15338 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:14:08.778916 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:08.711004 15816 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.782491 11251 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:08.744050 15561 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:14:08.766531 15807 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:08.795691 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 67290us
I20260709 12:14:08.749727 15089 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.758234 15713 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.784243 15574 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.782030 15764 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.796141 15749 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:08.796257 15780 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:08.780396 15751 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.819748 11325 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:08.819406 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:08.790002 15317 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.816184 15749 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.834478 11330 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:08.821377 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:08.835016 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:08.823527 15815 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.823640 15801 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.854501 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:08.858405 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.859297 11174 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:08.860270 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.861029 11251 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:08.862419 15757 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:08.862955 15757 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:08.863423 15757 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:08.865011 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.867826 15815 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.876372 11253 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:08.847645 15780 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:08.873924 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.874245 15089 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.802834 15460 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.838333 15749 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:08.874768 15807 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.893110 15807 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:08.886474 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:08.897850 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 766) took 1684 ms. Trace:
I20260709 12:14:08.898487 11322 rpcz_store.cc:276] 0709 12:14:07.163195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.163659 (+   464us) service_pool.cc:224] Handling call
0709 12:14:07.164353 (+   694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.186131 (+ 21778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.186177 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.186238 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:07.186288 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.826091 (+1639803us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:08.830021 (+  3930us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.847222 (+ 17201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48535808}
I20260709 12:14:08.833803 15786 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:08.899956 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72894.9us
I20260709 12:14:08.834599 15737 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:14:08.906566 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.907827 11248 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:08.911199 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:08.911993 11928 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:08.912122 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.912842 11254 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:08.913158 15830 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:14:08.914391 15737 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:08.914798 15737 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:08.915207 15317 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.911309 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.917249 11250 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:08.877976 15779 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:08.922694 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:08.925611 11174 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:08.927073 11249 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:08.928169 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:08.923064 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139210us
W20260709 12:14:08.930977 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.933691 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:08.933892 15823 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:08.934314 11329 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:14:08.934465 15823 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:08.936646 15823 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:08.941509 15797 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:08.941859 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.925040 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:08.946906 15713 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.947296 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:08.916765 15626 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:08.948158 15626 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:08.949947 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:08.950579 15626 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:08.950824 11176 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:08.954015 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:08.955437 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.958709 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:08.969492 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:08.970242 11928 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:08.973325 11326 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:14:08.975695 11252 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:08.915944 15737 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.060325 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:08.971741 15763 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:09.067401 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:09.061573 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:09.071293 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 146328us
W20260709 12:14:09.071821 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 108647us
I20260709 12:14:09.072512 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.073298 15680 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:09.073959 15492 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:09.074000 15789 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:08.980127 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4200335240000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 12:14:09.075178 11296 connection.cc:256] RPC call timeout handler was delayed by -1.267933524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
I20260709 12:14:09.075915 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:08.977802 15816 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.077430 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.090224 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:09.070194 15763 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.101941 15631 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:09.105813 15737 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:09.102140 15751 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:09.108659 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:09.113983 15680 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:09.118525 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.086647 15726 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:09.120697 15726 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.121116 15726 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.082295 15574 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:09.107853 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.141930 11252 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
W20260709 12:14:09.141438 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 178661us
W20260709 12:14:09.152326 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 230653us
I20260709 12:14:09.153993 11329 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:09.157832 15317 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.158198 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:09.158520 15317 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.145910 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.124948 15763 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.087930 15517 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:14:09.151247 15680 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:09.182966 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:09.145099 15574 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:09.157832 15787 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:09.187975 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.083442 15780 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.190205 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:09.151515 15751 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:14:09.172024 15317 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:09.193428 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:09.198114 11173 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:09.195254 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:09.188818 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.188136 15574 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:09.213130 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.214226 15462 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.216214 15582 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:09.216728 15752 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:09.217165 15752 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:09.217445 15680 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:09.218127 15752 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.218612 15680 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.219358 15491 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:09.219753 15752 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.220005 15491 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.222340 15582 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:09.213418 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:09.227943 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:09.190976 15787 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:09.238400 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 161090us
W20260709 12:14:09.238932 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 50192.3us
W20260709 12:14:09.238369 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.241313 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.239758 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.243687 11182 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:09.243927 15815 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:09.205102 15626 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.253955 15467 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:09.255131 15467 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:09.256079 15751 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:09.256727 15751 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.257102 15751 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.258687 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.260365 15582 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:09.262061 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76726.8us
I20260709 12:14:09.262989 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.264721 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:09.266203 15491 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.245980 15462 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:09.269745 15462 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:09.270130 15462 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:09.272768 15838 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:14:09.256157 15467 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:09.278223 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.243340 15753 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:09.279240 15467 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:09.289788 15517 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:09.290481 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:09.294279 15787 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.309145 15787 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:09.310292 15472 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.614s	user 0.013s	sys 0.000s
I20260709 12:14:09.312172 15757 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.313836 11181 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
W20260709 12:14:09.318367 15778 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.002s
I20260709 12:14:09.321765 15779 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:09.321929 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:09.322463 15779 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:09.322844 11178 raft_consensus.cc:606] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.297596 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.326126 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:09.303804 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51911.9us
I20260709 12:14:09.308300 15777 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:09.329648 15777 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:09.330053 15777 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:09.331516 15817 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:09.331672 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.244302 15317 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:09.332998 11257 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:09.332830 15317 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.334811 11255 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:09.335047 15317 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:09.331103 15777 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.336966 15777 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:09.332042 15817 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.322899 15779 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:09.331130 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.317873 15626 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.307057 15467 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.340610 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.355922 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:09.347262 11174 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:09.357769 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:09.359244 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:09.347967 11256 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:09.360183 11180 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:09.347860 15517 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:09.379324 15751 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.370436 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.382944 11331 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:09.369879 15814 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.383988 11255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:14:09.384595 11255 raft_consensus.cc:606] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:09.384162 15814 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.385023 11255 raft_consensus.cc:614] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:09.386662 15467 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.346364 15753 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:09.391269 15626 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:09.390964 15753 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.361541 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:09.376915 15749 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.395758 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.396523 11256 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:09.396782 15492 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:09.396937 15749 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.397253 15492 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:09.397858 11175 raft_consensus.cc:2368] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:09.398845 15317 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:09.398317 15492 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.399266 15317 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.399720 15317 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.399911 15694 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:09.399991 15492 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:09.378417 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:09.402180 11329 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:09.403460 15647 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.345901 15817 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:09.375032 15757 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.408151 15827 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:09.415477 15827 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:09.416610 15827 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.417882 15816 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.418373 15751 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:09.422189 15827 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:09.423290 15631 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:09.428413 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.431725 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.433821 11182 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:09.434186 15786 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:09.436141 15817 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:09.437212 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.442160 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.442874 11329 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:09.443835 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 153445us
I20260709 12:14:09.445983 15814 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.446401 11331 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:09.447373 15814 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.436683 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.405715 11176 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
W20260709 12:14:09.460222 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 179930us
I20260709 12:14:09.463533 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:09.461009 11249 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:14:09.470402 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:09.473850 11329 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:09.464469 11182 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:09.474603 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.475101 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.460268 15752 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:09.477535 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.484097 11324 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:09.463778 15561 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:09.486194 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.488193 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:09.491382 11331 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:09.490695 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:09.492978 11327 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 8.
I20260709 12:14:09.496769 15460 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:09.498450 15462 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.500846 15462 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:09.500743 15574 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.484560 15626 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:09.460073 15814 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:09.510311 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.491660 15815 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:09.491458 15764 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:09.490082 15787 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:09.479158 15801 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:14:09.503556 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.518514 11327 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:09.521449 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.522549 11328 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:09.506892 15561 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:09.508028 15626 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.556303 15626 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.570047 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.571585 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:09.572185 15814 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.573081 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.573802 11328 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:09.574904 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:09.575871 11179 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:09.577497 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:09.578799 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.580309 15817 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.557690 15805 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:09.589963 11326 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:14:09.591095 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:09.516464 15815 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:09.594369 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:09.600557 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.602499 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 112337us
W20260709 12:14:09.606329 15807 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.713s	user 0.013s	sys 0.000s
I20260709 12:14:09.608753 11331 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:09.609575 11326 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:09.614481 15726 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.551591 15787 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:09.595278 15764 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:09.617913 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 138113us
I20260709 12:14:09.516799 15763 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.619473 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:09.618502 15787 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.618266 15764 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.615808 15317 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.603686 15815 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:09.602093 15816 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:09.605223 15561 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:09.630302 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:09.665550 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.665707 11328 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
W20260709 12:14:09.676558 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.683451 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.626361 15764 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:09.584164 15574 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:09.700805 15797 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.759s	user 0.006s	sys 0.003s
I20260709 12:14:09.764885 11328 pending_rounds.cc:85] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 5
W20260709 12:14:09.769023 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.714136 15467 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:09.764089 15582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.028s	sys 0.037s
I20260709 12:14:09.780833 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:09.765357 11328 pending_rounds.cc:107] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:14:09.745507 15797 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:09.788506 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.782733 11328 raft_consensus.cc:2889] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:09.793480 11182 raft_consensus.cc:606] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:09.794116 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2085) took 1212 ms. Trace:
I20260709 12:14:09.794634 11181 rpcz_store.cc:276] 0709 12:14:08.572062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.574625 (+  2563us) service_pool.cc:224] Handling call
0709 12:14:09.784344 (+1209719us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54000d7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:09.784474 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1625850240}
I20260709 12:14:09.770864 15814 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.800071 11256 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:14:09.801409 15661 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:09.801585 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.774740 15472 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:09.807940 11257 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
W20260709 12:14:09.819082 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.819788 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.829844 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:09.831446 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 212566us
I20260709 12:14:09.803730 15786 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:09.779222 11326 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:09.837467 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:09.838697 11327 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:14:09.783936 15467 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:09.796705 15574 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:14:09.807485 15661 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:09.835518 15762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.014s	sys 0.060s
W20260709 12:14:09.860967 15762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.014s	sys 0.060s
I20260709 12:14:09.868088 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:09.851716 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:09.872980 11322 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 9: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:09.834616 15786 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.880793 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:09.883359 11935 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:09.900527 11928 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:09.902838 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.906298 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:09.910099 11927 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:09.918773 15661 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.920635 15317 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:09.847874 15814 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:09.922894 15777 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.003s
I20260709 12:14:09.923650 15814 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:09.926255 15467 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:09.927676 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 116407us
W20260709 12:14:09.940049 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.895928 15786 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:09.941074 15726 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:09.942098 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 72599.3us
I20260709 12:14:09.942937 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:09.944362 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:09.947029 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.951357 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:09.929437 15647 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.002s
W20260709 12:14:09.872937 15780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.028s	sys 0.044s
W20260709 12:14:09.970871 15780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.028s	sys 0.045s
I20260709 12:14:09.971938 11253 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:09.832878 15472 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:09.986907 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:09.965420 15680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.053s	sys 0.041s
I20260709 12:14:10.015643 11327 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:10.016181 11327 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:09.982465 15089 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:09.982981 15834 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:09.988304 15777 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:10.005541 15779 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.001s
W20260709 12:14:10.004729 15680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.054s	sys 0.041s
I20260709 12:14:10.023849 11323 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
W20260709 12:14:10.025848 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.033486 11323 pending_rounds.cc:85] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:14:10.029992 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:10.033777 11323 pending_rounds.cc:107] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:14:10.034051 11323 raft_consensus.cc:2889] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:10.034839 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 90591.3us
I20260709 12:14:10.028358 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.037922 11927 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:10.034041 11322 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:10.052374 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.053129 11927 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:10.067564 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:09.876677 15561 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.020849 15089 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:10.079651 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.078619 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:10.082222 11322 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:14:10.090842 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.078135 11327 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.083124 11254 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:10.026144 15647 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
W20260709 12:14:10.086987 15789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.010s
I20260709 12:14:10.121613 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:10.084425 15089 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.033236 15779 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:10.124992 11251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.012s	sys 0.017s
W20260709 12:14:10.131968 11251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.012s	sys 0.017s
I20260709 12:14:10.129617 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:10.133406 11322 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:10.126822 15789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.008s	sys 0.011s
I20260709 12:14:10.075167 15764 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.141736 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.138386 11327 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:10.142781 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2015) took 1240 ms. Trace:
I20260709 12:14:10.147356 11254 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:10.147941 11177 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.010s	sys 0.007s
W20260709 12:14:10.148303 11177 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.011s	sys 0.007s
I20260709 12:14:10.150323 11249 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:10.152768 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:10.166285 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:10.178421 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 249896us
I20260709 12:14:10.178684 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:10.180851 11254 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:14:10.175700 15680 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:10.212222 11327 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 12:14:10.212455 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70823.3us
I20260709 12:14:10.227397 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:10.228216 11182 raft_consensus.cc:606] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:10.223534 11327 pending_rounds.cc:85] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:14:10.230631 11327 pending_rounds.cc:107] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:14:10.230981 11327 raft_consensus.cc:2889] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:10.235177 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.236296 11254 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:10.213608 15680 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.199072 15817 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:10.225898 15574 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:10.256615 15680 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:10.264614 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:10.242151 15851 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:14:10.240614 15472 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.287602 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.290102 11254 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:10.289191 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:10.265609 15467 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:10.278165 15855 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:14:10.305148 11173 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.013s
W20260709 12:14:10.315618 11173 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.010s	sys 0.013s
I20260709 12:14:10.266294 15574 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.298502 15631 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:10.307214 15849 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 12:14:10.323680 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.288883 15817 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.324270 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:10.307443 15779 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.328480 11252 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:10.323165 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.331316 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.332082 11932 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:10.335204 15863 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.335834 15863 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.337863 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.338824 11933 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:10.318364 15467 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:10.340653 15854 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:14:10.343225 15661 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:10.346364 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.347429 11926 raft_consensus.cc:2368] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:10.347631 15631 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:10.348191 15631 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:10.309326 15855 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.355913 15855 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:10.320276 15764 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:10.356904 15626 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.794s	user 0.007s	sys 0.005s
I20260709 12:14:10.300727 15786 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:10.356812 15845 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:14:10.377799 11931 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:14:10.357614 15626 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.378858 11928 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:10.380009 15626 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:10.380369 11249 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:10.383782 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.387801 11252 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:10.391788 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.393021 15467 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.359491 15786 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.360945 15817 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.363107 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 782) took 1551 ms. Trace:
W20260709 12:14:10.364722 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.369385 15492 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:10.367529 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.369341 15574 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.370544 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2014) took 1469 ms. Trace:
I20260709 12:14:10.375604 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.376647 11179 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:10.376935 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47565.4us
I20260709 12:14:10.377046 11327 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:10.377305 11256 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:10.401933 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:10.402096 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:10.402686 15763 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:10.403249 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:10.404471 11927 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:10.405948 11329 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:10.375048 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:10.410741 11254 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:10.411298 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.416479 11322 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:10.461151 11253 rpcz_store.cc:276] 0709 12:14:08.901408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.901563 (+   155us) service_pool.cc:224] Handling call
0709 12:14:08.919002 (+ 17439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.972458 (+1053456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.370303 (+397845us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002a62d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:10.370481 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420877568}
I20260709 12:14:10.373857 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:10.419996 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.143309 11251 rpcz_store.cc:276] 0709 12:14:08.901803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.901946 (+   143us) service_pool.cc:224] Handling call
0709 12:14:08.924250 (+ 22304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.085231 (+160981us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:09.506135 (+420904us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.142292 (+636157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:10.142370 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:10.142428 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.142511 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:10.142643 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":680674,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":6738,"raft.queue_time_us":8673,"thread_start_us":910153,"threads_started":2}
I20260709 12:14:10.421200 15680 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.473268 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:10.421761 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 677) took 1195 ms. Trace:
I20260709 12:14:10.473881 15492 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.474256 11173 rpcz_store.cc:276] 0709 12:14:09.226784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.227021 (+   237us) service_pool.cc:224] Handling call
0709 12:14:09.274736 (+ 47715us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:09.303635 (+ 28899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.303670 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:09.351826 (+ 48156us) log.cc:844] Serialized 52 byte log entry
0709 12:14:09.789551 (+437725us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:10.413982 (+624431us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.420624 (+  6642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.420698 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:10.420737 (+    39us) raft_consensus.cc:1685] finished
0709 12:14:10.420748 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:10.421700 (+   952us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6317,"prepare.queue_time_us":363033,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1828,"raft.queue_time_us":25013,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":9090,"thread_start_us":930052,"threads_started":2,"wal-append.queue_time_us":744704}
I20260709 12:14:10.395848 15863 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.460584 11926 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.477977 15492 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:10.431855 11323 rpcz_store.cc:276] 0709 12:14:08.811440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.811582 (+   142us) service_pool.cc:224] Handling call
0709 12:14:08.819190 (+  7608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.074273 (+1255083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.362858 (+288585us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54004290d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:10.363049 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2715467008}
I20260709 12:14:10.479301 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:10.479388 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:10.480108 11323 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:10.480211 11322 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:10.480997 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:10.482275 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:10.482862 11323 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:10.483997 11323 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:10.485138 11323 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:10.461169 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:10.420383 15777 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:10.486310 15786 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:10.486838 15786 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.487198 15786 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:10.488956 15752 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:10.421548 15751 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.489988 15751 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.490538 15752 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:10.431569 11252 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:10.492169 15751 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:10.473951 11324 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.494455 15089 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.496682 15089 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:10.481351 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:10.498311 11322 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:10.433010 15691 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:10.486182 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:10.504154 11932 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:10.476022 15680 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:10.507804 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 104665us
I20260709 12:14:10.485427 11323 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:10.513762 11934 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
W20260709 12:14:10.514246 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 75010us
I20260709 12:14:10.515383 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:10.509711 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:10.518929 11926 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:14:10.439404 15816 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:10.503788 15815 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.799s	user 0.011s	sys 0.003s
W20260709 12:14:10.539072 15874 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:14:10.538935 15815 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.542641 11331 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:10.543073 11331 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:10.543927 11331 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.544452 11935 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:10.505393 15849 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:14:10.545037 11331 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:10.531817 11330 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:10.546072 11330 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:10.546896 11330 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.547830 11330 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:10.549433 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 789) took 1065 ms. Trace:
I20260709 12:14:10.549875 11329 rpcz_store.cc:276] 0709 12:14:09.486026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.492782 (+  6756us) service_pool.cc:224] Handling call
0709 12:14:09.493105 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.467351 (+974246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.549211 (+ 81860us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400285ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:10.549373 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098910080}
I20260709 12:14:10.550423 15764 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:10.521651 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.520522 15626 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.538057 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:10.532872 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:10.559994 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:10.562119 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:10.564226 11322 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:10.564009 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48766.2us
I20260709 12:14:10.565940 15726 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:10.522994 15762 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:10.570236 11252 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:10.571583 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 169751us
I20260709 12:14:10.551394 15763 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:10.585947 15763 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:10.558449 15816 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.586963 15763 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.588035 15816 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:10.588647 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.571273 15680 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:10.589604 15680 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:10.590607 15680 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:10.592418 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.570968 15865 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:10.550987 15764 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.577153 11323 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.577730 11255 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:14:10.560014 15492 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:10.543889 15815 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:10.580718 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.530648 15780 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.572872 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:10.598865 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2d37a48723c42228c9717f514174704"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:10.605224 11173 raft_consensus.cc:606] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:10.560684 15751 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:10.601425 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:10.578994 15847 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:14:10.610810 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:10.602458 11324 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:14:10.613170 11256 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:10.609333 11251 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:10.617272 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:10.615556 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:10.605602 11173 raft_consensus.cc:614] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:10.604758 11250 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:10.604732 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:10.607229 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:10.619913 11322 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:10.573433 15855 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:10.605134 11932 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:10.603075 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.621973 11253 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:10.622330 11252 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:10.622826 11331 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:14:10.623239 11331 pending_rounds.cc:85] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:14:10.623520 11331 pending_rounds.cc:107] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:14:10.623821 11331 raft_consensus.cc:2889] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:10.603843 15089 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.632869 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.564080 15626 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.609129 15751 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.569867 15752 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:14:10.643018 11329 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:10.648538 11253 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:10.631806 15467 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:10.644488 15855 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:10.646456 15751 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:10.682709 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 65539.9us
I20260709 12:14:10.668694 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:10.671686 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.675874 11252 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:10.610513 15472 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:14:10.612603 15561 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:14:10.612784 15865 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:14:10.680154 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.690066 11256 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.684814 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:10.611925 15780 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.691556 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:10.651343 15154 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:10.674625 15805 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:10.733700 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:10.741285 11252 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:10.734345 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 80146.8us
W20260709 12:14:10.764988 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 122600us
I20260709 12:14:10.770087 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:10.773428 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.775134 11330 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:14:10.776140 11330 pending_rounds.cc:85] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:10.780691 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:10.778901 11330 pending_rounds.cc:107] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:14:10.781924 11330 raft_consensus.cc:2889] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:10.644554 15089 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.782907 11253 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
W20260709 12:14:10.784735 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 95817us
I20260709 12:14:10.794541 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:10.795379 11252 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:10.764715 15862 raft_consensus.cc:2764] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:14:10.814551 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:10.816046 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.803723 11330 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:10.833907 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:10.803328 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 177673us
I20260709 12:14:10.785264 15631 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.817118 15816 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:14:10.837322 15855 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:10.840313 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 75541.6us
I20260709 12:14:10.852665 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:10.861696 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:10.864636 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70795.5us
I20260709 12:14:10.867245 11249 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:14:10.847232 15780 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:10.871834 15764 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:10.872433 15626 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:10.838672 15472 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:10.873090 15472 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:10.851577 15726 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:10.873884 15472 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.875294 15472 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:10.875598 15726 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:10.876310 15726 consensus_queue.cc:1243] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:10.876781 15726 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:14:10.841085 15882 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:14:10.879588 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:10.848816 15631 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:10.884163 11256 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:14:10.843717 15089 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:10.878393 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:10.901305 11329 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
W20260709 12:14:10.894790 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 102993us
I20260709 12:14:10.899712 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:10.877362 15726 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:10.903803 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:10.906523 15777 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:10.906975 15777 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.907399 15777 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:10.909809 15574 raft_consensus.cc:2764] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:14:10.910714 15574 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:10.911221 15862 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:10.850514 15892 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:10.912340 15822 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:10.912770 15822 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:10.835260 15787 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:14:10.913769 15822 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:10.915032 15822 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:10.921054 11249 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:10.921604 11249 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:10.850193 15752 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:14:10.852679 15779 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:14:10.886489 15467 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:10.928041 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:10.929556 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:10.949241 11254 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:10.930063 11249 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.826973 15786 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.839421 15891 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:14:10.932365 15789 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:14:10.943868 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:10.954485 11249 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:10.944583 15661 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:10.955734 15870 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:10.956208 15870 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:10.958645 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:10.946197 15726 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:10.901270 15868 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:10.898497 15805 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:10.986868 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.054561 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 110772us
I20260709 12:14:11.055816 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:11.057025 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.058256 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:10.919745 15888 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:11.108896 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 180913us
I20260709 12:14:11.124964 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:11.125728 11322 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:11.085263 15631 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:11.085400 15888 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:11.152361 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94170.5us
W20260709 12:14:11.155493 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.085458 15752 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:11.170521 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 282970us
I20260709 12:14:10.957154 15870 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.085263 15574 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.106243 15868 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:11.123257 15862 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:11.186546 15862 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.189109 15862 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:11.171416 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.128707 15764 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:11.184247 15467 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.185459 15752 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:11.172052 15871 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:14:11.199683 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:11.203723 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 52717us
I20260709 12:14:11.205864 15863 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:11.169087 15888 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:11.176591 15898 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:11.213122 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1861) took 1434 ms. Trace:
I20260709 12:14:11.226500 11325 rpcz_store.cc:276] 0709 12:14:09.778976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.780003 (+  1027us) service_pool.cc:224] Handling call
0709 12:14:09.814909 (+ 34906us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.135745 (+320836us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:10.506778 (+371033us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.805299 (+298521us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.888376 (+ 83077us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.888992 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.889238 (+   246us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:10.889587 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.951730 (+ 62143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.951866 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.212828 (+260962us) raft_consensus.cc:1685] finished
0709 12:14:11.212852 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.213044 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46823,"prepare.queue_time_us":73622,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":11362,"raft.queue_time_us":139290,"raft.run_cpu_time_us":4155,"raft.run_wall_time_us":47982,"spinlock_wait_cycles":105344,"thread_start_us":842449,"threads_started":4,"wal-append.queue_time_us":275737}
I20260709 12:14:11.220984 15691 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:11.222002 15902 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:11.190631 15574 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.223364 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.191005 15900 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:11.232741 11932 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:11.243582 11926 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:11.244798 15779 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:14:11.245589 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 451119us
W20260709 12:14:11.246132 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46557.1us
I20260709 12:14:11.246564 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.247488 15472 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:11.247536 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:11.248590 15779 raft_consensus.cc:2764] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:14:11.211411 15764 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:11.275810 15764 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.121137 11249 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:11.280179 15574 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:11.231316 15902 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:11.281955 15902 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:11.226343 15898 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.286314 15898 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:11.265126 15786 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:11.266458 15868 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.267690 15900 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.198743 15751 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.311487 15900 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:11.311801 15751 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:11.270243 15752 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:11.206362 15805 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.231038 15691 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:11.301923 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:11.317159 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.350623 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.367905 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:11.380501 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 689) took 1475 ms (client timeout 1587 ms). Trace:
W20260709 12:14:11.381040 11179 rpcz_store.cc:269] 0709 12:14:09.905039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.905967 (+   928us) service_pool.cc:224] Handling call
0709 12:14:11.380234 (+1474267us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400001ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:11.380417 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079688448}
I20260709 12:14:11.385392 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:11.385612 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:11.386286 11932 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:11.391886 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 142302us
I20260709 12:14:11.380458 15751 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:11.385942 15821 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:11.398149 15821 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:11.399199 15821 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.399902 15902 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.386873 15777 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.402695 15902 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.403240 15777 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.406250 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.346750 15752 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:11.408041 15752 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.408639 15752 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:11.409296 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.410421 15862 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:11.410859 15862 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:11.411296 15862 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:11.385870 15631 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:11.418937 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.421808 15904 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.422533 15855 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:11.422484 15904 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.424480 15789 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.425122 15904 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.423187 15855 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.425925 15855 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:11.428828 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:11.429530 11325 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:11.425055 15789 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.431372 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.348356 15574 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.394029 15863 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.407586 15888 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.443789 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.452149 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.453045 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.454352 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.444478 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.455902 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.456297 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.457098 11934 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:11.461282 11932 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:11.463119 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.459764 15631 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:11.451071 15574 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.466996 15574 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:11.469527 15888 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.470849 15789 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.472352 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:11.474591 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:11.471591 15751 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:11.471552 15862 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.478350 15777 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:11.479187 15777 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:11.481273 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.481590 15777 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:11.456990 15863 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.484539 15862 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:11.484938 15863 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:11.476182 15751 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:11.491820 15751 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.494603 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:11.497376 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.497828 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:11.498353 11928 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:11.498682 11934 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:11.499013 15154 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:11.504451 15752 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:11.507926 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.509984 15888 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:11.510712 15888 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.513276 15888 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.516462 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:11.520256 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.525784 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.526489 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.520444 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57418.5us
I20260709 12:14:11.521132 15888 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:11.518908 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:11.522526 15855 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.528230 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:11.464035 11325 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:11.543399 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 822) took 1373 ms. Trace:
I20260709 12:14:11.543748 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.544399 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:11.544934 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.546631 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:11.548666 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:11.522962 15752 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:11.543879 11935 rpcz_store.cc:276] 0709 12:14:10.161431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.161588 (+   157us) service_pool.cc:224] Handling call
0709 12:14:10.179478 (+ 17890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.544898 (+365420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.534508 (+989610us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540041aad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:11.534767 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838795520}
I20260709 12:14:11.553184 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:11.554864 11330 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:11.559022 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.561038 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.561978 11256 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:14:11.580881 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:11.585958 15691 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.587194 11330 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:14:11.544544 15779 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:11.554788 15786 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:11.556984 15574 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:11.541414 15868 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:11.563995 15816 raft_consensus.cc:2764] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:14:11.554610 15753 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:11.554186 15867 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:14:11.602931 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:11.626273 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 75716.8us
W20260709 12:14:11.619372 11252 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.004s	sys 0.024s
W20260709 12:14:11.618106 11329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.011s	sys 0.018s
I20260709 12:14:11.627576 11330 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
W20260709 12:14:11.627707 11252 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.004s	sys 0.024s
W20260709 12:14:11.627925 11329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.011s	sys 0.018s
I20260709 12:14:11.638180 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:11.638784 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:11.646198 11325 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:11.650583 11330 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:11.590574 15898 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.622535 15816 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:11.648730 15631 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:11.654686 15561 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:11.716254 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.587s (SENT) [suppressed 22 similar messages]
W20260709 12:14:11.730000 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.658073 15816 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:11.661903 15752 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:14:11.765763 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 206818us
I20260709 12:14:11.642513 15753 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
W20260709 12:14:11.719916 15871 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:14:11.768064 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.813735 11296 leader_election.cc:400] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:11.814141 11296 leader_election.cc:403] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:11.728022 15561 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.717262 15764 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:14:11.765590 15816 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:11.776367 15805 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.778082 15574 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:11.820469 15691 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:11.867434 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 229378us
I20260709 12:14:11.877655 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:11.817230 15902 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.879712 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:11.866446 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 794) took 1251 ms. Trace:
I20260709 12:14:11.868693 15154 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:11.820047 15561 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:11.776600 15789 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:11.818955 15752 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:11.867710 15855 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.885905 11324 rpcz_store.cc:276] 0709 12:14:10.615485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.626428 (+ 10943us) service_pool.cc:224] Handling call
0709 12:14:10.645132 (+ 18704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.645325 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:14:10.645671 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.522301 (+876630us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18003fd8a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:14:11.865680 (+343379us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:11.865806 (+   126us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:11.865888 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.866021 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.866315 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":118,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":12539,"spinlock_wait_cycles":6184704,"thread_start_us":342933,"threads_started":2}
W20260709 12:14:11.890136 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 123529us
I20260709 12:14:11.898939 15753 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:11.899389 15916 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:14:11.899519 15753 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:11.899547 15927 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:11.883252 15751 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:11.899683 15691 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.900305 15753 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:11.879397 15805 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.917582 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.917436 11324 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:11.930226 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.934489 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:11.935482 11253 raft_consensus.cc:2410] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:14:11.936362 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.937062 11253 raft_consensus.cc:2368] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:11.937328 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:11.937932 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:11.938671 11933 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:11.941830 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.943473 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:11.941797 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.945129 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.946521 11254 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:14:11.947283 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 231130us
I20260709 12:14:11.952265 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:11.957544 15752 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.947794 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:11.963949 15898 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:11.910082 15691 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:11.971820 15691 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:11.903183 15789 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.975008 15789 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.948371 15574 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:11.978425 15923 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:14:11.892843 15902 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:11.984333 15902 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:11.990190 15917 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:14:11.996904 15805 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:11.951433 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:11.998864 11255 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:12.002305 11250 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:12.010258 15737 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:12.010805 15737 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:12.061283 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.061617 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 181979us
W20260709 12:14:12.062043 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.063351 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.063987 11173 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:12.033751 15561 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.066111 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.068050 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.068706 11179 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:14:12.068763 15870 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.076165 15561 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:12.082391 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.083894 15938 raft_consensus.cc:993] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:14:12.097992 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.123456 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:12.124306 11256 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:12.124549 11933 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:12.084836 15888 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:14:12.125526 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:12.126363 11253 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:12.126669 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:12.127029 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.127265 11930 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.086800 15900 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:12.128607 15574 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.129084 15574 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.131130 15574 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:12.094288 11323 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:12.131608 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:12.132220 11935 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.090000 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:12.132503 11296 connection.cc:256] RPC call timeout handler was delayed by -1.9544147599999633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 12:14:12.132925 11296 connection.cc:256] RPC call timeout handler was delayed by -1.94001475999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 12:14:12.132997 11174 raft_consensus.cc:2368] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
W20260709 12:14:12.133262 11296 connection.cc:256] RPC call timeout handler was delayed by -1.6295147599998927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
I20260709 12:14:12.098490 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.103082 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.095697 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.090453 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.096534 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:12.133623 11296 connection.cc:256] RPC call timeout handler was delayed by -1.5476147599999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
W20260709 12:14:12.136353 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4513147600000593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260709 12:14:12.136687 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4252147600000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
W20260709 12:14:12.136971 11296 connection.cc:256] RPC call timeout handler was delayed by -1.387414759999956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
W20260709 12:14:12.137362 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3649147599999196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 12:14:12.137967 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:12.135212 11179 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:12.141486 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.142203 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2418) took 1227 ms. Trace:
I20260709 12:14:12.144348 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.145237 11174 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:12.146042 11933 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
W20260709 12:14:12.152956 15920 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.265s	user 0.001s	sys 0.002s
I20260709 12:14:12.160722 11174 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:14:12.161666 11935 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:12.090680 15751 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:12.134151 15752 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:12.164979 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.167126 15817 raft_consensus.cc:1081] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:14:12.141513 15574 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:12.176235 11256 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:12.174095 11326 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:12.167896 15888 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:12.087651 15816 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.180451 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.181124 11178 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:12.181572 11174 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:14:12.142854 11329 rpcz_store.cc:276] 0709 12:14:10.915169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.919090 (+  3921us) service_pool.cc:224] Handling call
0709 12:14:10.942489 (+ 23399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:10.957938 (+ 15449us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:10.958639 (+   701us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.776993 (+818354us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.006866 (+229873us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:14:12.059409 (+ 52543us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:12.086305 (+ 26896us) log.cc:844] Serialized 52 byte log entry
0709 12:14:12.086476 (+   171us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:12.086513 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.086587 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.141901 (+ 55314us) raft_consensus.cc:1685] finished
0709 12:14:12.141934 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.142147 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38842,"prepare.queue_time_us":2094,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":406591,"raft.run_cpu_time_us":1370,"raft.run_wall_time_us":157872,"spinlock_wait_cycles":5467008,"thread_start_us":1099407,"threads_started":3,"wal-append.queue_time_us":767840}
I20260709 12:14:12.184652 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:12.187585 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.097390 15753 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.189581 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.183540 15822 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:12.191910 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.192459 15822 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.194809 15822 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:12.195406 15807 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:12.195880 15807 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:12.196516 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.197304 11255 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:12.197760 15817 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:12.198266 15817 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.198680 15817 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:12.198985 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.201864 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.202519 11250 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.206511 15886 raft_consensus.cc:993] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:14:12.207002 15886 raft_consensus.cc:1081] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:14:12.177374 15752 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.191166 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:12.212903 11173 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:12.214293 11935 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:12.196874 15807 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.216014 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:12.219555 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 81669.9us
I20260709 12:14:12.220367 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:12.222415 11256 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:14:12.224596 11250 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:12.217924 15807 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:12.230084 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.230815 11255 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:12.231792 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.232937 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.234050 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.235368 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.236114 11252 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 9: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:12.235891 15737 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.237973 11178 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:14:12.233518 11255 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:12.239670 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:12.240734 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.241160 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.241580 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:12.244088 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 158850us
I20260709 12:14:12.244961 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.243158 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.246150 11257 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:12.246348 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.249874 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:12.240638 15751 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:12.253515 11331 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:12.241902 15631 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:12.258054 11250 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:12.259840 11932 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:12.243678 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:12.247890 15862 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:12.265794 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:12.258071 15865 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:12.268350 15938 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:12.273415 11323 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:14:12.277128 11323 pending_rounds.cc:85] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:14:12.277442 11323 pending_rounds.cc:107] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:14:12.261124 15751 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:12.278522 15817 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.278935 15751 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.280865 15817 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:12.276655 15862 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:12.261528 15786 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.283604 15862 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:12.285141 11181 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:12.264297 15902 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:12.281693 11323 raft_consensus.cc:2889] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:12.282030 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:12.283783 15786 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.288028 15816 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:12.289022 15753 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:12.289400 15786 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:12.286098 15752 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:12.290138 11296 leader_election.cc:400] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:12.291497 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:12.294873 15938 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:12.299885 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:12.306576 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.307215 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.313899 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.314658 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.285805 15862 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.315326 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.319377 11256 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:12.316201 11930 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:12.317127 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1871) took 1882 ms. Trace:
I20260709 12:14:12.313942 15816 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:12.330900 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.329588 11323 rpcz_store.cc:276] 0709 12:14:10.434335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.484704 (+ 50369us) service_pool.cc:224] Handling call
0709 12:14:10.484977 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.093698 (+1608721us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5800734fd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66480aa 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66eeb49 00007f3ec66ef416 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4
0709 12:14:12.316757 (+223059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.317036 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3232106752}
I20260709 12:14:12.328967 15816 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:12.315443 15896 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:12.305814 15902 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:12.352093 15938 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.352984 15737 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:12.338979 15863 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.354043 15863 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:12.355422 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.356251 15863 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:12.341706 15816 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.358284 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:12.357192 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:12.358973 11935 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:12.361871 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:12.362542 11927 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:12.364682 15937 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:12.366170 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.366981 11250 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:12.370844 11251 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:12.299597 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:12.375147 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 75619.5us
I20260709 12:14:12.376479 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.377125 11257 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:14:12.377861 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.376437 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:12.378819 11930 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:12.379884 11256 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:12.376036 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:12.384402 11250 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:12.386073 11296 leader_election.cc:400] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:12.386564 11296 leader_election.cc:403] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:12.365299 15937 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.391723 15937 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:12.392519 15902 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.379292 15631 raft_consensus.cc:2764] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:14:12.394256 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.395507 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:12.396697 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:12.397703 11179 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:12.397814 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.398561 11251 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:12.410045 15574 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:12.410616 15574 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.410996 15574 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:12.361372 15863 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.414125 15863 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.416510 15863 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:12.417482 15763 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:12.418419 15777 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.417269 15834 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:12.419173 11256 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:12.423051 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:12.425745 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.438601 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.442186 15949 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.442756 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.443905 11179 raft_consensus.cc:2410] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:14:12.440784 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54945us
I20260709 12:14:12.445036 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:12.445873 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.445986 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:12.446900 11179 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:12.447405 15900 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:12.448508 15926 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:12.448638 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:12.449412 15900 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:12.450464 15926 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:12.450627 15900 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:12.451071 15926 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:12.451552 15926 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:12.452236 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:12.452004 15900 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.452994 15647 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:12.453552 15647 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:12.454248 15900 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.454653 15647 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.450843 15940 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:12.458050 15940 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:12.458392 15940 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:12.458699 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:12.459051 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.459751 15931 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:12.459177 15940 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.442790 15949 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.460356 15931 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:12.460749 15940 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:12.461364 15931 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.462116 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.464538 15949 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:12.466259 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:12.466497 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 189156us
I20260709 12:14:12.480628 11296 leader_election.cc:400] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:12.482182 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.488938 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:12.443138 15834 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:12.508862 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.512979 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.491473 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.477649 15631 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:12.511073 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:12.541824 11179 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:12.478638 15863 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:14:12.546123 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.547305 11179 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:12.489939 15946 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.552656 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.557490 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:12.490259 15935 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:12.443027 15777 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.561694 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:12.544628 15631 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.566126 15946 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:12.572523 15919 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.755s	user 0.003s	sys 0.000s
I20260709 12:14:12.573676 15906 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:12.574457 11182 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 12:14:12.574831 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.579569 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:12.607131 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.580351 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.581957 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.582657 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.582705 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:12.609896 11930 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:12.581460 11180 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 12:14:12.591265 11935 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:11.185091 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:12.610853 11927 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 10. Current term is 12. Ops: []
W20260709 12:14:12.593310 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 171312us
I20260709 12:14:12.539842 15834 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:12.510767 15949 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:14:12.558925 15753 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:12.616184 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.629531 11179 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:14:12.617731 11178 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:12.618148 11926 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:12.597015 15631 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.633244 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.641796 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.642550 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:12.642765 11178 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:12.632305 15779 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.679929 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.709190 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.603781 15906 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:12.711249 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.711862 11178 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:12.550395 15926 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.753237 11176 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.740085 11182 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 15. Ops: []
I20260709 12:14:12.769399 11178 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:12.713083 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:12.889855 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 401002us
I20260709 12:14:12.890538 11180 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:12.741019 15868 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.783470 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.740659 15946 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:12.750681 15753 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:12.794279 11079 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:12.792493 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.908691 11253 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:12.766834 15647 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.923105 15647 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.893399 15763 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:12.929481 15763 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:12.930557 15957 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:14:12.931137 15936 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:12.795682 11178 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:12.796267 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.712592 15822 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:14:12.763187 15779 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:12.885283 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.785771 15764 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.781123 11173 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:12.890386 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:12.890808 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:12.744939 15937 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:12.891993 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.946703 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.948491 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:12.949221 11178 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:12.949775 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59044.7us
I20260709 12:14:12.951578 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:12.952689 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:12.892441 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.953821 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.893457 11176 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:12.955327 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.891078 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:12.956388 11256 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:12.901755 11182 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.903203 11174 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:12.794436 11255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:14:12.958271 11255 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:12.633150 15726 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.907593 15868 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:12.912231 11932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 714) took 1418 ms (client timeout 1837 ms). Trace:
I20260709 12:14:12.915092 15821 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:12.916529 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2412) took 2323 ms. Trace:
I20260709 12:14:12.961417 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:12.917435 15862 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.587s	user 0.006s	sys 0.000s
I20260709 12:14:13.000871 15862 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.003666 15862 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:12.992411 11178 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:13.000749 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.022984 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.033819 11322 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:12.636113 15935 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:12.959748 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.039024 11329 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:13.023456 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:12.963289 11176 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:12.965772 15789 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.966430 15647 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:12.967617 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.047266 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 434992us
I20260709 12:14:12.975059 15779 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:12.982108 15777 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:12.983433 15787 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:12.986094 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:13.051155 11174 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:13.053071 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:12.988694 15940 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.527s	user 0.004s	sys 0.005s
I20260709 12:14:12.990012 15906 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:12.953707 15937 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:12.960218 15944 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.992362 15726 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:12.993927 15943 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.814s	user 0.002s	sys 0.002s
I20260709 12:14:12.995651 15868 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:12.997077 11256 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:12.998550 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.106132 11251 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:14:13.000331 11175 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:12.999603 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.107884 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.108475 11325 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:13.108395 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.108716 11175 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:13.109095 11327 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:13.001302 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.023765 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:12.958666 11255 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:13.110891 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.111897 11323 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:13.022047 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:13.113840 15574 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.001s
I20260709 12:14:13.114284 15574 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.114969 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.115842 11331 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:13.116431 15574 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:13.021214 15631 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:12.960992 11932 rpcz_store.cc:269] 0709 12:14:11.494127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.494274 (+   147us) service_pool.cc:224] Handling call
0709 12:14:12.911928 (+1417654us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b5400555354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:12.912113 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704517504}
I20260709 12:14:13.048889 15967 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:13.013254 11180 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:13.118125 15967 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.118633 15631 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:13.120417 15967 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:12.958128 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:12.953619 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:13.122241 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 168669us
I20260709 12:14:13.122532 11179 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:13.123243 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:13.124385 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:13.005528 11182 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:13.126466 11182 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:13.127825 11182 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:13.128556 11325 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:13.128870 11182 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:13.129822 15967 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:13.130460 15868 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:13.131150 15868 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:13.131372 15863 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.002s
I20260709 12:14:13.131807 15863 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:13.131971 15967 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:13.006575 11077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 169) took 14732 ms (client timeout 15000 ms). Trace:
W20260709 12:14:13.009316 15952 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:14:13.005947 15959 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.094s	user 0.002s	sys 0.000s
I20260709 12:14:13.049463 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:13.018117 15953 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 12:14:13.054107 11329 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:13.015287 15949 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:13.018791 15816 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:13.018543 15964 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.019040 15931 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:13.019901 15926 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:13.048457 15737 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:13.090942 15787 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:13.100404 15966 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:12.950788 15764 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.093875 15906 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:12.961855 11326 rpcz_store.cc:276] 0709 12:14:10.593204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.128920 (+1535716us) service_pool.cc:224] Handling call
0709 12:14:12.129224 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.178209 (+ 48985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.916255 (+738046us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b54000c08d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:12.916458 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102711040}
I20260709 12:14:13.153512 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.132788 11175 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:14:13.133246 11328 raft_consensus.cc:2368] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10.
I20260709 12:14:13.110031 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.214432 11331 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:13.214466 11327 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:13.113174 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.223047 11323 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:14:13.135692 11182 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:13.136350 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:13.107220 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.224424 11324 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:13.007555 11079 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:13.136694 11174 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:13.067931 11322 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
W20260709 12:14:13.062750 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.129451 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.229997 11175 pending_rounds.cc:85] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 9
I20260709 12:14:13.230451 11175 pending_rounds.cc:107] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 11.10
W20260709 12:14:13.145231 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.028s	sys 0.030s
W20260709 12:14:13.148888 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.018s	sys 0.024s
I20260709 12:14:13.231072 11175 raft_consensus.cc:2889] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:13.231196 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.028s	sys 0.030s
W20260709 12:14:13.231331 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.018s	sys 0.024s
I20260709 12:14:13.209431 11329 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:13.230309 11325 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:13.143867 11180 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:14:13.232846 11180 pending_rounds.cc:85] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 10
I20260709 12:14:13.233098 11180 pending_rounds.cc:107] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:14:13.233330 11180 raft_consensus.cc:2889] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:13.234575 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 98304.9us
I20260709 12:14:13.149025 15949 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:13.016474 15753 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.235446 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:13.255849 11330 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:13.053349 15789 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.274446 15929 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:13.275758 15929 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:13.256987 15777 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.276819 15777 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.279320 15777 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.279484 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:13.279944 15777 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:13.280831 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:13.282604 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:13.284077 11079 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 7 to 9, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 9 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:13.284971 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:13.287992 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:13.290012 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:13.292074 15822 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:13.301126 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:13.276181 15929 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:13.303565 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:13.282539 15779 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:13.093482 15834 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.178138 15787 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:13.307608 11255 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.316499 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.333410 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:13.337412 11327 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:13.338966 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.318709 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:13.340907 11175 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 12:14:13.342927 11327 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:13.259336 15968 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:13.344255 15968 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.344730 15968 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:13.348055 11177 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:13.146662 15944 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.352303 15789 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:13.354432 15949 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.355751 15900 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:13.355868 15949 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:13.320854 15631 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:14:13.358916 15631 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:13.359369 15631 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:13.360368 15631 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.280889 15777 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:13.365732 15777 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:13.366690 15777 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:13.367496 15777 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.368116 15777 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.370922 15787 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.372506 15787 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:13.373888 15967 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.374449 15929 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.376399 15929 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:14:13.315055 15962 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:13.377800 15962 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:13.379767 15834 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.168471 15964 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.385756 15779 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:13.387272 15779 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.388135 15964 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:13.389128 15779 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:13.406852 11176 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:14:13.355044 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54050.5us
I20260709 12:14:13.424688 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:13.426179 11220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.058s	sys 0.048s
I20260709 12:14:13.395219 15764 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:13.405851 15752 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:13.396888 15937 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.314028 15966 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.402700 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.379761 15961 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:13.395929 11255 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:13.406643 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.256596 15737 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:13.422669 15945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.031s	sys 0.058s
I20260709 12:14:13.419096 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 611) took 1053 ms. Trace:
I20260709 12:14:13.262707 15574 raft_consensus.cc:2764] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:13.391266 11079 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:13.473079 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.473511 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 140220us
I20260709 12:14:13.483639 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:13.429289 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:13.429831 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.045s	sys 0.053s
W20260709 12:14:13.484982 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.045s	sys 0.053s
W20260709 12:14:13.485605 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 843120us
W20260709 12:14:13.430390 11220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.058s	sys 0.048s
W20260709 12:14:13.487061 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.487545 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.440135 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:13.443889 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.398689 15726 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:13.447810 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:13.491554 11079 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 8 to 11, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 11 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:13.462035 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:13.406177 15822 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:13.353125 15789 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:13.508045 15888 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.486402 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:13.475867 11254 rpcz_store.cc:276] 0709 12:14:12.365408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.365934 (+   526us) service_pool.cc:224] Handling call
0709 12:14:12.374913 (+  8979us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:12.375360 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:12.375388 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:12.375824 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:14:12.604243 (+228419us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:12.605839 (+  1596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.605941 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:13.017323 (+411382us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:13.331680 (+314357us) raft_consensus.cc:1685] finished
0709 12:14:13.331713 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.418989 (+ 87276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1328,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7782,"spinlock_wait_cycles":35309824,"thread_start_us":216610,"threads_started":1,"wal-append.queue_time_us":525088}
I20260709 12:14:13.505966 11322 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:14:13.504758 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.138198 11077 rpcz_store.cc:269] 0709 12:13:58.274480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.277851 (+  3371us) service_pool.cc:224] Handling call
0709 12:14:13.006275 (+14728424us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:01.546746 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:01.547453 (+   707us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:01.547497 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:14:01.547520 (+    23us) tablet.cc:662] Decoding operations
0709 12:14:01.773845 (+226325us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:01.781821 (+  7976us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:01.786196 (+  4375us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:14:02.023580 (+237384us) tablet.cc:701] Row locks acquired
0709 12:14:02.023606 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:14:02.067204 (+ 43598us) write_op.cc:270] Start()
0709 12:14:02.107499 (+ 40295us) write_op.cc:276] Timestamp: P: 1783599242062584 usec, L: 0
0709 12:14:02.107546 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 12:14:02.230730 (+123184us) log.cc:844] Serialized 4459 byte log entry
0709 12:14:03.223307 (+992577us) op_driver.cc:464] REPLICATION: finished
0709 12:14:03.440104 (+216797us) write_op.cc:301] APPLY: starting
0709 12:14:03.456510 (+ 16406us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:04.113088 (+656578us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:04.121829 (+  8741us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:04.705300 (+583471us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:04.734758 (+ 29458us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 12:14:04.739164 (+  4406us) write_op.cc:312] APPLY: finished
0709 12:14:04.869168 (+130004us) log.cc:844] Serialized 105 byte log entry
0709 12:14:04.987784 (+118616us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:04.990521 (+  2737us) write_op.cc:454] Released schema lock
0709 12:14:04.995792 (+  5271us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:05.065162 (+ 69370us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:05.069419 (+  4257us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":90373,"reported_tablets":32,"spinlock_wait_cycles":13056,"child_traces":[["op",{"apply.queue_time_us":115745,"apply.run_cpu_time_us":32932,"apply.run_wall_time_us":1648814,"cfile_cache_hit":42,"cfile_cache_hit_bytes":43974,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6221,"cfile_init":2,"lbm_read_time_us":84255,"lbm_reads_10-100_ms":3,"lbm_reads_lt_1ms":9,"num_ops":11,"prepare.queue_time_us":586832,"prepare.run_cpu_time_us":52203,"prepare.run_wall_time_us":1083718,"raft.queue_time_us":302271,"raft.run_cpu_time_us":11262,"raft.run_wall_time_us":185969,"replication_time_us":1055934,"thread_start_us":1261396,"threads_started":5,"wal-append.queue_time_us":578818,"wal-append.run_cpu_time_us":41671,"wal-append.run_wall_time_us":2518191}]]}
W20260709 12:14:13.517822 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 418050us
I20260709 12:14:13.522835 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:13.523674 11176 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:14:13.525506 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:13.490734 15945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.032s	sys 0.058s
I20260709 12:14:13.467595 15964 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:13.529047 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.469199 15764 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:13.533434 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.535185 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:13.536939 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50641.1us
I20260709 12:14:13.537401 11330 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:14:13.537958 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:13.537127 11329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:14:13.539623 11329 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:13.540084 11329 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:13.482424 15631 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:13.486410 15821 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.546511 11326 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:13.569844 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.546222 15764 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.546391 15822 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:13.517110 15900 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:13.546181 15789 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:13.579926 11079 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 9, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 9 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:13.582549 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:13.582648 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:13.527645 15751 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:13.592070 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:13.596432 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.598116 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:13.615259 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:13.616106 11249 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:13.603188 11182 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:13.603483 11250 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:13.620147 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:13.617192 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:13.621393 11249 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:13.612746 11255 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
W20260709 12:14:13.594556 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 56696us
W20260709 12:14:13.607487 11257 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.052s	sys 0.091s
W20260709 12:14:13.623811 11257 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.052s	sys 0.092s
I20260709 12:14:13.538730 15950 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.638767 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:13.631531 11330 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:13.631879 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:13.613685 11329 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.640821 11254 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:13.633836 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:13.634016 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:13.635336 11328 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:13.652952 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:13.653704 11248 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:13.660036 11182 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:13.661751 11249 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:13.662482 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:13.663508 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.661134 11256 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:13.596432 15786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.002s	sys 0.012s
I20260709 12:14:13.669076 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.669781 11330 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 9.
I20260709 12:14:13.612905 15789 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:13.593158 15965 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:13.671412 11900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.009s	sys 0.013s
W20260709 12:14:13.680761 11900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.009s	sys 0.013s
W20260709 12:14:13.681241 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 700044us
I20260709 12:14:13.532793 15888 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.635915 15946 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:13.673250 11255 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 12:14:13.674597 15902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.005s	sys 0.016s
W20260709 12:14:13.712689 15902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.005s	sys 0.016s
I20260709 12:14:13.675694 15822 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:13.673930 11079 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 7 to 10, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 10 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:13.714498 15822 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.714931 11079 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 9 to 11, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 11 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:13.591390 15821 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.719694 15821 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:13.683704 11328 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:14:13.671290 15786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.003s	sys 0.013s
I20260709 12:14:13.686393 11324 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:13.688176 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:13.661056 15950 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.709525 11248 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:13.691201 11326 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
W20260709 12:14:13.696383 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 73131.2us
I20260709 12:14:13.723276 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:13.662885 15868 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:13.723961 15950 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.724123 15868 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.725353 15888 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:13.726538 15868 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:13.717049 11079 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 5 to 9, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 9 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:13.727197 15737 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:13.680173 15752 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:13.663867 11252 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.016s	sys 0.035s
W20260709 12:14:13.728516 11252 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.017s	sys 0.036s
I20260709 12:14:13.728096 11079 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:13.728016 11326 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:13.632377 15900 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.709411 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:13.680410 15154 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.009s	sys 0.013s
I20260709 12:14:13.683290 15926 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.781106 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2439) took 1208 ms. Trace:
I20260709 12:14:13.742245 15974 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:14:13.732532 15964 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.740015 15946 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:13.750249 15786 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:13.743562 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:13.806607 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 63125.2us
I20260709 12:14:13.698719 15751 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:13.773722 15963 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:14:13.777892 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:13.820922 15968 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.781524 11177 rpcz_store.cc:276] 0709 12:14:12.573117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.580105 (+  6988us) service_pool.cc:224] Handling call
0709 12:14:13.780930 (+1200825us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400347bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:13.781043 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1701470976}
W20260709 12:14:13.784116 11173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.004s	sys 0.051s
I20260709 12:14:13.786201 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:13.747195 11079 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 5 to 8, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 8 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:14:13.836432 11173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.005s	sys 0.051s
W20260709 12:14:13.838433 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.839071 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.811053 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:13.840317 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:13.822899 15906 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.000s
I20260709 12:14:13.806022 15834 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:14:13.828614 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:13.780359 15777 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.779103 15966 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:13.728232 15752 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:13.830705 15763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.005s	sys 0.028s
W20260709 12:14:13.843225 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57137.6us
I20260709 12:14:13.887276 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:13.844515 11930 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:13.890280 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.830685 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52934.3us
I20260709 12:14:13.843231 11323 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:13.842382 15779 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:13.850885 11178 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:13.856781 11935 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:14:13.747469 15154 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.009s	sys 0.013s
W20260709 12:14:13.860673 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.007s	sys 0.032s
W20260709 12:14:13.906473 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.008s	sys 0.033s
I20260709 12:14:13.877166 11329 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.841678 11330 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:14:13.873518 15950 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:13.882201 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:13.900182 15779 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.964989 15779 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:13.967235 11249 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:14:13.969002 11249 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:14:13.873003 11079 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 9 to 10, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 10 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:13.903599 15949 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.000s
W20260709 12:14:13.846912 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.006s	sys 0.076s
I20260709 12:14:13.897051 11257 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:13.836006 15786 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:13.695349 15965 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.902256 11254 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:13.903290 11148 leader_election.cc:400] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 9 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10.
I20260709 12:14:13.971885 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:13.972062 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 68886.9us
I20260709 12:14:13.902128 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:13.862268 11250 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:14:13.799657 15926 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.908012 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2435) took 1336 ms. Trace:
I20260709 12:14:13.986591 11181 rpcz_store.cc:276] 0709 12:14:12.571752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.611646 (+ 39894us) service_pool.cc:224] Handling call
0709 12:14:13.907747 (+1296101us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54000d9654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:13.907928 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2883995904}
I20260709 12:14:13.826646 15821 raft_consensus.cc:2764] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:14:13.909082 11182 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:13.914548 11323 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:14:13.914304 15751 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:13.915091 15726 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:14:13.915724 15900 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.917007 15968 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:13.902750 15753 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:13.897056 15737 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.902590 15777 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:13.924808 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:13.925863 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:13.954734 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 50197.9us
I20260709 12:14:13.812029 15964 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.999791 11251 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 9.
I20260709 12:14:13.836401 15789 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:13.977807 15787 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:13.896377 15763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.005s	sys 0.029s
W20260709 12:14:13.958400 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 429414us
I20260709 12:14:13.992807 15764 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:13.970623 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.006s	sys 0.076s
I20260709 12:14:13.978652 11329 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:13.980111 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.985183 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.989899 11079 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 7 to 9, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 9 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:14.003557 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:13.962198 11330 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:13.960260 15946 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.015304 11255 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:14.018154 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:14.028548 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:14.006979 11250 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:14.001256 15737 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:14.009236 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 719286us
I20260709 12:14:14.013461 11329 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:14.013542 11330 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:14.061233 11251 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:14:14.038681 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.006176 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:14.065163 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.070174 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:14.071055 11330 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:14.073063 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:14.074874 15976 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:14:14.028622 11079 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 7 to 10, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 10 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:14.014022 15968 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.024082 15789 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:14.005872 15888 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:14.086023 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.061939 15906 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:14.082005 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:14.083498 11251 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:14.019408 15752 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:14.089843 15978 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:14:14.090118 11329 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:14.026670 15971 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:14:14.008598 15787 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.035463 15786 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.079490 11079 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:14.115239 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:14.093760 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.099764 15944 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:14.103199 15902 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:14.106050 15764 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:14.024845 15753 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 12:14:14.107797 11325 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.022s	sys 0.026s
W20260709 12:14:14.127321 11325 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.023s	sys 0.027s
I20260709 12:14:14.080854 15726 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.110877 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:14.112951 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.113409 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:14.137233 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:14.021237 15777 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:14.119918 15631 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.003s
I20260709 12:14:14.005327 15950 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:14.068189 15972 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:14:14.077369 15949 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:14:14.080945 15945 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:14:14.088610 15906 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.125389 11079 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 2 to 10, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 10 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:14.106721 15974 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:14.132733 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:14.154474 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.086508 15968 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:14.171029 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:14.212299 11329 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:14.213488 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:14.214406 11329 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:14.171555 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:14.172549 11322 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:14.124631 15834 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:14.185113 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:14.215438 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 43987.4us
I20260709 12:14:14.221199 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:14.217202 11251 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:14:14.158839 15816 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.158625 15900 raft_consensus.cc:2764] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:14:14.224259 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:14.224915 11250 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:14.225013 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2065) took 1423 ms. Trace:
W20260709 12:14:14.247340 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 177998us
I20260709 12:14:14.190970 15982 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.177241 15764 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:14.190904 15868 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:14.218256 15906 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:14.268267 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1887) took 1262 ms. Trace:
I20260709 12:14:14.258894 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:14.226959 15834 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.219266 15631 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:14.225605 15958 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:14.163502 15726 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.238374 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:14.289189 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50924.4us
I20260709 12:14:14.250725 15965 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:14.230767 15969 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:14.219859 15926 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:14.278312 15967 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:14.282569 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:14.279920 11327 rpcz_store.cc:276] 0709 12:14:13.006309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.420205 (+413896us) service_pool.cc:224] Handling call
0709 12:14:13.420704 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.421256 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.421303 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.422012 (+   709us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.014240 (+592228us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:14.014283 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.014348 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.267941 (+253593us) raft_consensus.cc:1685] finished
0709 12:14:14.267972 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.268183 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":151,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":42522,"thread_start_us":486529,"threads_started":1,"wal-append.queue_time_us":721491}
W20260709 12:14:14.299069 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 293845us
I20260709 12:14:14.336264 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:14.337553 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:14.332890 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 84508.7us
I20260709 12:14:14.274447 15737 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:14.274816 15777 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:14.336310 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:14.252974 15154 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:14:14.340189 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:14.286931 15834 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:14.345310 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:14.347412 11930 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:14.360021 11935 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:14.348549 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 346129us
I20260709 12:14:14.257988 15931 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:14.368346 15822 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:14.368093 15931 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.368762 15822 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:14.369280 15726 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:14.270373 15900 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:14:14.338963 11901 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:14.350641 11327 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:14.347674 15988 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:14.361605 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 79126.1us
I20260709 12:14:14.381079 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:14.276423 15888 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:14.292837 15966 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:14.384768 15966 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.281808 15982 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.384824 15888 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:14.387645 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:14.387244 15982 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:14.378361 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:14.234561 15816 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:14.378700 11935 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:14.256202 15961 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:14:14.387812 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:14.366437 15737 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:14.377571 15929 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.001s	user 0.014s	sys 0.000s
I20260709 12:14:14.386315 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1892) took 1378 ms. Trace:
I20260709 12:14:14.386479 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:14.396696 11255 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:14.384538 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.348456 15969 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:14.350376 15763 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:14.274850 15950 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.351109 15967 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:14.391662 11930 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:14.393512 11250 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:14.330046 15992 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:14.369769 15822 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:14.399825 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:14.377743 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 371629us
I20260709 12:14:14.411891 11926 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:14.404867 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:14.375768 15991 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:14.385289 15966 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:14.432520 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92446.4us
I20260709 12:14:14.416069 11255 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:14.292060 15958 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:14.398140 11325 rpcz_store.cc:276] 0709 12:14:13.007705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.528802 (+521097us) service_pool.cc:224] Handling call
0709 12:14:13.577931 (+ 49129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.578147 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:14.136007 (+557860us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.361510 (+225503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:14.361589 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:14.361694 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.361934 (+   240us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.386057 (+ 24123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12237,"prepare.queue_time_us":687500,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":12413,"spinlock_wait_cycles":12925312,"thread_start_us":762017,"threads_started":2}
I20260709 12:14:14.435555 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:14.394891 15868 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:14:14.446616 11926 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:14.395918 15737 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.450685 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:14.460392 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:14.417493 15991 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.418339 15931 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:14.583464 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.510989 11250 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
W20260709 12:14:14.583117 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 244229us
W20260709 12:14:14.564435 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.481477 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1891) took 1474 ms. Trace:
I20260709 12:14:14.593164 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:14.398885 15982 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:14:14.595046 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:14.610055 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.391028 15988 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.593915 11322 rpcz_store.cc:276] 0709 12:14:13.007439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.527939 (+520500us) service_pool.cc:224] Handling call
0709 12:14:13.528299 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.215475 (+687176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.481231 (+265756us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54000c58d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:14.481417 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481038592}
I20260709 12:14:14.640105 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:14.448997 15937 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:14.451437 15753 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.199s
I20260709 12:14:14.451834 15968 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:14.458853 15737 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:14.653149 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 192820us
I20260709 12:14:14.659569 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:14.395917 15764 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.592586 15986 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:14.667728 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.594817 15822 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:14.480361 16002 raft_consensus.cc:2764] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:14:14.672181 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 308052us
I20260709 12:14:14.456032 15974 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.197s
I20260709 12:14:14.594086 15929 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:14.633791 15970 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:14.416764 15888 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.472898 15945 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.192s
I20260709 12:14:14.596902 15984 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.649241 15937 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:14.598770 15991 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:14.651413 15949 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 12:14:14.458444 16005 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.304s
I20260709 12:14:14.651840 15786 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:14:14.792608 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.793761 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:14.802196 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2432) took 1701 ms. Trace:
I20260709 12:14:14.828624 11327 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:14.829129 11327 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:14.817473 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:14.820940 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:14.830176 11327 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.830958 11173 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:14.645363 16007 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.156s
I20260709 12:14:14.777889 15970 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:14.664024 15972 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.833856 11182 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:14.758325 16004 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:14.831055 11175 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:14.782922 16002 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.827744 11324 rpcz_store.cc:276] 0709 12:14:13.100938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.723092 (+622154us) service_pool.cc:224] Handling call
0709 12:14:13.723372 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.723566 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:14.134925 (+411359us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.638457 (+503532us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.801419 (+162962us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b1800034600. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:14.801665 (+   246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:14.801737 (+    72us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:14.801804 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.801904 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.802099 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178851,"prepare.queue_time_us":581038,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":242310,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":9813,"spinlock_wait_cycles":251219840,"thread_start_us":875102,"threads_started":3,"wal-append.queue_time_us":678008}
I20260709 12:14:14.838104 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:14.839429 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:14.804069 15937 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:14.859421 11177 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:14.814275 15992 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:14.845240 15981 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.490s	user 0.001s	sys 0.001s
W20260709 12:14:14.859189 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 408546us
W20260709 12:14:14.857859 15993 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.493s	user 0.001s	sys 0.000s
W20260709 12:14:14.827021 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 167547us
I20260709 12:14:14.889371 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:14.873571 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:14.831203 11327 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:14.826025 15982 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:14.838675 15986 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:14:14.882128 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 287148us
I20260709 12:14:14.892563 11175 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:14.886085 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:14.870442 15926 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:14:14.906302 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 266260us
I20260709 12:14:14.856784 15786 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.869132 15753 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:14.672474 15969 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.901614 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:14.907367 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:14.904755 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 136242us
I20260709 12:14:14.937662 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:14.917097 11327 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
W20260709 12:14:14.935648 11328 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.004s	sys 0.011s
I20260709 12:14:14.938395 11327 pending_rounds.cc:85] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
W20260709 12:14:14.938524 11328 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.004s	sys 0.011s
I20260709 12:14:14.938737 11327 pending_rounds.cc:107] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:14:14.939144 11327 raft_consensus.cc:2889] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:14.938645 11324 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:14.943634 11329 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.013s
W20260709 12:14:14.944028 11329 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.008s	sys 0.013s
I20260709 12:14:14.796890 15967 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:14.904783 15779 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.939s	user 0.010s	sys 0.002s
W20260709 12:14:14.903939 15987 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.511s	user 0.001s	sys 0.000s
I20260709 12:14:14.930603 15868 consensus_queue.cc:1243] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:14.950196 15786 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:14.937094 15961 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.813279 15984 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:14.987174 11178 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:14:15.003082 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95799.1us
W20260709 12:14:15.003101 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 101617us
I20260709 12:14:15.004835 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:14.857699 15972 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:14.994553 11182 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:15.008200 11182 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:14.919869 15816 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:14.917708 15982 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:15.036082 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.014261 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:15.016098 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.993845 15961 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.041290 15726 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.041884 15726 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.044006 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:14.897480 15974 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 12:14:14.923559 16005 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 12:14:15.049669 15902 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:14:15.053050 15949 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:15.053817 15726 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.017863 11182 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:15.011839 15994 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.546s	user 0.000s	sys 0.001s
I20260709 12:14:14.991797 15888 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:14.983501 16025 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:14.871170 16002 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.006424 15900 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:15.011332 15752 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:15.075817 16011 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:14:15.036296 15763 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:15.046006 15961 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.085503 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.002707 15779 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:15.058099 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 163429us
W20260709 12:14:15.082697 16012 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.412s	user 0.001s	sys 0.000s
I20260709 12:14:15.063848 11182 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:15.104571 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.094846 11254 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:15.056175 16026 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:15.096014 15752 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.106199 16026 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.002630 15753 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.093926 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.106467 15752 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.107216 11249 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:15.102473 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.108753 15752 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.109146 15753 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.061116 15982 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.096585 15968 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:15.098661 15988 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:15.101815 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 152157us
I20260709 12:14:15.080605 16025 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:15.059813 16009 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:14:15.081605 16018 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:15.100598 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:15.134434 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.111621 16026 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:15.117517 16008 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:15.119462 16021 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:14:15.121184 15984 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:15.092708 16036 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.126286 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:15.081351 15816 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:15.128253 11933 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:15.130496 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.100826 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.138763 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.139848 15787 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.056771 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.063165 16020 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.009181 15970 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:15.144229 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 206653us
I20260709 12:14:15.194931 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.060600 15949 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:14:15.128618 15988 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:15.171597 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.177582 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.197093 11934 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
W20260709 12:14:15.178580 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 52380.6us
I20260709 12:14:15.177876 11327 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:15.112164 15726 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.088447 15900 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:14.225590 11252 rpcz_store.cc:276] 0709 12:14:12.801407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.802141 (+   734us) service_pool.cc:224] Handling call
0709 12:14:12.898826 (+ 96685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.899121 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:13.728968 (+829847us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.224501 (+495533us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:14.224590 (+    89us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:14.224653 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.224744 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.224936 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8398,"prepare.queue_time_us":123134,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":96,"thread_start_us":1277012,"threads_started":2}
I20260709 12:14:15.198688 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:15.201239 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.200821 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.223675 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:15.223793 11324 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:15.223974 11327 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:15.056368 15992 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.129475 16025 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:15.225031 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.197707 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.172479 15946 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:15.172686 15974 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:15.232337 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1890) took 2225 ms. Trace:
W20260709 12:14:15.227700 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.244340 11324 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
W20260709 12:14:15.259733 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 203025us
I20260709 12:14:15.262506 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.257716 11331 rpcz_store.cc:276] 0709 12:14:13.007170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.527861 (+520691us) service_pool.cc:224] Handling call
0709 12:14:13.528299 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.578212 (+ 49913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.578241 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.578735 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.136770 (+558035us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:14.136829 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.265127 (+128298us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b54003241d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:14.265185 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.220730 (+955545us) raft_consensus.cc:1685] finished
0709 12:14:15.220758 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.232262 (+ 11504us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24137,"raft.queue_time_us":812,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":22820,"spinlock_wait_cycles":97240064,"thread_start_us":471006,"threads_started":1,"wal-append.queue_time_us":739287}
I20260709 12:14:15.144251 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.264452 11257 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 9.
I20260709 12:14:15.197022 15961 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:15.248442 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.225560 11327 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:14:15.267472 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69835.3us
W20260709 12:14:15.232264 16015 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 12:14:15.200664 15988 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:15.169703 11182 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:14:15.274019 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1913) took 1207 ms. Trace:
W20260709 12:14:15.268805 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43976.8us
I20260709 12:14:15.276666 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.221293 15786 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:15.233366 16018 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:15.266697 16021 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:14:15.281482 11330 rpcz_store.cc:276] 0709 12:14:14.066630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.092534 (+ 25904us) service_pool.cc:224] Handling call
0709 12:14:14.092969 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:14.094007 (+  1038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:14.094051 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:14.094611 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.392161 (+297550us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:14.392198 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.392274 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.273737 (+881463us) raft_consensus.cc:1685] finished
0709 12:14:15.273762 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.273948 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"thread_start_us":294994,"threads_started":1,"wal-append.queue_time_us":748776}
I20260709 12:14:15.265491 16027 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:15.304955 16017 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:15.199800 15986 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:14:15.256989 15946 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.137375 15964 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.265076 15868 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:14:15.278105 15998 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.576s	user 0.001s	sys 0.000s
I20260709 12:14:15.221526 15787 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:14:15.196238 15816 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.180058 15752 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 12:14:15.349583 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.283627 15926 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:15.361796 16022 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.245080 15726 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.201350 16020 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.228096 16008 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.316720 15974 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:14:15.378743 16046 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.403069 15964 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:14:15.388682 16030 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:15.403002 15816 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:15.443336 15154 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:15.443830 15154 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:15.444680 15154 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.446516 15972 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:15.409898 16021 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:14:15.386874 16033 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.426295 16017 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:14:15.454291 16033 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.455924 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2072) took 1594 ms. Trace:
I20260709 12:14:15.388635 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1901) took 1870 ms. Trace:
I20260709 12:14:15.456338 11253 rpcz_store.cc:276] 0709 12:14:13.861685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.883761 (+ 22076us) service_pool.cc:224] Handling call
0709 12:14:13.884116 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.912802 (+ 28686us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:14.389379 (+476577us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.764751 (+375372us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.771435 (+  6684us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:14.796245 (+ 24810us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.796396 (+   151us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:14.796433 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.796504 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.455678 (+659174us) raft_consensus.cc:1685] finished
0709 12:14:15.455699 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.455856 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208123,"prepare.queue_time_us":214699,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":2458,"raft.queue_time_us":316805,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":97436,"thread_start_us":605176,"threads_started":2,"wal-append.queue_time_us":432543}
I20260709 12:14:15.457827 11326 rpcz_store.cc:276] 0709 12:14:13.518149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.747560 (+229411us) service_pool.cc:224] Handling call
0709 12:14:13.766114 (+ 18554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.770790 (+  4676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.770824 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.771331 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.137348 (+366017us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:14.339673 (+202325us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.351272 (+ 11599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.351348 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.388343 (+1036995us) raft_consensus.cc:1685] finished
0709 12:14:15.388374 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.388559 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11311,"prepare.queue_time_us":128889,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":25386,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":5870,"thread_start_us":499126,"threads_started":2,"wal-append.queue_time_us":621648}
I20260709 12:14:15.389346 16032 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.389391 11331 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:15.460575 16032 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.390888 16041 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:15.391448 15988 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.361079 15946 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:15.394181 15787 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.348920 16005 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 12:14:15.401321 15906 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.111s	user 0.007s	sys 0.002s
I20260709 12:14:15.403594 16039 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:15.188114 16036 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.439488 15900 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:15.453792 15999 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.982s	user 0.002s	sys 0.004s
W20260709 12:14:15.387895 16000 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 12:14:15.436915 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2428) took 2337 ms. Trace:
I20260709 12:14:15.437777 16008 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.556600 11328 rpcz_store.cc:276] 0709 12:14:13.099835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.720719 (+620884us) service_pool.cc:224] Handling call
0709 12:14:13.721132 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.726134 (+  5002us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:14.137724 (+411590us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.939023 (+801299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:14.939083 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:14.939139 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.436629 (+497490us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b540026cad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:15.436692 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.436846 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":539941,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":394903,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":11381,"spinlock_wait_cycles":10927872,"thread_start_us":1212080,"threads_started":3,"wal-append.queue_time_us":1264141}
I20260709 12:14:15.457296 16033 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:15.504401 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 212633us
W20260709 12:14:15.397557 15737 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.001s
I20260709 12:14:15.433682 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:15.412631 15966 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.960s	user 0.008s	sys 0.002s
W20260709 12:14:15.435549 15834 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.059s	user 0.007s	sys 0.003s
W20260709 12:14:15.542917 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 162459us
I20260709 12:14:15.551713 15154 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:15.283866 16023 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:15.433444 15726 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:15.527956 15789 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.440s	user 0.005s	sys 0.009s
I20260709 12:14:15.462150 16020 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:15.515196 15949 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:15.350026 16018 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.539353 16004 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:14:15.528638 15944 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:15.439378 16030 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.560843 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:15.546372 15967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.002s	sys 0.011s
I20260709 12:14:15.498708 15786 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
W20260709 12:14:15.592145 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 402113us
W20260709 12:14:15.566232 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 256769us
I20260709 12:14:15.434010 16028 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 12:14:15.437668 16022 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.568655 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.551054 15779 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.438220 16035 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 12:14:15.559895 16002 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:15.553110 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.591339 15967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.003s	sys 0.011s
I20260709 12:14:15.544260 15787 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.551294 15906 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.438375 16046 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.623550 15787 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:15.607106 15969 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:15.607539 15753 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.628252 15753 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.630885 15753 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.631593 16022 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.599870 15965 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 12:14:15.599148 15821 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:14:15.638525 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.582399 16040 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:14:15.593492 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:15.596619 16020 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:15.640182 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.583370 15949 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.648247 15949 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:15.649205 15964 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.649813 15964 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:15.650369 15991 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.813s	user 0.007s	sys 0.003s
I20260709 12:14:15.650847 15991 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.652097 15964 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.651566 16026 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.653214 11327 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:15.653822 16054 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:15.654403 16054 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:15.654881 16054 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:15.659482 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51411.4us
W20260709 12:14:15.663367 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60250.2us
I20260709 12:14:15.664281 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:15.664786 16036 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.665423 16036 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:15.667848 16018 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:15.592128 16030 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:15.675204 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63889.2us
I20260709 12:14:15.679685 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.680460 11178 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:15.680817 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.681558 11175 raft_consensus.cc:2376] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12. Re-sending same reply.
I20260709 12:14:15.683019 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.680473 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:15.606376 15888 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.531s	user 0.002s	sys 0.009s
I20260709 12:14:15.691701 16026 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:15.545382 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 336207us
W20260709 12:14:15.588397 15822 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.811s	user 0.012s	sys 0.000s
I20260709 12:14:15.603683 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.696949 11326 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:15.604481 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.700599 15950 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.605283 11322 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:15.701233 15950 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.581434 11927 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:15.586881 15789 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.568328 15737 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.584992 16023 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:15.574967 15834 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.587157 15944 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:15.569998 15966 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.590132 16032 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:15.597440 15937 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.000s
I20260709 12:14:15.614857 16048 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 12:14:15.616806 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:15.722298 11176 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:15.722579 11325 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:15.618371 16045 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:14:15.622151 11328 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:15.723441 11328 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:15.622570 16002 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:15.692267 15961 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:15.693270 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.728709 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 756) took 1332 ms. Trace:
I20260709 12:14:15.729204 11935 rpcz_store.cc:276] 0709 12:14:14.395870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.692958 (+1297088us) service_pool.cc:224] Handling call
0709 12:14:15.728633 (+ 35675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:15.705530 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.595458 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:15.707075 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.710633 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:15.710891 11181 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:15.708302 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.732532 11175 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
W20260709 12:14:15.732904 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 137760us
I20260709 12:14:15.719381 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.728703 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.652982 15991 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:15.726779 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.777356 11256 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:15.717447 11182 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:15.625455 15906 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:15.625447 16046 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.719448 15789 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.608163 16027 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:15.746474 11932 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:15.734994 16025 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.742570 15944 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.749833 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.722183 16026 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:15.733901 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:15.741950 15834 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.741927 15966 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.742878 15949 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:15.776006 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.747736 15737 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:15.782712 11253 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:15.787631 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.804118 11252 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:15.749428 11328 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:15.794555 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60724.7us
I20260709 12:14:15.805410 11328 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:15.807004 11180 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:14:15.795580 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.731650 15937 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.749851 16032 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:15.800012 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.779903 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.821485 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.822237 11182 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:15.800968 11256 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:15.822453 11177 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 12:14:15.822734 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.799868 15949 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.750356 16023 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.826095 15949 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:15.828629 15937 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:15.797524 16026 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.802114 16027 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.831318 16026 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:15.801965 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:15.802232 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.832858 11176 raft_consensus.cc:2410] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:14:15.789804 15972 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.810557 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.833235 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.834100 11182 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:15.835335 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.821010 11250 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:15.832779 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.798689 15991 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:15.834115 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:15.844919 11328 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:14:15.845273 11253 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:15.842782 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.843577 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:15.846086 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:15.845319 11328 pending_rounds.cc:85] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:14:15.847990 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.837123 11929 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:15.850929 11182 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:15.802736 15944 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:15.848111 11328 pending_rounds.cc:107] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:14:15.862196 11328 raft_consensus.cc:2889] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:15.862391 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:15.861719 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.864037 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.864693 11177 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:15.845059 16032 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:15.871433 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.872320 16032 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:15.873803 15931 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:14:15.875095 16018 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:15.853386 15972 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.877684 16040 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:14:15.877921 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.878749 11173 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:15.860456 16028 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:15.854339 15967 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.879438 16028 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.879859 16028 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:15.879792 15967 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.879642 16033 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:15.880739 16033 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.881155 16033 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:15.853807 15777 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:15.882282 15777 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:15.878763 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:15.861548 15964 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:15.883123 15964 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:15.883327 16051 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:15.883555 15964 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:15.883882 16051 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:15.884445 16051 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:15.870110 15991 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:15.883733 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:15.860572 16002 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:14:15.885983 15992 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:15.885740 15991 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.886389 15992 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.886453 15764 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:15.886921 15992 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:15.886991 15764 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:15.888011 15764 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.888516 16025 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.853709 15737 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:15.871433 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.889667 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:15.857547 16001 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.768s	user 0.006s	sys 0.006s
I20260709 12:14:15.847649 16026 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:15.911659 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:15.913252 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.911314 15764 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:15.914220 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:15.921183 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:15.807710 15753 consensus_queue.cc:1175] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 12 status { last_received { term: 11 index: 7 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:14:15.927484 16006 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.095s	user 0.000s	sys 0.003s
I20260709 12:14:15.929400 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1906) took 2409 ms. Trace:
I20260709 12:14:15.911883 15900 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:14:15.929837 11323 rpcz_store.cc:276] 0709 12:14:13.519666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.999691 (+480025us) service_pool.cc:224] Handling call
0709 12:14:14.015160 (+ 15469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:14.015556 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:14.015602 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:14.016207 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.391720 (+375513us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:14.408693 (+ 16973us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.765142 (+356449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.765247 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.927095 (+1161848us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b240025b2d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363e09d 00007f3ec627010e 00007f3ec628788c 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221
0709 12:14:15.929075 (+  1980us) raft_consensus.cc:1685] finished
0709 12:14:15.929172 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.929332 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16657,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":398113,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":16151,"spinlock_wait_cycles":29180160,"thread_start_us":711386,"threads_started":2,"wal-append.queue_time_us":637162}
I20260709 12:14:15.931334 15967 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.915753 16026 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.935724 16026 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:15.915889 16018 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:15.936339 16018 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:15.936733 16018 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:15.940289 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.944464 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.906266 15946 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.950544 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.952018 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.953173 11933 raft_consensus.cc:2368] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:15.955555 16034 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:15.956139 16034 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.956532 16034 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:15.961480 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:15.962251 11182 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:15.962637 11296 leader_election.cc:400] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:15.962993 11296 leader_election.cc:403] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:15.963768 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:15.970501 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:15.971122 15964 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:15.983048 11179 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:14:15.883262 15777 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.987323 11182 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:15.992600 15964 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:15.968336 15991 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:14:16.006953 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 279416us
I20260709 12:14:15.964870 15949 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:15.999330 15968 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:16.025599 16030 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.028149 16030 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.030006 15946 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.032478 11931 raft_consensus.cc:2410] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:14:16.032385 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:16.033036 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.033073 16023 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.033797 16023 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.040493 16054 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.034273 15968 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:16.088264 15834 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.033743 15937 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:16.088872 15777 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:16.163578 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.164033 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.164317 11323 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:16.165192 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.089061 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:16.089816 16057 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:16.093604 15966 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.094122 11254 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:16.180994 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:16.094467 15958 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.012099 16027 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:16.094784 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.095013 15950 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.095868 15949 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.096237 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.203727 16054 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.203828 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:16.143306 15764 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.204560 11255 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:16.204809 15764 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.097126 15970 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:16.098093 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.207083 15764 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.162863 15964 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.207760 11256 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:14:16.208051 15964 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.095899 11250 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:16.099670 11177 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:16.097275 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.211377 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:16.101488 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.102240 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.103077 15961 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:16.105533 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.212970 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:16.101528 11175 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:16.115185 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.117447 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.215306 11325 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:16.119803 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.216146 11253 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:16.149025 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:16.156395 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.157063 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.150635 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.152980 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:16.175132 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:16.175179 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 86226.2us
I20260709 12:14:16.218422 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:16.219221 11178 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:16.187188 11254 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 10: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:16.188030 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.174747 15974 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:16.222573 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69722us
I20260709 12:14:16.228570 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.201036 16018 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.223646 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.174319 15944 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:16.222991 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.230427 11255 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:16.183683 15967 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:16.229377 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:16.239944 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82942.1us
I20260709 12:14:16.187202 15966 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.240089 11253 raft_consensus.cc:2368] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:16.232710 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.244304 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.232635 11324 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:16.246296 11323 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:14:16.238091 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:16.250504 11324 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:14:16.222226 16074 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:14:16.252341 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:16.104190 16028 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.162434 15834 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.222482 16020 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:16.256669 11934 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:16.257418 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:16.244550 15994 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:14:16.174041 15937 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.270924 11323 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:14:16.165690 15992 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.275202 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:16.254957 11325 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:16.244189 16041 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.213272 16027 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.287791 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2129) took 1472 ms. Trace:
I20260709 12:14:16.289179 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:16.294173 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.295008 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.298492 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.258488 15967 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:16.183279 16057 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.239692 15974 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.255113 15929 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:16.307834 15937 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.301299 16026 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:16.338570 16078 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:14:16.338349 16026 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.283221 16020 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.307003 11249 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:16.314739 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2448) took 1052 ms. Trace:
I20260709 12:14:16.341790 16020 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.342566 16026 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.232306 16049 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:16.254182 15764 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:16.351473 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.332773 11323 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:16.303313 16033 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.335013 11181 rpcz_store.cc:276] 0709 12:14:14.815688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.817132 (+  1444us) service_pool.cc:224] Handling call
0709 12:14:16.287340 (+1470208us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b5400364f54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:16.287719 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912443776}
I20260709 12:14:16.335906 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.342941 15929 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:16.332818 16041 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.340746 11330 rpcz_store.cc:276] 0709 12:14:15.261853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.353215 (+ 91362us) service_pool.cc:224] Handling call
0709 12:14:15.381542 (+ 28327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.598102 (+216560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:15.598145 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:15.598186 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:15.598217 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.314505 (+716288us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54000c6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:16.314557 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.314678 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497502080}
I20260709 12:14:16.274434 15961 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:16.343654 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.317057 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 827) took 1137 ms. Trace:
I20260709 12:14:16.367635 11325 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:16.254526 16024 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:14:16.250283 15944 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.355877 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.336355 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:16.345868 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.367367 11327 rpcz_store.cc:276] 0709 12:14:15.179556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.272784 (+ 93228us) service_pool.cc:224] Handling call
0709 12:14:15.275379 (+  2595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:15.653775 (+378396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.316701 (+662926us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54003237d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:16.316974 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868233600}
I20260709 12:14:16.377424 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:16.378254 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.379144 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:16.380015 11181 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:16.342933 15834 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.354720 15764 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.381263 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:16.358722 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:16.384864 11932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 757) took 1986 ms (client timeout 2598 ms). Trace:
W20260709 12:14:16.385370 11932 rpcz_store.cc:269] 0709 12:14:14.398681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.398800 (+   119us) service_pool.cc:224] Handling call
0709 12:14:16.384626 (+1985826us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400415fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:16.384789 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3012670336}
I20260709 12:14:16.386799 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.387617 11932 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:16.390013 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.391369 15929 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.382102 15937 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:16.394678 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:16.396085 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.396499 15967 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.398427 15789 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.406199 15764 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:16.407317 15988 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:16.410658 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.411180 16041 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.411384 11175 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:16.412024 11929 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:16.332044 15902 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:14:16.394840 15992 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.414788 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.415215 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:16.356101 15964 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.361375 15900 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.438813 11926 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:16.368655 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.441402 11324 raft_consensus.cc:2368] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:16.309662 16051 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.347616 16034 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.452965 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.453668 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.453995 11182 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:16.454288 11326 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:16.454906 16034 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.455590 11182 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:14:16.457657 16027 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:16.423969 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44909.5us
I20260709 12:14:16.459954 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:16.462641 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.463217 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:16.463850 11249 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:16.462664 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:16.468624 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:16.469038 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.469655 11252 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:14:16.423700 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65047.8us
I20260709 12:14:16.426294 15822 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:16.427023 15971 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:16.427227 15966 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.425745 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.430354 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.499568 11181 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:16.499756 11928 raft_consensus.cc:2410] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:16.430573 15986 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:16.432238 15991 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
I20260709 12:14:16.432651 15902 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:16.433569 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:16.437805 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2096) took 1343 ms. Trace:
W20260709 12:14:16.442236 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.551846 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.552567 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.553186 11257 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:16.443735 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.554062 11932 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:16.554147 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.556373 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.444149 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.417078 11174 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:16.558782 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.420121 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.451308 16051 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.452642 15974 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.564631 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.565464 11250 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:16.468564 11328 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:16.478539 11255 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:14:16.418640 15961 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:16.513669 15946 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:16.586582 15946 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.587116 15946 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:16.587715 15944 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.507797 16028 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.500752 15950 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:16.590714 15950 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:16.591325 15950 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:16.482533 16034 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 12:14:16.594105 16018 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.505576 16026 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:14:16.506182 16043 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:14:16.594096 11932 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:16.448064 15968 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.598654 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:16.600834 15968 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.598699 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.605659 15966 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:16.484026 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:16.606782 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 122832us
I20260709 12:14:16.607146 15986 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:16.607754 15986 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:16.608263 15986 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:16.608762 11932 raft_consensus.cc:2410] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:16.504755 11249 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:16.502221 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:16.505446 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.492663 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.438220 16023 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.518128 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:16.611071 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 108935us
I20260709 12:14:16.504426 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:16.550578 11176 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:16.611799 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.612452 11327 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:16.447727 16022 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:16.445775 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.560051 11929 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:16.562057 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.550494 11251 rpcz_store.cc:276] 0709 12:14:15.094633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.094791 (+   158us) service_pool.cc:224] Handling call
0709 12:14:15.095153 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:15.461130 (+365977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:15.461164 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:15.461832 (+   668us) log.cc:844] Serialized 52 byte log entry
0709 12:14:15.817831 (+355999us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:15.817874 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.405473 (+587599us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54000f8cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:16.405555 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.405606 (+    51us) raft_consensus.cc:1685] finished
0709 12:14:16.405618 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.437724 (+ 32106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2932,"raft.queue_time_us":37374,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":11972,"spinlock_wait_cycles":840654208,"thread_start_us":355529,"threads_started":1,"wal-append.queue_time_us":479304}
I20260709 12:14:16.619217 11324 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:16.489430 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.570717 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.571457 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.562907 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.620095 11928 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:16.578078 15972 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.581720 15726 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:16.581375 11177 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:14:16.584174 16051 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.635327 16051 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.626350 16026 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:16.447456 15900 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.639266 15900 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.639904 11330 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:16.507390 16056 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:14:16.605005 11322 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:16.609483 16020 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:16.567204 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.656008 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.656193 11926 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:16.656884 11330 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:16.613351 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.614233 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:16.546376 11326 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:16.572842 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.662771 11934 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:16.620263 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.664727 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:16.567716 15763 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.620311 11254 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:16.623160 11251 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:14:16.623788 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.671197 11932 raft_consensus.cc:2368] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12.
I20260709 12:14:16.672375 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.674036 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.627334 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.674970 11929 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:16.629722 15816 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:16.631662 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:16.618449 15968 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.558085 15902 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:16.668601 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.675576 11255 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 10.
I20260709 12:14:16.683133 11257 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:14:16.616950 16041 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:16.683689 11257 pending_rounds.cc:85] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 10
I20260709 12:14:16.684046 11257 pending_rounds.cc:107] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:14:16.683979 16041 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.684564 11257 raft_consensus.cc:2889] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:16.684871 16041 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:16.677593 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46140.7us
I20260709 12:14:16.686916 15764 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.687765 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.688603 11933 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:16.606488 15966 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.680441 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:16.696959 11926 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:16.689452 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.698293 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:16.699038 11932 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15. Re-sending same reply.
I20260709 12:14:16.679701 15763 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.700572 11932 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:16.701390 11256 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:16.705884 15789 raft_consensus.cc:2764] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:16.709275 15789 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:16.658135 16051 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:16.729218 11330 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:16.713331 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.696758 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.734624 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:16.735808 11926 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:16.665859 16020 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.724205 16057 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:16.749567 11927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 845) took 1648 ms (client timeout 1944 ms). Trace:
I20260709 12:14:16.753451 11176 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:16.757830 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.769580 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:16.770145 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:16.770767 11325 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:16.771750 11327 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:16.772631 11322 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:16.620961 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 131594us
I20260709 12:14:16.774235 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:16.720876 11256 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:16.768512 11927 rpcz_store.cc:269] 0709 12:14:15.101142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.102214 (+  1072us) service_pool.cc:224] Handling call
0709 12:14:16.749357 (+1647143us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54000429d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:16.749499 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087317760}
I20260709 12:14:16.736711 16039 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:16.717061 15968 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.832253 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.738978 16056 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:16.834143 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 413946us
W20260709 12:14:16.838070 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 331844us
I20260709 12:14:16.854022 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.811765 16030 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:16.857028 16004 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:16.831990 16091 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:16.859772 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.869093 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:16.881474 11296 leader_election.cc:400] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12.
W20260709 12:14:16.883773 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.885586 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.886446 11257 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:16.890650 11927 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:16.891116 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:16.865768 15764 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:16.858074 16051 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:16.858647 16030 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.859963 16091 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.910051 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.911213 11176 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:16.800050 15902 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.920009 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2472) took 1176 ms. Trace:
I20260709 12:14:16.920507 11173 rpcz_store.cc:276] 0709 12:14:15.743948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.305509 (+561561us) service_pool.cc:224] Handling call
0709 12:14:16.306871 (+  1362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.307260 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.307290 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.307776 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:14:16.670431 (+362655us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:16.724614 (+ 54183us) log.cc:844] Serialized 14 byte log entry
0709 12:14:16.724994 (+   380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.725110 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.919744 (+194634us) raft_consensus.cc:1685] finished
0709 12:14:16.919768 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.919937 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3527,"prepare.queue_time_us":53864,"prepare.run_wall_time_us":53,"raft.queue_time_us":3387,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":18809,"spinlock_wait_cycles":2199936,"thread_start_us":362236,"threads_started":1,"wal-append.queue_time_us":590416}
I20260709 12:14:16.922971 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:16.923662 11173 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:16.925119 11173 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:14:16.901432 16022 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:16.927850 16022 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:16.928894 16002 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:16.908943 16030 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:16.931418 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2137) took 1121 ms. Trace:
I20260709 12:14:16.932574 15968 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.929155 16022 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:16.933902 16096 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:14:16.934234 16056 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:16.934827 16056 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:16.936614 16049 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 12:14:16.937111 16049 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:16.938027 11250 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:16.931923 11179 rpcz_store.cc:276] 0709 12:14:15.810085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.312405 (+502320us) service_pool.cc:224] Handling call
0709 12:14:16.312974 (+   569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.333188 (+ 20214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.333218 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.333736 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 12:14:16.734119 (+400383us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:16.734173 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.734243 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.931117 (+196874us) raft_consensus.cc:1685] finished
0709 12:14:16.931148 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.931340 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19749,"raft.queue_time_us":21404,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":363254,"threads_started":1,"wal-append.queue_time_us":501953}
I20260709 12:14:16.946796 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:16.947705 11249 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:14:16.929404 16002 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:16.953402 16002 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.955364 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:16.956032 11176 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:16.954612 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:16.957104 15961 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:16.957624 15961 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.957996 15937 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:16.958518 15937 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.959887 15961 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:16.960497 15937 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.961055 15937 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:16.954674 16023 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.962154 16023 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:16.962864 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 403972us
I20260709 12:14:16.963917 11932 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:16.964016 11223 leader_election.cc:400] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:16.964442 11223 leader_election.cc:403] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:16.965116 16023 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.965780 15992 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.966323 15992 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:16.967432 15902 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:16.968916 15992 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:16.970245 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:16.914985 16091 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:16.973860 15946 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:16.975067 11181 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:14:16.984683 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:16.964018 11249 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:16.998198 11176 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:16.956187 16002 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:16.999429 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:16.999634 11929 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:16.961405 15937 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.006120 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:16.998503 15961 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:17.010214 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:16.998922 16005 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:17.052297 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46294us
I20260709 12:14:17.074309 16020 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.011459 16040 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:17.013291 15937 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:17.075744 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.008026 16019 raft_consensus.cc:2764] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:17.058452 11322 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:17.059577 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:17.056640 15988 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.002s
I20260709 12:14:17.061146 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.078262 11935 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:17.061870 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.082564 16028 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:17.082293 16041 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.053820 15946 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.084192 11934 raft_consensus.cc:2368] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:17.062436 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.062959 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.063267 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.063803 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.010810 15777 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:17.067886 15970 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:17.015347 16049 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:17.073954 16005 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:17.057407 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47285.1us
W20260709 12:14:17.089126 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.090529 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.098088 11253 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:17.098346 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:17.091200 11178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.006s	sys 0.028s
W20260709 12:14:17.099399 11178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.006s	sys 0.028s
I20260709 12:14:17.086241 11180 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:17.100185 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2471) took 1356 ms. Trace:
I20260709 12:14:17.098237 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.055850 16023 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.101207 11251 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
W20260709 12:14:17.105283 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 270021us
I20260709 12:14:17.091113 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:17.108155 11179 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:17.107949 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.109333 11177 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:17.110678 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.099080 15777 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.101585 16020 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:17.112845 15982 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:17.113047 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.159054 11257 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:17.141590 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50629.9us
I20260709 12:14:17.142331 11181 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:17.100662 11178 rpcz_store.cc:276] 0709 12:14:15.743730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.250108 (+506378us) service_pool.cc:224] Handling call
0709 12:14:16.270354 (+ 20246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.270708 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.270735 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.271210 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:14:16.486048 (+214838us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:16.486381 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.099841 (+613460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.099923 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.099982 (+    59us) raft_consensus.cc:1685] finished
0709 12:14:17.099996 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.100130 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21482,"prepare.queue_time_us":13995,"prepare.run_cpu_time_us":5551,"prepare.run_wall_time_us":25301,"raft.queue_time_us":116549,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":32712,"thread_start_us":807497,"threads_started":2,"wal-append.queue_time_us":376656}
I20260709 12:14:17.116252 15970 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:17.120479 15967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.007s	sys 0.030s
I20260709 12:14:17.114914 16091 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:17.144788 16050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.009s	sys 0.023s
I20260709 12:14:17.141746 15726 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.101044 16056 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:17.155972 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 296270us
I20260709 12:14:17.091768 15988 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.169690 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:17.207938 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:17.209669 16091 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:17.187872 15967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.007s	sys 0.034s
I20260709 12:14:17.154229 15992 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:17.191569 11249 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:17.114202 16049 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:17.150578 15834 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.169221 16027 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:17.119042 16023 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.155651 15929 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:17.155385 15946 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.157194 15777 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:17.168272 11222 leader_election.cc:400] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12.
W20260709 12:14:17.231568 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 68655.6us
I20260709 12:14:17.206350 15726 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:17.212932 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 199891us
I20260709 12:14:17.214124 15816 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:17.215340 15988 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.217549 16034 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.158836 15982 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:17.221920 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2442) took 3040 ms. Trace:
I20260709 12:14:17.118366 15937 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:17.228787 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 108451us
W20260709 12:14:17.196089 16050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.009s	sys 0.023s
I20260709 12:14:17.243525 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.246415 11248 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:17.232632 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:17.216869 16032 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.212797 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:17.236295 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:17.236049 15950 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.000s
W20260709 12:14:17.238377 15986 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.001s
I20260709 12:14:17.238263 15816 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:17.225493 16027 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.243564 11329 rpcz_store.cc:276] 0709 12:14:14.172370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.224732 (+ 52362us) service_pool.cc:224] Handling call
0709 12:14:14.225025 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.944356 (+719331us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:15.088862 (+144506us) log.cc:844] Serialized 14 byte log entry
0709 12:14:16.093211 (+1004349us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18003f4720. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:14:16.624687 (+531476us) log.cc:844] Serialized 14 byte log entry
0709 12:14:16.625834 (+  1147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:16.625891 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:16.625966 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.652668 (+ 26702us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b18003f4720. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:17.212210 (+559542us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54001c5cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:17.212261 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.212387 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8857,"prepare.queue_time_us":228628,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":98,"raft.queue_time_us":470914,"raft.run_cpu_time_us":439,"raft.run_wall_time_us":16201,"spinlock_wait_cycles":521479552,"thread_start_us":1180994,"threads_started":4,"wal-append.queue_time_us":1173967}
I20260709 12:14:17.253254 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:17.216565 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 151532us
I20260709 12:14:17.256758 16091 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:14:17.254585 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.256600 11180 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:14:17.260659 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.235947 16041 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:17.267707 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:17.227151 15992 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.253434 15929 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:17.241549 11257 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:17.286669 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.289192 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:17.260565 15950 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.310314 15950 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.310652 16005 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.255105 15970 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.287391 15971 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:14:17.292507 16032 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.314224 11175 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:17.288859 16049 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.288291 16091 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:17.319754 16091 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:17.323518 16111 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:14:17.293159 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49790.3us
I20260709 12:14:17.325229 15992 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.325311 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:17.323549 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55950.2us
I20260709 12:14:17.294689 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:17.326563 16106 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:14:17.327282 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:17.327821 15929 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:17.328289 15974 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:17.328581 16107 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:17.249866 16103 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:14:17.328086 15786 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:17.323638 15822 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:17.261027 15986 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.325877 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:17.290710 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:17.339183 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48400.6us
I20260709 12:14:17.285209 16027 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:17.298537 15972 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:17.299309 16020 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:17.329416 15929 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:17.344269 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.299726 15984 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:17.340857 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:17.348739 16110 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:14:17.350311 16116 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:14:17.352310 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.338207 15971 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:17.357686 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.358727 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.336071 15786 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:17.360256 11253 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 12:14:17.339509 15992 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.363906 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:17.343510 15972 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:17.365492 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.366382 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.377120 11176 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:17.339335 15822 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:17.330142 16120 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
W20260709 12:14:17.367069 16099 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:14:17.378985 11182 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
W20260709 12:14:17.368407 16004 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 12:14:17.381590 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.382707 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.383512 11182 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:17.363251 15786 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:17.366252 16032 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:17.366549 16020 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.298313 15982 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.389041 16023 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.390108 15777 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.391103 16034 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.391678 16056 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.395138 16049 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:17.364670 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 266403us
I20260709 12:14:17.403486 15968 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:17.426877 15992 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:17.385145 16004 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.409520 16120 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.367957 15984 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:17.376718 15937 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.378700 15972 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:17.426901 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:17.431937 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 104705us
I20260709 12:14:17.436496 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:17.438596 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:17.442914 16030 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.006s
I20260709 12:14:17.444325 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:17.445017 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 117122us
I20260709 12:14:17.463835 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:17.445937 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1935) took 1301 ms. Trace:
I20260709 12:14:17.466430 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:17.447238 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.467401 11927 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:14:17.449010 16091 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.457345 15929 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:17.443360 15992 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.514142 15937 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:17.514358 15992 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:17.514678 15929 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.514844 15937 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.458981 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.517406 15937 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.460495 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:17.406594 11176 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:17.434484 16023 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:17.466373 11331 rpcz_store.cc:276] 0709 12:14:16.144846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.211272 (+ 66426us) service_pool.cc:224] Handling call
0709 12:14:16.211616 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.356682 (+145066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.356713 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.357195 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:14:16.809611 (+452416us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:16.809672 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.809758 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.445672 (+635914us) raft_consensus.cc:1685] finished
0709 12:14:17.445694 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.445852 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39760,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12668,"thread_start_us":581306,"threads_started":2,"wal-append.queue_time_us":847747}
I20260709 12:14:17.466912 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:17.470323 16032 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:17.472734 16033 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.475399 15986 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.442099 15982 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:17.477856 16030 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.432801 16020 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.440758 16034 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.470325 15777 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.489266 16132 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:14:17.488498 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.490885 16027 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.491403 16004 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.545885 16004 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.548048 16004 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.550316 15950 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:17.532475 16107 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.532927 16032 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:17.432026 16022 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:14:17.462621 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
W20260709 12:14:17.495662 16118 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:14:17.441864 16049 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.496788 15822 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.499926 11327 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:17.500392 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.504058 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:17.505889 11935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 764) took 1640 ms (client timeout 1873 ms). Trace:
I20260709 12:14:17.506237 15786 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.518575 15937 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:17.519196 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58770.8us
W20260709 12:14:17.572849 11935 rpcz_store.cc:269] 0709 12:14:15.865857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.865996 (+   139us) service_pool.cc:224] Handling call
0709 12:14:17.505703 (+1639707us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540041b754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:17.505811 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2677478144}
W20260709 12:14:17.520134 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 83768.1us
I20260709 12:14:17.523818 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.528081 11933 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:17.529238 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:17.530251 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:17.529515 11222 leader_election.cc:400] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:17.574174 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.575023 11929 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:17.577945 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:17.579257 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.581410 11330 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
W20260709 12:14:17.582367 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53045.1us
W20260709 12:14:17.531320 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64525.8us
I20260709 12:14:17.524683 15968 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:17.535341 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.545233 16027 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.462898 16120 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.556093 16132 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:17.556246 15950 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.524827 16023 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.559337 16034 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:17.560346 16020 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.523211 15789 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:14:17.561323 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 852) took 1611 ms. Trace:
I20260709 12:14:17.562141 16030 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.574156 11176 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:17.603499 11930 rpcz_store.cc:276] 0709 12:14:15.950424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.063747 (+113323us) service_pool.cc:224] Handling call
0709 12:14:17.507270 (+1443523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.507702 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:17.507746 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:17.507811 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:17.507860 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.507946 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.561136 (+ 53190us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540041b750. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:17.561259 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"spinlock_wait_cycles":3332045056}
I20260709 12:14:17.584842 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.587349 15968 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.552064 16133 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:17.617823 15968 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:17.617937 16133 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.618690 16133 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:17.574479 16032 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.623143 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.624392 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.595232 11929 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:17.597797 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:17.541679 15984 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.586022 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:17.591199 16048 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:17.602782 16026 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
W20260709 12:14:17.646355 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60371.7us
W20260709 12:14:17.647330 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49626.7us
I20260709 12:14:17.610214 16132 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.600270 16022 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:14:17.659204 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.613814 16130 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:17.610905 15950 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.604723 15763 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.612458 15786 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.612918 16023 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.613945 16091 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.614858 16120 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.601663 16131 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:14:17.615123 15726 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:17.615769 16129 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:14:17.591112 16107 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:14:17.616317 16049 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.586890 15937 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.600687 16027 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:14:17.683835 15726 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:17.688483 16109 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 12:14:17.632733 11176 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:17.651185 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:17.655215 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.691318 11930 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:17.693876 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.694484 11253 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:17.694794 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.695163 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.696213 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.656515 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.657903 15834 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:17.703256 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.704443 11253 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:17.658221 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:17.661759 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.660372 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.654808 16033 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:17.655017 16034 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:17.666994 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.659420 16028 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:17.663574 16130 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:17.655146 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:17.713267 11326 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:17.713554 11929 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:17.714567 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.714919 11256 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:17.715149 11935 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:17.689113 15968 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.706148 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.719327 11179 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:17.720220 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.720800 11179 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:17.722011 11929 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:17.710032 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:17.710418 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.716732 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.700881 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:17.722901 11257 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:17.723587 11179 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:17.724870 16033 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:17.725373 16033 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:17.725589 16137 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:14:17.725843 16033 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:17.725982 15834 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:17.726500 15834 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:17.727468 15834 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.729225 15834 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:17.729889 16026 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:17.718019 16028 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.734862 15816 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.718130 16130 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:17.680658 16048 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:14:17.737798 15789 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.738431 15789 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.719642 16034 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.711964 16120 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:14:17.707508 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47200.7us
I20260709 12:14:17.735495 16028 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:17.743091 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 150832us
I20260709 12:14:17.750414 11329 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:17.751149 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.751519 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.753810 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.753845 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:17.754688 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:17.755190 11329 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:17.756681 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.679807 15950 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:14:17.771688 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:17.768383 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 110219us
I20260709 12:14:17.772646 11330 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:17.773411 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:17.764591 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:17.775740 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:17.777123 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.778023 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2109) took 1215 ms. Trace:
I20260709 12:14:17.779688 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.779826 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.778579 11250 rpcz_store.cc:276] 0709 12:14:16.562309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.566250 (+  3941us) service_pool.cc:224] Handling call
0709 12:14:16.566592 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.943456 (+376864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.777709 (+834253us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b540010a754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:17.777942 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":853765376}
I20260709 12:14:17.784804 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.787554 15900 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:17.788106 15900 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:17.767207 16109 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:17.781931 15763 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:17.790988 15968 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:17.791932 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2477) took 1527 ms. Trace:
I20260709 12:14:17.763011 16027 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:17.793082 16027 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.792348 11174 rpcz_store.cc:276] 0709 12:14:16.264702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.570129 (+305427us) service_pool.cc:224] Handling call
0709 12:14:16.570559 (+   430us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:16.604726 (+ 34167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.604776 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:16.605755 (+   979us) log.cc:844] Serialized 78 byte log entry
0709 12:14:17.101512 (+495757us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:17.118701 (+ 17189us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.228282 (+109581us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.228487 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.228587 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.687822 (+459235us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:17.791697 (+103875us) raft_consensus.cc:1685] finished
0709 12:14:17.791721 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.791872 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146119,"prepare.queue_time_us":38014,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":1370,"raft.queue_time_us":158492,"raft.run_cpu_time_us":3058,"raft.run_wall_time_us":590685,"spinlock_wait_cycles":50565888,"thread_start_us":495231,"threads_started":1,"wal-append.queue_time_us":630293}
I20260709 12:14:17.795892 16027 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.810026 15763 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:17.810591 15763 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:17.810772 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:17.811532 16033 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:17.812667 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.813288 11250 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:17.810983 15763 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:17.813787 16023 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:17.814437 16023 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:17.815591 16023 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.819456 16033 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:17.821014 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:17.753214 15966 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:17.825524 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 61070us
I20260709 12:14:17.826700 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.766608 15816 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:17.837142 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.772320 15929 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:17.838814 11329 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:17.840919 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.842448 11324 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:17.848353 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.848459 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.849284 11248 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:17.797763 16103 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:17.855660 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.818148 16131 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
W20260709 12:14:17.862464 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.840049 16120 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:17.789265 15900 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.846242 15966 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:17.841840 16127 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:17.875206 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.882145 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:17.876518 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 223210us
I20260709 12:14:17.888790 11324 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:17.893572 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.894317 11323 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:17.894237 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.900995 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.850111 15929 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.880383 16034 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:17.907610 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.904348 11929 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [10.10-13.11]
I20260709 12:14:17.906363 11926 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:17.859611 16056 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:14:17.882145 15789 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.911989 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:17.916184 11249 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 15 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:14:17.861388 15972 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.863369 16103 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.913990 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.898764 16048 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:17.895666 16030 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:17.921129 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:17.893164 15966 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:17.904354 15900 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:17.927546 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.944720 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:17.945366 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.932981 15929 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:17.933667 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:17.950182 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.950341 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.934341 16027 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:17.940614 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:17.945194 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.930380 11927 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:17.947131 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.948297 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:17.969619 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.971277 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:17.948417 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92896.2us
I20260709 12:14:17.948933 11250 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:17.950996 11179 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:17.976353 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:17.977208 11179 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:17.952668 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:17.953264 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 290713us
I20260709 12:14:17.979517 11176 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:17.984849 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.985064 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.985920 11250 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:17.986896 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:17.987802 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.903214 16133 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:17.991273 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:17.992417 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:17.926461 16131 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.000044 11932 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:17.923333 16048 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:17.942202 15937 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:18.026494 11322 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:18.027727 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:18.028376 11329 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:18.001960 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 181024us
I20260709 12:14:18.030503 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:17.967267 16147 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:18.032382 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:17.961441 16030 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:18.032908 11173 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:14:18.040092 11179 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:17.992029 15929 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:17.995572 16027 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:18.031076 15777 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:18.075265 15937 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:18.082134 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:18.090417 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:18.090422 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59346.7us
I20260709 12:14:18.094547 16161 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:18.097525 16027 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:18.099045 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:18.095247 11147 connection.cc:256] RPC call timeout handler was delayed by -1.1381884689999424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
I20260709 12:14:18.107169 16056 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:18.085564 11329 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:18.160269 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:18.094326 16008 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.101727 15966 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.087219 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:18.160290 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:18.140091 11254 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:18.080862 11250 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:18.150688 11324 raft_consensus.cc:2368] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13.
I20260709 12:14:18.031374 16131 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:18.152208 11323 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:18.162158 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:18.162869 11179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01e75f61a948459a884630690e51c686"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:18.184785 11179 raft_consensus.cc:606] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:18.185308 11179 raft_consensus.cc:614] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:18.180122 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:18.180212 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 93158.6us
I20260709 12:14:18.187345 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:18.150611 15929 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:18.190795 11254 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:14:18.148275 16147 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:18.194290 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:18.214871 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:18.192960 15777 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:18.211344 15944 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:18.260924 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:18.197552 16056 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:18.191457 16162 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:18.264904 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:18.182301 16008 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:18.268776 11173 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:18.145197 16048 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.256058 15937 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:18.193272 15929 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.334705 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:18.270562 16056 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.222313 16024 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:18.275573 15753 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:18.329186 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:18.262068 16148 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:18.325886 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51945.6us
I20260709 12:14:18.344038 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:18.332654 15834 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 12:14:18.262550 16147 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:18.266045 16135 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:14:18.329741 15971 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:18.328543 15972 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.260375 15763 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.337368 15929 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:18.250942 16030 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:18.372365 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.372344 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 192686us
I20260709 12:14:18.345992 16056 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:18.343565 16048 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:18.350184 16149 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 12:14:18.343318 15937 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:18.251596 16027 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:18.345911 16024 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:18.373358 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:18.363502 11255 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:14:18.375645 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.597s (SENT) [suppressed 3 similar messages]
I20260709 12:14:18.358676 16133 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.360733 16147 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:18.386508 16147 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:18.380985 16152 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.276s	user 0.001s	sys 0.001s
I20260709 12:14:18.390208 16125 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:18.361889 16148 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.393330 16125 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:18.400844 16160 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:14:18.404024 16019 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:18.407477 16103 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:18.408715 16167 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:14:18.409536 16163 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:14:18.480710 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:18.383188 15822 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 12:14:18.410354 16165 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:14:18.410179 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:18.411113 16154 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:14:18.411028 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:18.411818 15937 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.413079 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 770) took 1124 ms. Trace:
I20260709 12:14:18.487346 11932 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:18.415504 11931 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: [11.9-12.10]
I20260709 12:14:18.416568 16008 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.421666 11148 leader_election.cc:400] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13.
I20260709 12:14:18.489502 11148 leader_election.cc:403] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:14:18.490219 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 68738.5us
I20260709 12:14:18.487413 11933 rpcz_store.cc:276] 0709 12:14:17.289161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.289279 (+   118us) service_pool.cc:224] Handling call
0709 12:14:18.412835 (+1123556us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400416c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:18.413001 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280161024}
W20260709 12:14:18.492223 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.385481 16024 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:18.429455 11324 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:14:18.481324 16156 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:14:18.386878 16162 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:18.483084 11326 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:18.495931 11322 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:18.481653 16140 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:18.498595 11932 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:18.371963 15971 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:18.497138 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:18.507491 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:18.514878 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:18.501613 15970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.123s	user 0.000s	sys 0.018s
I20260709 12:14:18.489074 16131 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:18.906148 15970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.530s	user 0.001s	sys 0.039s
W20260709 12:14:18.888026 15900 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.940s	user 0.010s	sys 0.004s
W20260709 12:14:18.906774 16048 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
W20260709 12:14:18.910652 16170 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.474s	user 0.001s	sys 0.007s
W20260709 12:14:18.912773 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 395351us
I20260709 12:14:18.914764 15906 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:18.901527 16174 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:14:18.916566 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 724) took 2515 ms. Trace:
I20260709 12:14:18.916985 11175 rpcz_store.cc:276] 0709 12:14:16.401178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.576478 (+175300us) service_pool.cc:224] Handling call
0709 12:14:16.576890 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.314944 (+738054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.916266 (+1601322us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b5400364554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:18.916470 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695055104}
I20260709 12:14:18.918782 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 731) took 1812 ms. Trace:
I20260709 12:14:18.919219 11180 rpcz_store.cc:276] 0709 12:14:17.106663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.106849 (+   186us) service_pool.cc:224] Handling call
0709 12:14:17.107362 (+   513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.288539 (+181177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.918467 (+1629928us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002701d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:18.918717 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341012864}
I20260709 12:14:18.922780 15763 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:18.504190 15822 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:18.889861 16140 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:18.480419 16019 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:18.892366 16028 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.123s	user 0.012s	sys 0.006s
I20260709 12:14:18.892616 16024 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:18.894079 15992 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.366s	user 0.012s	sys 0.016s
W20260709 12:14:18.894456 16130 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.158s	user 0.015s	sys 0.012s
I20260709 12:14:18.895423 16158 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:18.684932 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 307932us
W20260709 12:14:18.895906 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.896668 16056 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
W20260709 12:14:18.901140 16147 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.513s	user 0.004s	sys 0.005s
I20260709 12:14:18.520313 16125 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.499652 15752 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 12:14:18.923460 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 744) took 1457 ms. Trace:
I20260709 12:14:18.889225 16148 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:18.927678 15971 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.939556 11926 rpcz_store.cc:276] 0709 12:14:17.466387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.466537 (+   150us) service_pool.cc:224] Handling call
0709 12:14:17.469876 (+  3339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.910131 (+440255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.923219 (+1013088us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b540041aad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:18.923392 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003198848}
W20260709 12:14:18.944276 15929 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.568s	user 0.014s	sys 0.000s
I20260709 12:14:18.942337 16161 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:18.938501 16130 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:18.949612 15929 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.936043 16028 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.948735 16161 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.945955 16125 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:18.945205 16019 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:18.943399 16056 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.936869 15992 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:18.939721 16158 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:18.970923 16024 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:18.971333 15992 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:18.972932 16158 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.963171 16161 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:18.966611 16130 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.974668 16056 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.968844 15822 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13.
I20260709 12:14:18.969269 16028 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.938421 16147 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:18.976658 15929 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:18.976972 16147 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:18.901932 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 745) took 1062 ms. Trace:
I20260709 12:14:19.018051 11174 rpcz_store.cc:276] 0709 12:14:17.839277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.839506 (+   229us) service_pool.cc:224] Handling call
0709 12:14:17.839931 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.840185 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:17.840215 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:17.846734 (+  6519us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.037249 (+190515us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:18.037306 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.037412 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:18.901706 (+864294us) raft_consensus.cc:1685] finished
0709 12:14:18.901728 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.901868 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":148996,"threads_started":1,"wal-append.queue_time_us":284396}
I20260709 12:14:18.893316 11177 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:19.022516 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:19.029474 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.306s	user 0.027s	sys 0.031s
W20260709 12:14:19.030092 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.307s	user 0.029s	sys 0.031s
W20260709 12:14:19.032450 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 1.33151e+06us
I20260709 12:14:19.042366 15992 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:19.043944 15992 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.051568 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:19.056285 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 646) took 1303 ms (client timeout 1597 ms). Trace:
I20260709 12:14:19.059659 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:19.061048 11177 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:14:19.056998 11255 rpcz_store.cc:269] 0709 12:14:17.752448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.752662 (+   214us) service_pool.cc:224] Handling call
0709 12:14:19.055588 (+1302926us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400068e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:19.055918 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1399404800}
I20260709 12:14:19.065943 11180 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:19.067215 11180 raft_consensus.cc:606] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:19.067857 11180 raft_consensus.cc:614] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:19.075770 11932 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:19.082229 11932 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:19.084686 15992 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.092871 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 742) took 1627 ms. Trace:
I20260709 12:14:19.093696 11931 rpcz_store.cc:276] 0709 12:14:17.465100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.465279 (+   179us) service_pool.cc:224] Handling call
0709 12:14:18.415241 (+949962us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:18.489780 (+ 74539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.092212 (+602432us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b5400416c50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:19.092600 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183484672}
I20260709 12:14:19.096909 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2126) took 1141 ms. Trace:
I20260709 12:14:19.098035 15961 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:19.037400 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:19.099871 15961 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.105083 15961 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:19.114586 16171 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.480s	user 0.005s	sys 0.007s
W20260709 12:14:19.115513 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.117055 16181 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.224s	user 0.001s	sys 0.001s
W20260709 12:14:19.119124 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 82005.2us
I20260709 12:14:19.119823 15992 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:19.122227 16161 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.126637 16030 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:14:19.132243 16030 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:19.131987 16161 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:19.119841 16187 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:14:19.135833 16197 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:19.136863 16197 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.140259 16197 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.097574 11252 rpcz_store.cc:276] 0709 12:14:17.955582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.955805 (+   223us) service_pool.cc:224] Handling call
0709 12:14:17.956203 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.956473 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:18.267298 (+310825us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.095549 (+828251us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b1800346840. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:19.095916 (+   367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:19.096023 (+   107us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:19.096137 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.096335 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.096592 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95973,"prepare.queue_time_us":218780,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":624291,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":241,"spinlock_wait_cycles":63382272,"thread_start_us":449390,"threads_started":2,"wal-append.queue_time_us":234840}
I20260709 12:14:19.142814 16189 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.144304 16189 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.147701 16004 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:19.148458 16004 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:19.149557 16161 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.147585 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:19.132754 16030 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:19.152287 16189 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.161206 16049 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:19.161737 16049 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:19.161811 16189 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:19.162781 16189 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.163321 16049 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.164745 16049 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:19.165619 16189 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.171749 16091 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:19.172363 16091 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:19.182133 16030 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.178939 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:19.187029 16091 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.191387 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:19.195266 16030 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:19.195468 15967 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:19.196185 15967 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:19.181375 16004 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.197597 15967 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.198261 11932 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:19.200246 16132 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:19.200760 16132 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:19.205837 16148 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:19.203967 16091 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:19.207314 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.208247 15992 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:19.217334 16173 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.307s	user 0.001s	sys 0.002s
I20260709 12:14:19.217581 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:19.210379 15992 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:19.229166 15752 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:19.229228 16061 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:19.232014 16061 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:19.238794 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.224828 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:19.223001 15992 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:19.248596 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.266505 11177 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:19.270085 11180 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:14:19.270697 11326 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:14:19.271023 16121 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.271867 16121 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:19.272969 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.273429 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.273924 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 56693.6us
I20260709 12:14:19.275538 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:19.277155 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:19.278052 11174 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:19.278556 16157 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.278755 15786 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:19.279289 16157 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.280299 15786 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:19.281078 15786 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:19.282331 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:19.282918 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:19.283782 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.284158 15974 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:19.284643 15974 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:19.285725 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.286693 11175 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 11: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:19.277897 16139 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:19.284789 11182 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:19.288085 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.281399 15786 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:19.307729 16202 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.308478 16189 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:19.315013 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.319602 16204 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:19.290086 16157 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.286022 15974 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.325505 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.322649 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.346715 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:19.321834 16121 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.353839 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:19.354633 15822 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:19.355786 11251 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:14:19.356366 16139 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.357787 16043 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:14:19.318172 16202 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.360126 11251 pending_rounds.cc:85] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 8
W20260709 12:14:19.343204 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 105250us
I20260709 12:14:19.360916 11180 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:19.335731 16061 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.350847 16148 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:19.320919 16189 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.363268 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1959) took 1338 ms. Trace:
I20260709 12:14:19.364610 11932 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:19.363240 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.377815 11253 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:19.358237 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.372928 11251 pending_rounds.cc:107] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:14:19.376673 11176 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:19.372006 15974 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:19.379558 11251 raft_consensus.cc:2889] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:19.370718 16202 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:19.365583 15988 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:19.388134 16177 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.499s	user 0.002s	sys 0.001s
I20260709 12:14:19.390501 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.391485 16139 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.392409 11324 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:19.393996 11330 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:14:19.394938 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:19.322973 16132 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.412032 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.413461 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:19.414701 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.415294 15900 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:19.418267 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:19.420873 16189 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:19.430428 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76857.4us
I20260709 12:14:19.431335 11256 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:19.432191 11323 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:14:19.432840 16148 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.375671 15786 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.448438 11932 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [10.10-13.11]
I20260709 12:14:19.449344 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:19.365789 15946 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 12:14:19.376926 11325 rpcz_store.cc:276] 0709 12:14:18.024731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.027459 (+  2728us) service_pool.cc:224] Handling call
0709 12:14:18.030354 (+  2895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.030569 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:14:19.246485 (+1215916us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.347189 (+100704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:19.351099 (+  3910us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:19.362671 (+ 11572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.362794 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.363059 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1215277,"prepare.queue_time_us":1255684,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":125,"raft.queue_time_us":18017,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":196,"thread_start_us":69314,"threads_started":1,"wal-append.queue_time_us":74421}
I20260709 12:14:19.429338 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:19.472332 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92047.5us
I20260709 12:14:19.468853 15900 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:19.385520 15822 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:19.491250 11327 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:19.491752 11327 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:19.482905 16204 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.494360 16204 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.494715 15900 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.496533 15900 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:19.497203 16204 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.499409 16148 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:19.492972 11327 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:19.504786 16186 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.607s	user 0.002s	sys 0.000s
I20260709 12:14:19.512324 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.520501 16105 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:19.516826 16189 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:19.522367 16189 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.524746 16189 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.481336 16139 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:14:19.520761 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.491775 16151 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:19.531941 11328 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:19.535459 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.516469 16132 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:19.542836 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.544245 11327 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:19.547672 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1970) took 1041 ms. Trace:
I20260709 12:14:19.549782 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:19.533799 16151 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:19.553833 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.562642 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.567380 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:19.549573 11329 rpcz_store.cc:276] 0709 12:14:18.493769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.494492 (+   723us) service_pool.cc:224] Handling call
0709 12:14:18.494860 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.495106 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:19.325981 (+830875us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.513278 (+187297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:19.517867 (+  4589us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:19.517927 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.526793 (+  8866us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.534732 (+  7939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":775863,"prepare.queue_time_us":818910,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":20061,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4785,"spinlock_wait_cycles":19490688,"thread_start_us":128270,"threads_started":2,"wal-append.queue_time_us":156958}
I20260709 12:14:19.568563 15737 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.569065 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.569186 15737 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.569747 11329 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:19.570695 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.571360 15737 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:19.572562 15966 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:19.573273 15966 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.578243 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:19.579229 16049 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:14:19.581044 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 151937us
I20260709 12:14:19.582229 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:19.590432 16039 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.589083 11252 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:19.590457 16049 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:14:19.600363 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.601061 11251 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:19.592947 16091 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.594807 16039 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.597225 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:19.604347 16039 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.605890 16057 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:19.606383 16057 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.606746 16057 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:19.588415 15966 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.594846 16105 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:19.610111 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.602105 16091 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.615217 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.614737 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:19.559466 16139 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:19.596112 16188 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.591s	user 0.001s	sys 0.004s
I20260709 12:14:19.562079 16202 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:14:19.633881 11328 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:19.653494 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.659122 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:19.660041 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.673974 11932 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 12:14:19.680449 16120 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.757s	user 0.022s	sys 0.048s
W20260709 12:14:19.680899 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57316.7us
W20260709 12:14:19.680963 16120 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.786s	user 0.023s	sys 0.049s
I20260709 12:14:19.703783 15822 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.709676 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:19.710588 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.712384 11327 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:14:19.714426 11256 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:19.713603 11932 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:19.687919 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:19.659766 16204 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:19.715376 11327 pending_rounds.cc:85] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 6
I20260709 12:14:19.718560 11327 pending_rounds.cc:107] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:14:19.718813 11327 raft_consensus.cc:2889] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:19.699329 16189 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:19.687644 15946 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.737290 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.735659 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.674185 16139 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.740279 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.740679 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.740998 11257 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:19.747205 11176 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:19.742057 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.688194 15966 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:14:19.772460 11177 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:19.762521 11256 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:19.746970 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.767557 11329 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:19.767805 16027 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:19.767881 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:19.769112 16091 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:19.757432 15974 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:19.775257 11324 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:19.738931 16189 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.717135 16120 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:19.770571 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:19.780808 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 234223us
I20260709 12:14:19.781962 11175 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 10: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:19.782428 11930 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:14:19.800599 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:19.784758 11325 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:19.785916 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.798337 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
W20260709 12:14:19.770478 15992 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.005s
W20260709 12:14:19.794098 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.804682 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.797009 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.713570 15944 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:14:19.790484 15764 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:19.798273 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50065.5us
I20260709 12:14:19.695413 16151 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.783924 16157 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.801189 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:19.802186 11323 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
I20260709 12:14:19.799254 16189 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:19.801689 16061 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:19.809263 16043 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:19.834767 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.837481 11323 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:19.837833 11249 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:19.838058 16139 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.823094 15974 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.838798 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:19.838871 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.839397 11256 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:19.839685 11323 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:19.839800 11250 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:19.814360 15764 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.846288 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:19.830880 16151 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:19.810827 15992 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:19.853986 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53459us
I20260709 12:14:19.856197 15764 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.861860 11250 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:19.863155 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:19.863894 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.812270 15946 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.866638 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:19.828162 16209 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:19.878477 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.867074 15974 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:19.880416 15949 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.880995 15949 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.882725 11328 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:19.883328 15949 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.885908 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:19.886205 11323 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:19.886842 11932 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:14:19.889899 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:19.891340 15946 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.893730 11326 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:19.869254 16028 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:19.848718 16043 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:19.891556 16213 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:14:19.895344 11174 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:19.898226 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.943681 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:19.946921 15971 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:19.947163 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.947608 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:19.947938 15971 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.948546 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:19.948696 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.901408 15764 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:19.950881 11249 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:19.951071 16148 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.951292 16120 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.951903 11926 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:19.951678 16148 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.947095 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:19.953940 16148 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.953760 16061 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.956558 16043 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.959898 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.960222 16209 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:19.961325 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:19.962034 11250 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:19.962627 16209 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:19.962847 16148 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:19.963471 16148 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.964473 15971 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:19.965341 16105 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:19.965618 16148 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:19.966029 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 115784us
I20260709 12:14:19.961349 11327 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
W20260709 12:14:19.967433 11296 connection.cc:256] RPC call timeout handler was delayed by -1.159971263999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260709 12:14:19.968415 11932 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:14:19.969507 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:19.969739 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:19.970350 11934 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:19.970516 11932 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:19.851861 16157 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:19.942215 16028 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:19.946187 15974 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:19.960039 16030 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.764s	user 0.012s	sys 0.003s
I20260709 12:14:19.994059 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.005460 11251 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:20.008466 11174 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:20.010419 11248 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:20.006930 11930 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:20.011844 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:20.013593 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.004544 16028 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:19.999879 15946 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.022388 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:20.031971 16178 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.135s	user 0.002s	sys 0.002s
I20260709 12:14:20.033802 16157 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.043980 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.044906 11174 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:20.050815 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:20.054019 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61450.9us
I20260709 12:14:20.051527 11930 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:20.021117 16214 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:20.038844 16148 raft_consensus.cc:2764] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:14:20.071470 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.072700 11174 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:20.027977 15974 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.085219 11928 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:14:20.086004 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:20.087949 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.039431 15946 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:20.098278 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.100191 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.101109 11174 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:20.101136 11179 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:20.106633 11934 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:20.058475 16057 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:20.111367 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 226560us
I20260709 12:14:20.111006 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.107939 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.118978 11933 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:20.120959 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.123517 16039 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.124194 16039 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.091571 16098 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:14:20.069787 16159 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:14:20.110422 16209 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:20.157863 11254 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:20.150667 11174 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:20.151185 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.160183 16209 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:20.160342 16098 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:20.160805 16098 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:20.164734 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.151546 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.169843 11933 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 12:14:20.170696 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53153us
I20260709 12:14:20.171276 11257 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
W20260709 12:14:20.120174 15937 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.108s	user 0.039s	sys 0.022s
I20260709 12:14:20.174489 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:20.152558 16179 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.249s	user 0.001s	sys 0.003s
I20260709 12:14:20.152048 16133 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.182485 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:20.176159 15937 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.284s	user 0.041s	sys 0.031s
I20260709 12:14:20.186275 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.187175 11257 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:20.175159 16027 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.130438 16189 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.190124 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:20.203454 16057 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.161960 16098 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.204747 16098 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:20.207872 16189 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.208547 16189 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.209807 11249 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:20.210033 15974 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.211558 11174 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:14:20.214375 15986 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:20.215790 11253 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:14:20.216414 15946 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.218173 16204 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:20.218705 16204 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:20.209039 16189 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.189442 16133 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.180862 16105 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:20.230942 16189 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.231447 15786 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:20.233312 15900 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.233935 15900 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.238209 15900 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.238612 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1947) took 2716 ms. Trace:
I20260709 12:14:20.238431 16027 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.241504 16027 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.242897 16132 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.243466 16132 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.245739 11176 raft_consensus.cc:606] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.232755 16028 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.247037 11927 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:20.247807 15900 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.239001 11331 rpcz_store.cc:276] 0709 12:14:17.522239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.522384 (+   145us) service_pool.cc:224] Handling call
0709 12:14:17.522741 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.563982 (+ 41241us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:19.357806 (+1793824us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.531391 (+173585us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:19.597670 (+ 66279us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.597888 (+   218us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:19.597944 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.238083 (+640139us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b540042b8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:20.238130 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:20.238175 (+    45us) raft_consensus.cc:1685] finished
0709 12:14:20.238286 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.238429 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1735093,"prepare.queue_time_us":1770334,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":15997,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":14780,"spinlock_wait_cycles":93874560,"thread_start_us":194995,"threads_started":2,"wal-append.queue_time_us":353295}
I20260709 12:14:20.248471 15900 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.249663 16039 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.250751 15900 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.251364 16061 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.257614 11250 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:20.257966 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.258841 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.260067 16098 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:20.260321 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.260982 16148 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:20.261225 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.262049 16203 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:20.305971 16203 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.306365 16203 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:20.307814 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46664.6us
I20260709 12:14:20.307936 15900 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:20.308535 15900 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.308913 15900 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:20.263337 16028 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:20.263497 16098 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:20.245918 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.266980 16140 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:20.311143 16140 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:20.311370 16098 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.312439 16140 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.274662 11255 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:20.305505 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:20.275110 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 85046.8us
I20260709 12:14:20.314283 11325 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:20.314713 16140 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:20.275363 15950 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:20.161540 16209 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.315263 15950 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:20.275888 16049 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:20.316179 15950 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.316778 16049 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:20.317216 16049 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:20.278160 11926 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:20.278806 16004 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:20.280292 11256 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.4-10.4]
W20260709 12:14:20.281566 11935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 862) took 2382 ms (client timeout 2082 ms). Trace:
I20260709 12:14:20.293929 16132 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.300819 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:20.301064 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:20.302134 11328 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:20.302815 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.317540 15950 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:20.327143 11330 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:20.261773 16105 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.327337 11323 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:20.329770 11935 rpcz_store.cc:269] 0709 12:14:17.898663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.898894 (+   231us) service_pool.cc:224] Handling call
0709 12:14:20.277461 (+2378567us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b5400555354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:20.281340 (+  3879us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5234496768}
I20260709 12:14:20.308861 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:20.337033 16105 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.341818 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.342252 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:20.342612 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:20.344770 11327 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:14:20.345201 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.345258 16189 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:20.346642 16027 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:20.344812 15966 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:20.348021 15966 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:20.345242 11327 pending_rounds.cc:85] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:20.348470 15966 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:20.346653 11323 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:20.350780 16148 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:20.352208 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:20.353075 15992 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:20.352155 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.348577 11327 pending_rounds.cc:107] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:14:20.353582 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.359306 11327 raft_consensus.cc:2889] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:20.359375 11933 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:20.359576 11328 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
W20260709 12:14:20.356769 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.363159 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.363724 11179 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:20.365191 11173 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:20.363993 11174 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:20.367991 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.370596 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:20.386770 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.383517 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.391000 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.391355 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.387279 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.392072 11182 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:20.386565 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.393956 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.392027 11328 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:20.358832 16148 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:20.393790 16057 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.401583 11934 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:20.406726 11928 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:20.406841 11176 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:20.410046 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.413302 11323 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 10.
W20260709 12:14:20.416870 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 64751.7us
I20260709 12:14:20.418481 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.419337 11328 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:20.421716 15986 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.422297 15986 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.424420 15986 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.426927 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:20.428558 16057 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.429849 16202 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:20.431216 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:20.431571 16057 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.432462 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:20.435139 11330 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:20.436473 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:20.441912 11176 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:20.441985 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.443650 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:20.449294 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.448400 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:20.417124 15937 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:20.447496 16157 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.468745 11327 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:20.450151 16220 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.470763 11330 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:14:20.472055 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 285159us
I20260709 12:14:20.479255 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2481) took 1163 ms. Trace:
I20260709 12:14:20.480377 11322 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:20.486364 16223 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.312s	user 0.000s	sys 0.002s
I20260709 12:14:20.503396 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.500461 11326 rpcz_store.cc:276] 0709 12:14:19.317732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.326003 (+  8271us) service_pool.cc:224] Handling call
0709 12:14:20.478910 (+1152907us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54001c43d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:20.479132 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251428736}
I20260709 12:14:20.456876 16028 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:20.452666 16189 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:20.425426 16204 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.450202 15986 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:20.502285 11175 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: [10.6-11.7]
W20260709 12:14:20.498611 16216 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.606s	user 0.004s	sys 0.000s
I20260709 12:14:20.508592 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.455180 15937 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.509946 16028 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.522002 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.525395 16120 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:20.525905 16120 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.507206 16159 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:14:20.507511 16157 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.545185 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.547004 16157 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.530831 16203 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.550455 16203 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:20.551841 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.510525 16121 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:20.554039 11932 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:20.528329 15986 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.556452 15986 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.526343 16120 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:20.502727 16220 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.563835 16220 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.563417 15900 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.564862 16159 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:20.565286 16159 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:20.566270 16159 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.566593 16049 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.568015 16159 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:20.538540 16027 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.538704 16189 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:20.583354 16027 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.583600 16189 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.541193 11182 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:20.585875 16027 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.542936 16204 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:20.568853 11257 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:14:20.570115 11325 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 13. Ops: []
I20260709 12:14:20.569870 15966 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.527055 16028 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:20.588912 15900 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.571380 16105 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.571924 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.572824 15937 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.549300 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.574878 16174 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:14:20.575685 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 75027.9us
I20260709 12:14:20.576467 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 877) took 1129 ms. Trace:
I20260709 12:14:20.578097 16234 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.536441 15789 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:20.525931 16209 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:14:20.541770 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 109391us
I20260709 12:14:20.593205 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 1972) took 1106 ms. Trace:
I20260709 12:14:20.604110 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.605487 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.606729 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2136) took 1010 ms. Trace:
I20260709 12:14:20.615193 11255 rpcz_store.cc:276] 0709 12:14:19.596440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.609774 (+ 13334us) service_pool.cc:224] Handling call
0709 12:14:20.606488 (+996714us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54002a62d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:20.606649 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526589696}
I20260709 12:14:20.610121 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.610881 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.611610 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.578114 16133 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:20.618675 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.612865 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.618963 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.616457 11182 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:20.619788 11328 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:20.602342 16105 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.611043 11252 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:20.622123 16105 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.622918 16049 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.617465 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.618498 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:20.627502 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.613502 11322 rpcz_store.cc:276] 0709 12:14:19.486391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.490846 (+  4455us) service_pool.cc:224] Handling call
0709 12:14:20.480054 (+989208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:20.501146 (+ 21092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.593027 (+ 91881us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54001c43d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:20.593151 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2273926016}
I20260709 12:14:20.590847 16098 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:14:20.642120 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 271670us
W20260709 12:14:20.645743 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55807.1us
I20260709 12:14:20.602409 11931 rpcz_store.cc:276] 0709 12:14:19.447307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.449895 (+  2588us) service_pool.cc:224] Handling call
0709 12:14:19.450230 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:19.474159 (+ 23929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:19.474202 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:19.516986 (+ 42784us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.794886 (+277900us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:19.794956 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.795026 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:20.497570 (+702544us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:20.576122 (+ 78552us) raft_consensus.cc:1685] finished
0709 12:14:20.576148 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.576401 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15852,"raft.queue_time_us":96,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4309,"spinlock_wait_cycles":12685184,"thread_start_us":247032,"threads_started":1,"wal-append.queue_time_us":357276}
I20260709 12:14:20.646736 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:20.647163 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.648108 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:20.652603 16049 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:20.612911 15789 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.610612 16204 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:20.653860 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:20.656350 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:20.667303 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.670382 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.654105 15937 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:20.656355 16121 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:20.655314 16027 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:20.644109 15950 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:20.658624 15789 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:20.653913 16209 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:20.679837 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.626299 16234 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.657269 16098 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:20.678282 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.681977 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.690907 11180 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
W20260709 12:14:20.691074 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 80291.7us
W20260709 12:14:20.682546 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.666236 16043 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:20.669397 16004 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:20.681246 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:20.694780 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.692087 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:20.688941 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.704965 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.706003 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.706445 11182 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:20.695768 11179 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 12:14:20.676669 15986 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:20.676577 15937 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:20.714349 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44052.8us
I20260709 12:14:20.665961 16203 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:14:20.718896 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.718896 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.719791 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.720438 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.720675 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.721050 11176 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 10.
I20260709 12:14:20.721233 11257 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 9.
I20260709 12:14:20.715004 11173 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:14:20.693368 16004 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:20.716681 11175 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:20.718174 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.662276 16220 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:14:20.710217 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.722605 11173 pending_rounds.cc:85] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 10
I20260709 12:14:20.719061 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.731994 11173 pending_rounds.cc:107] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:14:20.732262 11173 raft_consensus.cc:2889] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:20.727036 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.727901 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.732925 11926 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:20.733212 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.731330 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:20.737303 11935 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:14:20.737900 11182 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:20.742156 11250 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:20.743523 11179 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.4-10.4]
I20260709 12:14:20.701829 16189 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.707058 16133 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:20.745438 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:20.745541 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 118147us
I20260709 12:14:20.746340 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:20.730805 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:20.711763 16091 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:20.679093 16049 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:14:20.674870 16120 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.714712 16027 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:20.721837 16151 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:20.728374 16039 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:20.743355 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.678452 16242 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:14:20.722244 15937 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:20.747475 11249 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:20.758363 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.758661 11254 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:20.760195 11181 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:20.760432 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 89055.1us
I20260709 12:14:20.766266 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.768142 16234 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.769712 16105 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:20.736981 16148 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:20.770740 15966 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:20.773180 16120 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:20.774240 16236 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:14:20.745105 16245 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:14:20.775935 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:20.854434 11932 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:20.776448 11178 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:20.782109 16220 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:20.782781 16239 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:20.705935 16204 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:20.786159 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 88183.6us
I20260709 12:14:20.790102 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.791086 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:20.791990 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:20.792721 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.794396 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.860518 11934 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:20.796308 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.794723 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.795570 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:20.838145 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55581us
I20260709 12:14:20.802512 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.800437 11251 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:14:20.841403 11182 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 12:14:20.751605 16091 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.789149 16039 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.861003 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.864091 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:20.866283 11175 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:20.795166 11179 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:20.806939 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:20.864094 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 186736us
I20260709 12:14:20.870029 11930 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:20.870324 11180 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:20.870229 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:20.900825 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:20.870689 11177 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:20.794320 16203 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:20.911788 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.911190 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:20.748335 16028 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:20.804395 16105 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:14:20.910784 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 165422us
I20260709 12:14:20.929565 11181 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:20.929384 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:20.941711 11173 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:14:20.933314 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 126487us
I20260709 12:14:20.854780 16151 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:20.941090 11177 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:14:20.944248 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:20.869827 16039 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:20.943630 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:20.786306 16041 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.274s
I20260709 12:14:20.952750 11248 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:14:20.840363 16148 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:20.959481 16049 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:20.959951 15966 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:20.785995 16189 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 12:14:20.925827 16233 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:20.795953 11077 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 2 to 6, term changed from 6 to 12, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:21.000283 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.016283 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:20.963481 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 161055us
I20260709 12:14:20.858357 16220 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:20.869962 16091 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:20.967435 11926 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:14:20.975839 16133 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:20.983306 16151 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:20.864624 16157 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 12:14:20.989666 11173 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:20.990031 11932 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:20.996039 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:20.999493 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 131782us
I20260709 12:14:21.017817 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:20.928404 16120 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:14:20.945935 16105 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:20.950645 15986 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:21.023543 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56862us
W20260709 12:14:21.025406 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 155269us
I20260709 12:14:20.794729 16098 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.038833 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.027360 11329 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:21.030658 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:21.045145 11328 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:14:21.028831 11327 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:21.032238 15984 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:21.036645 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.048137 16148 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.049729 16148 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:21.050215 16250 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:14:21.051141 16248 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:21.051630 16248 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.051993 16248 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:21.054930 16039 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.057245 11935 raft_consensus.cc:2368] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:21.058238 15986 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.058882 16039 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:21.041244 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 45374.8us
I20260709 12:14:20.961748 16202 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.060820 16202 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.062906 16202 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:20.958092 16004 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.043488 16043 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:21.065497 16043 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:21.055729 16028 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.056039 15789 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.066092 16043 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:21.025372 16133 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.068929 11931 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:21.040685 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.027778 16203 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:21.076608 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:21.006455 16057 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:14:20.866286 16209 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.038167 16233 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:21.081111 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:21.079249 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:21.041162 15900 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:21.025863 16027 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.084327 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:21.086138 15950 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.091207 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:21.079674 16133 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:21.099112 16220 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.100554 16220 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:21.100379 15950 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.106518 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:21.107851 15950 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.082257 15786 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:21.109874 11929 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:21.085523 16253 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:21.103272 16238 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.376s	user 0.001s	sys 0.000s
I20260709 12:14:21.115976 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.095650 16157 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:21.085886 15789 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.095268 16189 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:14:21.119437 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 80737.1us
I20260709 12:14:21.079361 15984 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:21.140851 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 105638us
I20260709 12:14:21.132792 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:21.135564 15986 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:14:21.137197 16252 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:21.113727 16028 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:21.119982 16245 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:21.139037 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.129763 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:21.126266 16253 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.126266 16098 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:21.159636 16098 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.160085 16098 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:21.161994 16157 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:21.162369 16233 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 3, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.094554 16239 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:14:21.165794 16253 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:21.166683 16239 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:21.167096 16239 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:21.168044 16239 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.169305 16239 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:21.126292 16057 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:21.153882 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:21.108314 16203 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:14:21.122414 16202 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:14:21.156615 16189 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:21.157018 16241 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:21.158303 15984 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.148329 16251 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.110828 16242 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:14:21.177977 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:21.154757 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:21.192736 11328 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:21.191579 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:21.319792 11327 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:21.173800 11175 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:21.197472 11928 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [13.12-15.13]
W20260709 12:14:21.198143 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51351.5us
I20260709 12:14:21.197019 16241 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:21.186050 16057 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.192267 16157 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:21.308385 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:21.330835 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.153671 16245 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.332830 11175 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:14:21.310747 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:21.318142 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.320014 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:21.175673 15786 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:21.172633 16204 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:21.336140 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 181454us
I20260709 12:14:21.344062 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:21.333431 11331 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
W20260709 12:14:21.341410 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 163529us
I20260709 12:14:21.340041 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:21.346680 11181 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:21.332278 16209 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:21.351703 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:21.372000 11182 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:21.329869 16241 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:21.332983 16157 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:21.335748 16245 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:21.342865 16027 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:21.333624 16251 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:21.397009 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 281044us
W20260709 12:14:21.407521 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 63591us
I20260709 12:14:21.383399 16257 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:21.356909 16265 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:21.405309 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74553.3us
I20260709 12:14:21.410820 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:21.397221 16157 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:21.413870 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:21.340441 16091 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.344691 16248 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.428815 11327 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:21.397861 15986 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:21.454579 11182 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
W20260709 12:14:21.460755 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51114.6us
I20260709 12:14:21.382437 15984 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:14:21.415356 16265 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:21.486846 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 140113us
I20260709 12:14:21.488811 11324 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:21.495489 11248 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:21.490263 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.495669 11222 leader_election.cc:400] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:21.496201 11222 leader_election.cc:403] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:14:21.450022 16257 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:21.501799 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.461565 16266 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:21.507745 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:21.515506 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53659us
I20260709 12:14:21.513314 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.513545 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:21.518025 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.526188 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.485507 16209 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:21.489905 16203 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.529498 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.531597 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.533973 11250 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:21.530146 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:21.493289 16265 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:21.536994 11935 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:21.538587 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 127851us
I20260709 12:14:21.504602 16268 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:21.540617 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.527843 16266 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:21.543037 16264 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:14:21.549175 11254 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:21.542645 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.551192 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:21.482075 15986 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.554898 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:21.556231 16266 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:21.566217 16209 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.567829 16189 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:21.568809 16189 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:21.569034 15986 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.569236 16189 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:21.527674 16174 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:14:21.531188 16133 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.557580 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.528849 16278 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:21.585497 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.586786 11181 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:21.587774 11935 raft_consensus.cc:2368] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13.
I20260709 12:14:21.578565 11256 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:21.536700 16277 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:14:21.582091 16159 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.974s	user 0.011s	sys 0.001s
I20260709 12:14:21.538316 16004 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:21.586639 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:21.546754 16091 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.548507 16268 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:21.574520 15900 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:21.577984 16274 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:21.588898 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:21.538197 16203 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.594394 11251 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:14:21.612707 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.635362 11930 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:21.613430 11933 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:21.613821 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.636957 11934 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:21.631340 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 91633.3us
I20260709 12:14:21.612380 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.593933 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.640203 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.592577 16133 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.646458 16133 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:21.646916 16133 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.647367 16133 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:21.648842 16105 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 12:14:21.649381 16105 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.651949 16105 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.654263 16251 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.654845 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.587083 16057 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.596626 16174 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:21.658089 16174 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:21.658141 16057 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:21.659305 16174 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.660305 15992 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:21.660758 16174 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:21.661221 15992 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:21.662329 15992 consensus_queue.cc:1243] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:14:21.663987 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 1990) took 1060 ms. Trace:
W20260709 12:14:21.664620 15937 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.796s	user 0.000s	sys 0.009s
W20260709 12:14:21.610904 16271 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:14:21.599109 16277 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:21.668083 16277 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:21.669064 16260 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:14:21.555039 16282 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:21.594157 16220 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:21.674033 16220 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:21.674469 16220 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:21.665087 16133 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:21.675616 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 86800.2us
I20260709 12:14:21.677112 16133 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.600049 16257 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.679093 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:21.679996 11181 raft_consensus.cc:2410] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:14:21.685009 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:21.664417 11330 rpcz_store.cc:276] 0709 12:14:20.603757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.603863 (+   106us) service_pool.cc:224] Handling call
0709 12:14:21.663764 (+1059901us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b54000c71d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:21.663909 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599914240}
I20260709 12:14:21.573843 15822 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:21.606463 16259 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 12:14:21.610298 16262 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
W20260709 12:14:21.600757 16269 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 12:14:21.615492 16043 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.001s
W20260709 12:14:21.687702 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47595us
I20260709 12:14:21.590978 15986 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:21.709543 15986 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.710029 15986 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:21.634087 16278 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.715932 16278 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.717730 16203 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:21.591233 16283 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:14:21.718400 16261 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.548s	user 0.009s	sys 0.000s
I20260709 12:14:21.718688 16283 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:21.719794 16283 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.721333 16204 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:21.721807 16204 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:21.618484 16268 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.722771 16268 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:21.722797 16204 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 2, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.614409 16049 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.727540 16049 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:21.692710 16280 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:14:21.641418 16121 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.954s	user 0.008s	sys 0.004s
I20260709 12:14:21.677383 15984 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:21.605825 15786 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:21.560341 16027 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:14:21.696831 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:21.691115 16133 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.698297 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:21.698788 16255 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:21.699378 16273 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:14:21.703665 16270 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.214s	user 0.000s	sys 0.002s
I20260709 12:14:21.693813 16202 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:14:21.709030 16239 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.533s	user 0.004s	sys 0.003s
I20260709 12:14:21.640980 16274 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.689213 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:21.592445 16248 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.730121 11325 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:21.618222 16091 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:21.738042 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:21.810590 11182 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:21.788620 16253 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.002s
I20260709 12:14:21.665057 15937 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:21.754349 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56435.1us
I20260709 12:14:21.769831 11328 raft_consensus.cc:2410] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:14:21.815639 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.590845 16189 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:21.817564 15992 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:21.818183 15992 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:21.819157 15992 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.728111 15822 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.764602 16121 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:21.797344 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 276099us
W20260709 12:14:21.807607 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 118515us
I20260709 12:14:21.775297 16133 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.616973 16004 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:21.776445 15984 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:21.739827 16043 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.829084 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:21.845805 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:21.793666 16204 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:21.798053 16255 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:21.813251 16278 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:21.849257 11253 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:21.865657 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:21.869714 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:21.871996 16255 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.873998 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:21.875977 16245 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.876740 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:21.877352 11249 raft_consensus.cc:2368] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:21.877450 11176 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:21.878541 16245 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:21.844512 16148 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.792s	user 0.012s	sys 0.000s
I20260709 12:14:21.881441 11182 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:21.840664 16004 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:21.887873 11179 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:21.838408 16284 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:14:21.889089 11326 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:14:21.868577 16098 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.002s
I20260709 12:14:21.889854 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:21.869534 16202 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:21.887305 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:21.895093 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.897372 11181 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
W20260709 12:14:21.895198 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 275499us
I20260709 12:14:21.889413 15949 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:21.884169 16091 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:21.830297 16253 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.869262 16278 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:21.904084 16284 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:21.901185 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:21.906200 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:21.893250 16148 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.928126 11254 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:21.933211 11182 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:21.907660 16202 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:21.943795 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.913247 16091 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:21.896347 16098 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:21.912760 16297 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:21.951094 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44996.3us
I20260709 12:14:21.960095 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:21.952335 16286 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:14:21.926499 16028 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:21.925455 16224 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:21.967460 16301 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:21.974864 16242 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:21.975478 16242 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:21.975832 16267 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:14:21.976770 16253 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.958652 16239 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:21.978794 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2175) took 1276 ms. Trace:
I20260709 12:14:21.978143 16202 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.979466 16278 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.980803 16278 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:21.982354 16133 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.927362 15984 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.984795 16274 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:21.985476 16274 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:21.985877 15822 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:21.986124 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:21.986985 16043 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.987342 16028 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:21.987607 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:21.988418 16028 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.989859 16224 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:21.978188 16239 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:21.989039 16241 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.590s	user 0.000s	sys 0.010s
I20260709 12:14:22.001924 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.003101 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:21.976672 16242 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.008163 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:22.010321 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 43843.2us
I20260709 12:14:22.011312 11323 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 12:14:22.017302 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.018522 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.020323 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:22.023914 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:21.979240 11255 rpcz_store.cc:276] 0709 12:14:20.702644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.711233 (+  8589us) service_pool.cc:224] Handling call
0709 12:14:20.740131 (+ 28898us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:20.853293 (+113162us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:20.926111 (+ 72818us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.983253 (+ 57142us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b1800184360. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:21.309460 (+326207us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:21.345569 (+ 36109us) log.cc:844] Serialized 52 byte log entry
0709 12:14:21.345810 (+   241us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:21.346141 (+   331us) log.cc:844] Serialized 14 byte log entry
0709 12:14:21.496269 (+150128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.496343 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:21.978536 (+482193us) raft_consensus.cc:1685] finished
0709 12:14:21.978566 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.978723 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67100,"prepare.queue_time_us":72725,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":11959,"raft.queue_time_us":237324,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":87978,"spinlock_wait_cycles":6919040,"thread_start_us":423323,"threads_started":3,"wal-append.queue_time_us":207836}
I20260709 12:14:22.028254 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.027676 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:21.978024 15937 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:21.915656 16284 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.033735 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:22.034219 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.034737 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:21.944249 16148 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.035415 11248 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:21.991063 16301 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.042197 11331 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [13.12-15.13]
W20260709 12:14:22.017433 16157 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.004s
I20260709 12:14:22.012683 16159 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.011693 16041 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:14:22.028643 16209 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:22.020216 16251 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.052013 11325 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:22.055833 11330 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:22.059919 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.043500 16242 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:22.063100 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.063575 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.064416 11255 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:22.053560 16209 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:22.005393 16266 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.069705 16209 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.072863 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.046392 16301 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:22.076143 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 109917us
I20260709 12:14:22.076967 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:22.078115 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:22.078703 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:22.061031 16265 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 12:14:22.028502 15986 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.017572 16241 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.088672 11931 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 17. Ops: []
I20260709 12:14:22.090005 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:22.091749 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.093241 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.049316 16157 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.100991 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:22.101617 11929 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:22.105744 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:22.054920 16159 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.090902 16265 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.110826 16289 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:22.111372 16292 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:22.099915 16253 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:22.118916 15966 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:22.146276 15966 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:22.147385 15966 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.148680 15966 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.149792 16189 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.152026 16189 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.153566 16266 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.154738 16239 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.139850 16289 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.156363 16281 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.157411 16289 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.121112 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.061285 16251 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.160857 16251 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.122444 16148 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:22.161868 16148 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:22.122457 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.162914 11256 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:22.098708 15937 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:22.126362 11325 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:14:22.011613 16297 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.099699 16288 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.134250 11929 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:22.062830 16151 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:14:22.171600 11250 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:22.139650 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 146631us
I20260709 12:14:22.140564 11935 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:22.144076 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:22.145028 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:22.145277 16159 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:22.132458 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.130579 16244 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:22.176769 11934 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:22.096084 16275 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.068043 16091 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.162533 16148 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:22.211881 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.184523 16049 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.092159 16277 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
W20260709 12:14:22.214198 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 136108us
I20260709 12:14:22.220016 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:22.233894 16303 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:14:22.213970 16253 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.234895 16253 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.237176 16253 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:22.244151 16307 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:14:22.210892 16301 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.137676 16285 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.253s
I20260709 12:14:22.180354 16295 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:22.135193 16308 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 12:14:22.266834 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:22.230749 16224 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.276736 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:22.278833 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:22.279680 11935 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.261997 16098 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:22.178771 16288 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.297477 11933 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:22.285645 16244 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:22.321230 11930 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:22.268106 16245 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:14:22.272848 16204 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:14:22.323861 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:22.270673 16202 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:14:22.219440 16275 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.275175 16283 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:14:22.327018 16157 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:22.353760 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.331211 16302 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.331678 16304 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:22.336328 16288 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:22.337819 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:22.338973 16323 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:22.339146 16241 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:22.342085 11328 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:22.341789 16049 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.344080 16301 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.344750 16275 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:22.344945 16293 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.511s	user 0.000s	sys 0.002s
I20260709 12:14:22.345846 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 774) took 1211 ms. Trace:
I20260709 12:14:22.366422 11932 rpcz_store.cc:276] 0709 12:14:21.134144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.134260 (+   116us) service_pool.cc:224] Handling call
0709 12:14:21.148359 (+ 14099us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:21.191688 (+ 43329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:21.191724 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:21.307705 (+115981us) log.cc:844] Serialized 26 byte log entry
0709 12:14:21.482517 (+174812us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:21.482781 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.345415 (+862634us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54004312d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:22.345466 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.345517 (+    51us) raft_consensus.cc:1685] finished
0709 12:14:22.345531 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.345780 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45836,"raft.queue_time_us":11485,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":12373,"spinlock_wait_cycles":39726848,"thread_start_us":126309,"threads_started":1,"wal-append.queue_time_us":364948}
I20260709 12:14:22.346899 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:22.349404 16263 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.938s	user 0.000s	sys 0.004s
I20260709 12:14:22.368906 11331 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:22.369930 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.370829 11331 raft_consensus.cc:2410] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:14:22.349531 16258 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:14:22.361446 16220 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.686s	user 0.008s	sys 0.004s
W20260709 12:14:22.359539 16282 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.690s	user 0.004s	sys 0.008s
I20260709 12:14:22.363211 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:22.342912 16244 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:22.374405 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 283446us
I20260709 12:14:22.391064 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:22.296561 16004 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:22.382611 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.388712 16282 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14.
I20260709 12:14:22.362805 16302 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.374219 16288 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:22.395160 16288 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.395702 16288 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.397265 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.397887 11177 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:22.399075 11180 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:22.399976 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.400234 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:22.400643 11177 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:22.400904 11934 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:22.390012 16220 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.403390 11251 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:22.404724 16220 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.406348 16302 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.374822 11256 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:22.409832 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:22.405313 11935 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:22.413743 11932 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
W20260709 12:14:22.395649 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 172502us
I20260709 12:14:22.415009 11147 leader_election.cc:400] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:22.415493 11147 leader_election.cc:403] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:22.365468 16297 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.421227 16288 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.430050 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.428966 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.431185 11180 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:14:22.431757 16288 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.432296 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:22.432971 11253 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:22.377024 16258 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:22.434334 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:22.435300 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.436148 11253 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:22.438946 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.439532 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.439693 11181 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:22.440766 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.392541 16301 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:22.444895 11179 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:14:22.445277 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.445942 11177 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:22.447642 11249 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:22.450250 11934 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:22.453429 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.457948 11323 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:14:22.472668 11323 pending_rounds.cc:85] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:22.472931 11323 pending_rounds.cc:107] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:14:22.473212 11323 raft_consensus.cc:2889] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.412832 15786 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:22.473987 11174 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:14:22.469724 11255 raft_consensus.cc:2410] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:14:22.479875 11331 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:22.465440 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:22.479884 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.467180 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.481158 11176 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:22.434695 16258 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.466284 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.467093 16004 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.484546 11326 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:22.503758 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.476218 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:22.469849 16268 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.747s	user 0.009s	sys 0.005s
I20260709 12:14:22.510900 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:22.511018 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:22.512092 16148 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.512588 16288 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.512850 15986 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.512905 16319 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:22.512590 16148 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.478772 16234 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:22.515448 16234 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.515648 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.515827 16234 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.516538 16148 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:22.503360 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.518726 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:22.513098 16204 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:22.519480 16204 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.513195 16288 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.519932 16204 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:22.505576 11180 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:22.499517 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.501991 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.460352 16276 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:22.535244 16276 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:22.535512 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:22.536172 16276 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.537221 16251 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:22.537720 16276 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.537796 16251 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:22.538878 16251 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.506496 16285 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:22.542780 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.506829 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.506891 11178 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:22.544092 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.444733 16301 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.545411 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.508883 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.457578 16209 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.546062 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.525207 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:22.525521 11253 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:22.461263 11177 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:22.505504 11181 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:22.532896 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 868) took 1461 ms. Trace:
I20260709 12:14:22.533149 11327 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:14:22.519475 11256 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:22.534830 11329 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:14:22.526471 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.548939 11329 pending_rounds.cc:85] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:14:22.549398 11329 pending_rounds.cc:107] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:14:22.550516 11329 raft_consensus.cc:2889] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.508222 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:22.666690 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 158526us
I20260709 12:14:22.545115 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:22.524820 16105 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:22.552564 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.547456 16301 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:22.548645 11324 rpcz_store.cc:276] 0709 12:14:21.071663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.071802 (+   139us) service_pool.cc:224] Handling call
0709 12:14:22.532650 (+1460848us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54001c4dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:22.532823 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910283008}
I20260709 12:14:22.674369 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.526180 16288 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:22.619499 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:22.547058 11254 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:22.659070 16098 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:22.660801 16278 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:22.543030 16233 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:22.480072 16320 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.117s
W20260709 12:14:22.739143 16328 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:14:22.719733 16233 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:22.741420 16233 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.742982 16233 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:22.759191 16204 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:22.689786 16281 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.005s
I20260709 12:14:22.762125 16281 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:22.763443 16278 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.545835 16244 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.483157 16274 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:14:22.548424 15986 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:22.544625 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.668892 11256 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:22.668995 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.672168 11182 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:22.672535 11147 leader_election.cc:400] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13.
I20260709 12:14:22.788878 11147 leader_election.cc:403] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:22.675983 11179 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:22.677029 11324 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:14:22.682080 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.720438 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2165) took 1094 ms. Trace:
I20260709 12:14:22.720629 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:22.789355 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:22.792337 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:22.774151 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:22.805804 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:22.807158 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 134744us
I20260709 12:14:22.807936 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:22.790966 11257 rpcz_store.cc:276] 0709 12:14:21.625830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.626047 (+   217us) service_pool.cc:224] Handling call
0709 12:14:21.630212 (+  4165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.729185 (+ 98973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:21.729216 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:21.729259 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:21.729291 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.720166 (+990875us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b5400069854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:22.720215 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.720375 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227117952}
I20260709 12:14:22.809615 16276 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13.
I20260709 12:14:22.809810 15822 raft_consensus.cc:2764] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:14:22.810299 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:22.811142 15822 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:22.811359 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:22.811547 15822 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.811935 11256 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 10. Current term is 11. Ops: [10.4-10.4]
I20260709 12:14:22.812093 15822 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:22.810338 11173 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:14:22.813262 11256 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:22.805905 16151 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:22.813771 16281 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:22.814154 16151 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.814514 16281 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:22.816901 16151 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.818845 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.819705 11253 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:14:22.821295 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 276713us
I20260709 12:14:22.830142 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:22.831142 11935 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:22.834355 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:22.834646 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:22.834967 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.721621 16288 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.833523 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.842981 11257 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:22.846268 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:22.847137 11173 raft_consensus.cc:606] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.847572 11173 raft_consensus.cc:614] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:22.853826 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:22.859965 15986 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:22.860492 15986 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.860886 15986 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:22.862843 16282 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:22.863579 16282 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.864387 11934 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:22.865705 16302 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:22.853792 16253 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:22.866930 16253 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.867304 16253 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.849040 16297 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:14:22.869451 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:22.857409 16241 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.848346 16243 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.871275 16241 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.871842 16243 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.840337 16027 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:22.834209 16285 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:22.875705 16255 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.876098 16280 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.876284 16255 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.876664 16280 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.822728 15966 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:22.878221 15966 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:22.878854 15966 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:22.879134 16255 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.879974 15966 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.882282 16314 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:22.882973 16314 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:22.883423 16314 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.869375 15986 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.823909 16274 consensus_queue.cc:1243] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:14:22.895215 11253 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:22.857168 16275 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.903857 11254 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:14:22.893173 16209 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.854702 16288 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.906926 11257 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:22.907896 11253 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:22.910440 11256 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:22.915694 11254 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:22.925508 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.924480 11077 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:22.926432 11329 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:22.932734 11256 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:14:22.932843 11257 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:22.933285 11256 pending_rounds.cc:85] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 6
I20260709 12:14:22.933585 11256 pending_rounds.cc:107] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:14:22.934069 11256 raft_consensus.cc:2889] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.904934 16274 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:22.915853 16276 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.923240 16278 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.955119 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2188) took 1075 ms. Trace:
I20260709 12:14:22.956101 11255 rpcz_store.cc:276] 0709 12:14:21.880019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.074826 (+194807us) service_pool.cc:224] Handling call
0709 12:14:22.949936 (+875110us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b5400087ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:22.955029 (+  5093us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866959744}
I20260709 12:14:22.916404 16268 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:14:22.963809 11256 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:14:22.927419 16241 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.927474 15822 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.965785 11257 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:22.920542 16234 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.916770 16275 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.949536 16282 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.919297 16209 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.974329 16258 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.974864 15822 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.954457 16276 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:22.976637 16258 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:22.977797 16276 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:22.980259 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2167) took 1333 ms. Trace:
I20260709 12:14:22.982991 11257 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:14:22.989621 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 120290us
I20260709 12:14:22.987821 11251 rpcz_store.cc:276] 0709 12:14:21.646471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.736656 (+1090185us) service_pool.cc:224] Handling call
0709 12:14:22.736995 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:22.737364 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:22.737395 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:22.737904 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:14:22.930901 (+192997us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:22.930943 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.931019 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.980015 (+ 48996us) raft_consensus.cc:1685] finished
0709 12:14:22.980047 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.980192 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":92,"thread_start_us":192637,"threads_started":1,"wal-append.queue_time_us":239112}
I20260709 12:14:22.971815 16241 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.001299 16314 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.001947 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:22.985759 16243 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:22.960157 16274 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.009377 16057 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.017936 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.017030 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:23.024951 11933 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:23.024956 11935 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:23.014099 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.022653 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:23.026144 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 237377us
I20260709 12:14:23.026573 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:23.031833 11255 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:23.032260 11255 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:23.031862 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.033110 11180 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:23.028224 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:23.033931 16276 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:23.038504 16276 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.038690 15949 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.038913 16276 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:23.039225 15949 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:22.996167 15822 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:14:23.043505 11329 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:23.016454 16241 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.049095 11255 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.040585 16335 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:23.060194 11255 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:23.030886 16057 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.066141 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2509) took 1063 ms. Trace:
I20260709 12:14:23.066551 11330 rpcz_store.cc:276] 0709 12:14:22.002541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.017028 (+ 14487us) service_pool.cc:224] Handling call
0709 12:14:23.065964 (+1048936us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54000c6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:23.066080 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86764928}
I20260709 12:14:23.068792 11330 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:14:23.070251 11329 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:23.071762 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:23.057533 16337 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:23.083894 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.084825 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:23.086318 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.088187 11178 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:14:23.089349 15900 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:23.065018 16335 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.091279 16265 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.091706 16335 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:23.093452 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2522) took 1067 ms. Trace:
I20260709 12:14:23.093864 11323 rpcz_store.cc:276] 0709 12:14:22.026570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.130328 (+103758us) service_pool.cc:224] Handling call
0709 12:14:22.130667 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.482535 (+351868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.093218 (+610683us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54000c67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:23.093391 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751680384}
I20260709 12:14:23.095991 16301 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.097266 11331 raft_consensus.cc:2368] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:23.098557 16274 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:23.099037 16204 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:23.100667 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 682) took 1030 ms. Trace:
I20260709 12:14:23.080224 16288 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:23.101130 11249 rpcz_store.cc:276] 0709 12:14:22.070098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.120674 (+ 50576us) service_pool.cc:224] Handling call
0709 12:14:23.100453 (+979779us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540010b654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:23.100604 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541342976}
I20260709 12:14:23.102202 11249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:14:23.102754 11249 raft_consensus.cc:606] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:14:23.103511 16284 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:23.104178 11251 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:14:23.105242 16332 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:23.107405 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2515) took 1103 ms. Trace:
I20260709 12:14:23.091331 16241 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.107779 11328 rpcz_store.cc:276] 0709 12:14:22.004467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.023540 (+ 19073us) service_pool.cc:224] Handling call
0709 12:14:23.107240 (+1083700us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540042aed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:23.107350 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723928832}
I20260709 12:14:23.062155 16280 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.097174 16285 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:23.125993 11328 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:14:23.088207 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:23.128427 15949 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.129336 16275 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:23.132201 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46599.2us
I20260709 12:14:23.132504 15900 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:23.133153 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:23.133452 15900 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.140050 11933 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
W20260709 12:14:23.141198 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 83349.7us
I20260709 12:14:23.145081 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:23.131255 16314 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.146101 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.148881 11174 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:23.143307 16049 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.149338 16255 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:14:23.150468 16347 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:23.154399 16342 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:14:23.158390 16347 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:23.158792 16347 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:23.159578 16347 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.160770 16347 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:23.160704 16276 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.163069 16301 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:23.151897 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:23.146294 11173 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:23.151844 11180 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:23.148456 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.166018 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.166971 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.167332 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.168183 11182 raft_consensus.cc:2410] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:14:23.172983 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.142046 16209 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.178642 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.146350 16304 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:23.141922 16265 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.182842 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.195778 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.237706 11173 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:23.214701 11255 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
W20260709 12:14:23.218739 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46994.4us
I20260709 12:14:23.238855 11255 pending_rounds.cc:85] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 11
I20260709 12:14:23.239145 11255 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:14:23.239379 11255 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:23.239692 11255 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:14:23.239993 11255 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:23.242632 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.243423 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:23.245434 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.245889 11933 raft_consensus.cc:1217] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Deduplicated request from leader. Original: 8.9->[13.10-14.11]   Dedup: 13.10->[14.11-14.11]
I20260709 12:14:23.246249 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.246688 11933 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [13.10-14.11]
I20260709 12:14:23.248306 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.251283 11173 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:23.170778 16255 consensus_queue.cc:1243] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:14:23.247056 11178 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:23.199057 16248 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:23.266835 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2168) took 1620 ms (client timeout 2128 ms). Trace:
I20260709 12:14:23.114370 16337 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.151340 16004 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.091s
W20260709 12:14:23.267381 11252 rpcz_store.cc:269] 0709 12:14:21.646803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.654508 (+  7705us) service_pool.cc:224] Handling call
0709 12:14:23.264024 (+1609516us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:23.266770 (+  2746us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825512192}
I20260709 12:14:23.268573 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:23.269392 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:23.270175 11176 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:23.271597 11930 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:14:23.151901 16317 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 12:14:23.275305 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:23.273125 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.260258 16265 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.167253 16049 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.253420 16304 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:23.177402 16253 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.242566 16258 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 12:14:23.313251 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.314663 16280 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:23.290863 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.292697 16334 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:23.294857 16148 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:14:23.304478 16299 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:14:23.306610 16234 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.309121 16049 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:23.309290 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.371712 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.316772 16300 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:23.321795 16333 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:23.322217 15937 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:23.290225 16344 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:14:23.318516 16244 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:23.322399 16337 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.323004 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.317304 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:23.317812 16245 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:23.327333 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:23.333294 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:23.392095 11255 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:23.356916 16334 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:23.335345 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:23.336115 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:23.364125 16148 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:23.342561 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:23.343430 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.349697 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:23.350297 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.352030 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.352870 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.354261 11252 raft_consensus.cc:2410] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:14:23.359272 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2161) took 1849 ms. Trace:
I20260709 12:14:23.280674 16301 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:14:23.284955 16340 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.246s	user 0.004s	sys 0.000s
W20260709 12:14:23.361116 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 121476us
I20260709 12:14:23.267179 16248 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:23.412885 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70411.7us
I20260709 12:14:23.371943 11251 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:14:23.372510 11248 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:23.373904 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.286672 16098 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:23.272854 16209 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:14:23.383349 11254 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:14:23.384450 16333 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:23.396381 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69632us
W20260709 12:14:23.396461 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79228.1us
I20260709 12:14:23.337296 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.403457 11926 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:23.363255 16275 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:14:23.400216 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:23.285727 16342 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 12:14:23.406987 11173 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:23.405368 11250 rpcz_store.cc:276] 0709 12:14:21.509621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.540540 (+ 30919us) service_pool.cc:224] Handling call
0709 12:14:21.575930 (+ 35390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.172123 (+596193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.358959 (+1186836us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54000f8cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:23.359195 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369425664}
I20260709 12:14:23.395534 11176 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:23.508347 11256 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:23.408589 16301 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.467425 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.482422 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.387895 15937 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.435034 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:23.529623 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.435662 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.521696 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:23.538836 11182 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:23.538065 11179 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:23.538137 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:23.509300 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.483063 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.539553 11325 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
I20260709 12:14:23.505306 11255 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:23.541532 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.453485 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:23.542755 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.546743 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.435199 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.544965 11256 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:23.552726 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 117892us
I20260709 12:14:23.509764 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.573499 11328 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:23.574920 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.406739 16299 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:23.578528 11934 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:23.579344 11329 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:23.582711 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.409726 16337 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:23.467077 15900 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:23.526468 15950 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:23.595772 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.454080 16268 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.597275 11254 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:23.598161 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.439633 16300 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:23.453840 15992 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.433811 16098 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.419920 16248 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:23.577015 11256 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:14:23.576668 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.622189 11325 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:23.622596 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:23.623320 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.624681 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.624948 11256 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:23.625952 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.596689 11255 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:14:23.536603 16342 consensus_queue.cc:1243] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:14:23.432515 16057 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.431288 16028 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.630323 16028 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.631198 15937 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.632587 16028 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.633420 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.635105 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.547312 16301 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:23.639389 16333 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.602232 11252 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:23.604216 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.588191 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.642746 11250 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:23.431900 16354 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.162s
W20260709 12:14:23.647855 16351 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:14:23.578966 16273 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.651197 16273 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.587651 16253 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.654743 16346 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:23.655272 16346 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:23.656378 16346 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.613220 16091 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:23.658269 16091 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.661088 16091 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:23.662966 16331 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:23.663898 15992 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.664609 11174 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:23.580068 16349 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:23.658296 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 794) took 1268 ms. Trace:
I20260709 12:14:23.667029 11932 rpcz_store.cc:276] 0709 12:14:22.389532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.390240 (+   708us) service_pool.cc:224] Handling call
0709 12:14:22.413434 (+ 23194us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:22.414170 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.658065 (+1243895us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b5400416c50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:23.658228 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17230720}
I20260709 12:14:23.619215 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.663051 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.483693 16276 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.583308 16299 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:23.675335 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.626853 16265 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.610870 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:23.608119 15900 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:23.669989 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 81181.8us
W20260709 12:14:23.673097 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.678682 11174 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:23.673547 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:23.674198 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:23.680778 11174 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:14:23.684835 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74067.5us
I20260709 12:14:23.685577 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.685906 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.686257 11256 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:23.686574 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.687461 16057 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:23.687655 15966 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:23.688654 15966 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:23.687902 16057 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:23.689391 16057 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:23.689584 16300 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:23.690490 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.690709 16300 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.694029 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.599098 16243 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.595963 15950 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:23.687810 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.700904 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:23.697779 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.579891 16332 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:14:23.707144 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:23.708650 11935 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [13.12-15.13]
I20260709 12:14:23.710851 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.710743 16209 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:23.713281 16241 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:23.714071 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.665566 16098 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:23.715077 11256 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:23.748145 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:23.748713 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60960.1us
I20260709 12:14:23.749284 11324 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:23.750718 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.695636 16299 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:23.756341 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.758114 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.753243 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:23.760210 11926 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:23.694693 16253 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.766976 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:23.670747 16349 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.751353 16265 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.752100 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:23.763443 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.618156 16268 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.439220 16304 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.686146 16346 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:23.766165 16245 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:23.720587 16204 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:23.710307 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:23.754786 16028 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.705271 16091 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:23.710311 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:23.754501 15966 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:23.720992 16333 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:23.749330 16243 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.791610 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.793344 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:23.561697 16334 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.798532 11935 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:23.835836 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:23.837802 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.012s	sys 0.023s
W20260709 12:14:23.838403 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.012s	sys 0.023s
I20260709 12:14:23.767781 16004 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:14:23.844471 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51237.7us
I20260709 12:14:23.846383 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.848670 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.771489 16253 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.800544 16244 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:23.830199 16245 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:23.833026 11323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.034s
W20260709 12:14:23.870203 11323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.000s	sys 0.035s
I20260709 12:14:23.834308 11926 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:23.870766 11174 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:23.714090 16241 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.769146 16214 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 12:14:23.774621 11296 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.760s (SENT) [suppressed 4 similar messages]
I20260709 12:14:23.839900 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:23.878158 11255 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:23.878660 11174 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:23.779042 16273 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.798609 15950 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.805889 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 885) took 2315 ms. Trace:
I20260709 12:14:23.706384 16248 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.858932 16049 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:23.887892 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.871387 16333 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:23.894940 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2511) took 1891 ms. Trace:
I20260709 12:14:23.861804 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.896487 11255 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:23.862478 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.857801 16028 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.898043 11930 raft_consensus.cc:2368] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:23.898847 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:23.900182 16028 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.864192 15986 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.831607 16091 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.901295 16091 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:23.904464 16243 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.789498 16265 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.907852 16265 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.908232 16253 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:23.910204 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.832592 16204 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:23.913621 11935 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:23.879465 11931 rpcz_store.cc:276] 0709 12:14:21.485186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.485350 (+   164us) service_pool.cc:224] Handling call
0709 12:14:21.500036 (+ 14686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.091996 (+591960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.800102 (+1708106us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b5400650ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:23.800305 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353147136}
I20260709 12:14:23.895617 11327 rpcz_store.cc:276] 0709 12:14:22.003192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.005813 (+  2621us) service_pool.cc:224] Handling call
0709 12:14:22.532810 (+526997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:22.548770 (+ 15960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.894648 (+1345878us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54001c4dd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:23.894853 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211090688}
I20260709 12:14:23.875761 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:23.774194 16349 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:23.852025 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2013) took 1398 ms. Trace:
I20260709 12:14:23.859558 15937 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:23.870137 16245 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.930439 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.924317 11330 rpcz_store.cc:276] 0709 12:14:22.453080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.069441 (+616361us) service_pool.cc:224] Handling call
0709 12:14:23.069878 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.070076 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:23.247557 (+177481us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.851450 (+603893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:23.851543 (+    93us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:23.851616 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.851723 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.851927 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25874,"prepare.queue_time_us":247353,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":12199,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5986,"thread_start_us":708849,"threads_started":2,"wal-append.queue_time_us":651667}
I20260709 12:14:23.888721 16049 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.882740 15984 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:14:23.912201 16121 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:14:23.805500 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 95248.1us
I20260709 12:14:23.931120 11327 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:23.951591 11256 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:14:23.932493 16245 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:23.943572 16049 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:23.926373 16253 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:23.938055 16204 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:23.938984 16004 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:23.941408 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.971174 11330 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:23.924983 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.973181 11326 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:23.933065 15992 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:23.934990 16358 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.303s	user 0.002s	sys 0.001s
I20260709 12:14:23.955782 16320 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:23.942422 16091 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.013731 16091 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.959909 11256 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:23.960846 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:24.015174 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54396.1us
I20260709 12:14:24.016777 11328 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:14:24.018647 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.019330 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.997258 15984 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:24.021620 11256 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:14:23.972630 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:23.975697 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.976506 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:23.959100 16202 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.979760 16320 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:23.984470 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.036213 11178 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:23.986829 16255 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:23.991181 16241 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.992049 15900 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:23.977134 16299 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:23.997891 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.998054 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.000975 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:24.047775 11249 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:24.004424 16251 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:24.006558 16248 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.008489 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.025864 11330 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 12:14:24.026243 11327 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:14:24.033707 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.013851 16148 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:23.971498 16354 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
W20260709 12:14:24.046072 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 271456us
W20260709 12:14:24.056494 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48107.6us
I20260709 12:14:24.052151 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:24.037657 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65088.3us
I20260709 12:14:24.061978 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.056151 11933 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:24.034776 15937 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.043069 16255 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:24.042757 16265 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:24.063918 11252 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:24.059341 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.062024 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:24.070614 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:24.075290 11178 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:24.051153 16251 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:24.037529 16253 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:14:24.055373 16091 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.100644 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:24.103307 11327 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:24.082051 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.039144 16202 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.038810 16057 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.104379 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:24.100593 11148 leader_election.cc:400] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:24.109329 11148 leader_election.cc:403] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:14:24.106007 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:24.061362 16367 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:24.043483 15984 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.076501 16265 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.133275 11176 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:24.076936 16304 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:24.073591 16148 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:24.111745 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 201621us
I20260709 12:14:24.101895 16333 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:24.148173 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:24.148941 11327 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:24.114571 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:24.151252 11324 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:14:24.080910 16043 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:24.082481 15966 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:24.077610 15900 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.145601 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.115104 15992 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:24.152936 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:24.108068 16241 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:24.150945 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.106462 16203 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:24.077241 16004 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:14:24.111763 16313 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:14:24.147704 11257 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:24.168766 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.113353 15937 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:24.162715 16043 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:24.199838 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.081048 16301 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.176884 15992 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.180117 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.077289 16121 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:14:24.113205 15822 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:14:24.133224 16105 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:24.170071 16319 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:24.110967 16091 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.198738 16304 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:24.176258 16241 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:24.210588 16275 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:24.212819 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2513) took 2209 ms. Trace:
I20260709 12:14:24.212893 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.195149 16004 consensus_queue.cc:1243] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:14:24.189028 16214 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:24.169824 15900 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.196365 15937 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.136790 16265 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:24.200835 16308 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:24.203248 16043 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:24.222530 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105012us
I20260709 12:14:24.208865 15992 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:24.212714 16368 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:14:24.220563 16253 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.226477 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:24.211485 16312 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:24.187640 16375 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:14:24.213965 16332 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.224804 16241 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:24.244243 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:24.204962 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 152890us
I20260709 12:14:24.229966 11322 rpcz_store.cc:276] 0709 12:14:22.003863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.075838 (+ 71975us) service_pool.cc:224] Handling call
0709 12:14:23.058990 (+983152us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:14:23.062871 (+  3881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.063343 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:23.323201 (+259858us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b180035d100. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec66ff5e5 00007f3ec66ff568
0709 12:14:23.585219 (+262018us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.859467 (+274248us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.861407 (+  1940us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.872354 (+ 10947us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:23.872719 (+   365us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.944335 (+ 71616us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b180035d100. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:23.947745 (+  3410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.947820 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.212575 (+264755us) raft_consensus.cc:1685] finished
0709 12:14:24.212596 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.212749 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3837,"prepare.queue_time_us":218977,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":87,"raft.queue_time_us":78218,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":15677,"spinlock_wait_cycles":67953408,"thread_start_us":509640,"threads_started":2,"wal-append.queue_time_us":344849}
I20260709 12:14:24.241276 16385 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.220404 16203 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:14:24.251792 16098 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.215556 16337 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:24.248088 16299 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.232302 16308 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:24.254925 16202 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:24.237730 15937 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.231474 16028 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.269022 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:24.279644 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69173.7us
I20260709 12:14:24.287117 11322 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:24.254889 16301 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:24.260414 16214 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.258652 16333 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:24.295372 11331 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:14:24.298303 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:24.299947 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.287855 16203 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:14:24.289808 16369 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:14:24.316991 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:24.257968 16253 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.331972 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:24.273329 16004 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:14:24.335031 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2568) took 1003 ms. Trace:
I20260709 12:14:24.336681 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:24.335110 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:24.338102 11322 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:24.300092 16299 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:24.332901 15966 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:24.337420 11179 rpcz_store.cc:276] 0709 12:14:23.331866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.567772 (+235906us) service_pool.cc:224] Handling call
0709 12:14:23.568260 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.595972 (+ 27712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.596027 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.607948 (+ 11921us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.955885 (+347937us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:23.955926 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.956033 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.334791 (+378758us) raft_consensus.cc:1685] finished
0709 12:14:24.334813 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.334965 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78289,"raft.queue_time_us":23091,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":10964,"thread_start_us":241373,"threads_started":1,"wal-append.queue_time_us":375890}
I20260709 12:14:24.326376 16244 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.308116 16057 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:24.327317 16304 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:14:24.336673 16346 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
W20260709 12:14:24.334338 16366 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:14:24.360267 15937 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.337891 16203 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 12:14:24.362740 16378 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.144s	user 0.002s	sys 0.000s
I20260709 12:14:24.364794 16203 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:24.350893 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.352216 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:24.371784 11322 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:24.358265 16376 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:14:24.358536 16379 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:14:24.358974 16333 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.365341 16203 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:24.285899 16385 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:24.365566 16381 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:24.379542 16245 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.381068 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2219) took 1236 ms. Trace:
I20260709 12:14:24.381479 11177 rpcz_store.cc:276] 0709 12:14:23.145057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.145196 (+   139us) service_pool.cc:224] Handling call
0709 12:14:23.165016 (+ 19820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.174592 (+  9576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.174625 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.009147 (+834522us) spinlock_profiling.cc:243] Waited 989 us on lock 0x7b2c00013008. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:24.009607 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.163678 (+154071us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:24.163909 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.164123 (+   214us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.380737 (+216614us) raft_consensus.cc:1685] finished
0709 12:14:24.380761 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.380980 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11511,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":4011,"spinlock_wait_cycles":2274688,"thread_start_us":103929,"threads_started":1,"wal-append.queue_time_us":200009}
I20260709 12:14:24.383476 16043 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.348978 15966 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.388078 16049 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.376222 16057 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:24.392103 15900 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:24.394394 16349 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.338372 16301 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:14:24.398478 15937 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.400136 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2569) took 1068 ms. Trace:
I20260709 12:14:24.401079 16098 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:24.401726 16372 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:14:24.400568 11182 rpcz_store.cc:276] 0709 12:14:23.332102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.585216 (+253114us) service_pool.cc:224] Handling call
0709 12:14:23.590269 (+  5053us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:23.669379 (+ 79110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.669412 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:23.717059 (+ 47647us) log.cc:844] Serialized 52 byte log entry
0709 12:14:23.957471 (+240412us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:23.957513 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.959805 (+  2292us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.399888 (+440083us) raft_consensus.cc:1685] finished
0709 12:14:24.399909 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.400069 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31603,"raft.queue_time_us":79324,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":18540,"thread_start_us":168053,"threads_started":1,"wal-append.queue_time_us":290215}
I20260709 12:14:24.403297 11934 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:14:24.403475 16384 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:24.346673 16388 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:24.406874 16253 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.407877 15950 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.341719 15984 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:24.374769 16299 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.339908 16382 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:14:24.397360 16349 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:24.419315 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 68504.3us
I20260709 12:14:24.420653 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.348347 16308 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.424507 16308 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:24.405102 16385 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.430047 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.430859 11933 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:24.432301 11932 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:24.432381 16382 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:24.433152 16382 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:24.434209 16382 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.435294 16382 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:24.438261 16043 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.442592 11934 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:24.447278 11322 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:24.448547 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:24.450224 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 115247us
I20260709 12:14:24.450623 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2016) took 1974 ms. Trace:
I20260709 12:14:24.454310 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.441226 16265 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.459216 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.453207 11323 rpcz_store.cc:276] 0709 12:14:22.475709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.094850 (+619141us) service_pool.cc:224] Handling call
0709 12:14:23.095198 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.284406 (+189208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.284456 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.324707 (+ 40251us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.870645 (+545938us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:23.870689 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.443704 (+573015us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:24.443756 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.443819 (+    63us) raft_consensus.cc:1685] finished
0709 12:14:24.443832 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.450526 (+  6694us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11381,"raft.queue_time_us":1667,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":434146176,"thread_start_us":545515,"threads_started":1,"wal-append.queue_time_us":656773}
I20260709 12:14:24.485596 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:24.446321 15900 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:24.455826 16242 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:14:24.491580 16204 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.493536 11323 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
W20260709 12:14:24.493820 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.484107 16395 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:24.500734 16098 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:24.502187 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:24.506371 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 47659.2us
I20260709 12:14:24.455091 16304 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:24.527626 11326 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:14:24.529949 11326 pending_rounds.cc:85] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:14:24.530211 11326 pending_rounds.cc:107] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:14:24.530460 11326 raft_consensus.cc:2889] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:24.507938 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 445955us
I20260709 12:14:24.484349 16367 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.541476 16304 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:24.535467 16301 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.546666 16304 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.538034 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:24.549602 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:24.551455 11324 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:14:24.553557 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:24.555157 16337 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:24.555946 16265 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:24.557739 16390 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.557986 16304 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:24.558020 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:24.558961 11934 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:24.552871 16395 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:14:24.560002 16337 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:24.560493 16395 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:24.560892 16395 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:24.524822 15992 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.563663 15992 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:24.565548 16209 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:24.537604 16241 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.550540 11077 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:24.567157 16313 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:24.567662 16313 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:24.567986 16241 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:24.568598 16313 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:24.570734 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.574512 16233 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.574659 16320 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.575069 16233 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.575816 16320 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:24.578258 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.578009 16395 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:24.579130 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 334961us
I20260709 12:14:24.580467 16313 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:24.582996 11178 raft_consensus.cc:1217] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 8.9->[13.10-14.11]   Dedup: 13.10->[14.11-14.11]
I20260709 12:14:24.583611 11178 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [13.10-14.11]
I20260709 12:14:24.586004 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.587198 16386 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:24.588045 16386 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:24.588195 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.588557 16386 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:24.588917 11178 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.587383 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.535625 16333 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:14:24.598440 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.599202 11177 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:24.600544 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.611236 11179 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:24.620325 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.621789 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.629429 16203 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.631306 16388 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:24.573175 16251 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.634241 16251 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.641428 11178 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:24.643612 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.645674 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:24.647747 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.648666 11177 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:24.650076 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.650661 11177 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.651014 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:24.650656 16349 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.652369 16349 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.654247 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.654824 11182 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.654953 16349 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.659693 16242 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:24.660293 16242 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.661801 16276 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.662364 16276 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.663051 16242 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:24.642228 16281 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.665069 16399 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.692346 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2536) took 1129 ms. Trace:
I20260709 12:14:24.692476 16399 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.668869 11177 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:24.692742 11325 rpcz_store.cc:276] 0709 12:14:23.563083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.630634 (+ 67551us) service_pool.cc:224] Handling call
0709 12:14:23.630982 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.875589 (+244607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.875623 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.886875 (+ 11252us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.077045 (+190170us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:24.077096 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.690969 (+613873us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400428bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:24.691027 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.691072 (+    45us) raft_consensus.cc:1685] finished
0709 12:14:24.691084 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.692280 (+  1196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":75875,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4676,"spinlock_wait_cycles":555316608,"thread_start_us":176777,"threads_started":1,"wal-append.queue_time_us":213247}
W20260709 12:14:24.671550 16355 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.147s	user 0.001s	sys 0.002s
I20260709 12:14:24.694944 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:24.696741 11325 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:24.674712 11181 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:24.675190 16386 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.664222 16233 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.706099 16301 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:24.708060 16386 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.678452 16255 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.687492 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:24.684240 16091 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:24.665768 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.686736 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:24.691561 11182 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:24.717978 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:24.691655 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.719859 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.508338 16389 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:14:24.694944 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.651744 11324 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:24.673702 15986 raft_consensus.cc:993] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: : Instructing follower 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:14:24.672920 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.729496 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.694413 16257 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:24.734105 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:24.713400 11327 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:24.689543 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:24.738322 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:24.714190 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:24.692719 16234 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:14:24.748879 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59403us
I20260709 12:14:24.705389 11934 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:24.751379 11327 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:14:24.721469 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.752544 11933 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:24.726450 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.753757 11177 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:24.703482 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 884) took 1012 ms. Trace:
I20260709 12:14:24.754166 11329 rpcz_store.cc:276] 0709 12:14:23.691453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.691646 (+   193us) service_pool.cc:224] Handling call
0709 12:14:24.100292 (+408646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.100537 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.100575 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.100632 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:24.100678 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.100747 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.703243 (+602496us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b5400131fd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:24.703399 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938904960}
I20260709 12:14:24.734064 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:24.712199 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.752784 11327 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:24.725977 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.755434 11178 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:24.756950 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:24.770442 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:24.766885 11930 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:24.772169 11177 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:24.724735 16233 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:24.775027 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:24.775039 11935 raft_consensus.cc:1217] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Deduplicated request from leader. Original: 8.9->[13.10-14.11]   Dedup: 13.10->[14.11-14.11]
I20260709 12:14:24.779851 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.775204 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:24.724886 15984 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:24.782637 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:24.783193 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:24.735074 16382 raft_consensus.cc:1081] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Signalling peer 016411114c6d42039bd04025378dfc98 to start an election
I20260709 12:14:24.788659 11177 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:24.788836 11935 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [13.10-14.11]
I20260709 12:14:24.790408 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.750474 16234 consensus_queue.cc:1243] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 10.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:14:24.748245 16242 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:24.701828 16281 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.799937 11177 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:24.795614 11174 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:24.798607 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.774365 16346 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:24.770344 16276 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.780616 16386 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:24.806185 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 789) took 1351 ms. Trace:
I20260709 12:14:24.731120 16390 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:14:24.814031 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:24.788798 15986 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:24.788698 16105 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:24.817914 11180 rpcz_store.cc:276] 0709 12:14:23.454397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.684081 (+229684us) service_pool.cc:224] Handling call
0709 12:14:23.684487 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:23.747982 (+ 63495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.748017 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:23.768657 (+ 20640us) log.cc:844] Serialized 52 byte log entry
0709 12:14:24.021130 (+252473us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:24.050161 (+ 29031us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.050405 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.050488 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.708120 (+657632us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b1800424540. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:24.798301 (+ 90181us) raft_consensus.cc:1685] finished
0709 12:14:24.805974 (+  7673us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.806125 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29979,"prepare.queue_time_us":30250,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":64812,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":20966,"spinlock_wait_cycles":12933888,"thread_start_us":218646,"threads_started":1,"wal-append.queue_time_us":279474}
I20260709 12:14:24.750250 16257 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.824961 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.795095 16389 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:24.860929 11179 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:24.788046 16308 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:14:24.806062 15992 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:24.847083 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48953.6us
I20260709 12:14:24.774138 16004 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:14:24.798823 16375 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:14:24.892768 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.794906 15984 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.906548 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:24.912825 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:24.791514 16233 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.859529 16281 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:24.826116 16057 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 12:14:24.891696 15986 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:24.815755 16214 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:14:24.890667 16386 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:24.919112 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 117514us
I20260709 12:14:24.926417 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:24.931221 11930 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
W20260709 12:14:24.935410 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.937705 11251 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:24.938110 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:24.906170 15992 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:24.913157 16204 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:24.952322 11249 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:14:24.952729 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:24.953159 15984 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:24.953701 11250 raft_consensus.cc:2410] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:14:24.803166 16265 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:14:24.954150 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:24.954910 11255 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:24.942485 16281 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.956007 16281 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.957281 15986 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:24.958408 16281 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:24.911079 16389 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:24.959643 16389 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.952762 11249 pending_rounds.cc:85] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 9
W20260709 12:14:24.963207 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.963249 11249 pending_rounds.cc:107] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:14:24.964600 11249 raft_consensus.cc:2889] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:24.943984 11147 leader_election.cc:400] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:24.944660 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:24.946604 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.967192 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:24.967326 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:24.967818 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:24.951375 16233 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.968412 16257 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:24.939533 16308 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.951839 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2190) took 1267 ms. Trace:
I20260709 12:14:24.969007 16308 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.969372 11252 rpcz_store.cc:276] 0709 12:14:23.684277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.684852 (+   575us) service_pool.cc:224] Handling call
0709 12:14:23.685154 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.069150 (+383996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.951576 (+882426us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54002a67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:24.951764 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870568832}
I20260709 12:14:24.966353 11147 leader_election.cc:403] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:14:24.907449 16105 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:24.983985 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.930074 16386 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:24.981283 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:24.987689 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 43829.2us
I20260709 12:14:24.989049 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:24.940079 16121 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:24.992543 11926 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:24.976449 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:24.940976 16004 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:24.978960 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:24.966117 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:24.942976 15992 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:25.013566 11251 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 12:14:25.010075 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.997372 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:24.964082 16244 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.991508 16121 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:25.016530 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.017264 11252 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:25.015691 11249 raft_consensus.cc:2376] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15. Re-sending same reply.
I20260709 12:14:25.013087 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.018471 11255 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:25.020450 11256 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:25.027706 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:24.995791 16409 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:14:25.023237 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:25.028770 11254 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:24.995972 16285 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:14:24.991663 16410 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:14:24.981812 16299 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.043298 16314 raft_consensus.cc:2764] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:24.994833 16233 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15.
I20260709 12:14:25.051295 11249 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:14:25.048530 16346 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:25.070254 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:25.074183 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.097455 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:25.099803 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 565733us
I20260709 12:14:25.029911 16244 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.054999 16299 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.057387 16280 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 12:14:25.167748 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.171034 16244 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.159400 11324 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:25.162537 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:25.163641 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:25.063395 16300 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 12:14:25.146489 16382 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.177019 11255 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:25.177927 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.182262 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.185374 11254 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:25.187466 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.200618 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.203650 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:25.204471 11252 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:25.206140 16105 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:25.207959 16373 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.029s	user 0.000s	sys 0.002s
W20260709 12:14:25.213750 11328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 888) took 1520 ms (client timeout 1579 ms). Trace:
W20260709 12:14:25.214280 11328 rpcz_store.cc:269] 0709 12:14:23.693416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.747818 (+ 54402us) service_pool.cc:224] Handling call
0709 12:14:25.213463 (+1465645us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54000e5954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:25.213675 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613206400}
W20260709 12:14:25.216850 16408 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:14:25.194464 16367 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.123260 16121 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.218582 16367 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.219928 11330 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:14:25.181150 16346 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.220774 16367 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:25.224992 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.205364 16397 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.227070 16308 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.230343 16234 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:25.230332 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.231895 16242 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.233235 16299 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:25.195116 16382 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.241726 11328 raft_consensus.cc:1217] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 6.2->[8.3-11.5]   Dedup: 8.3->[9.4-11.5]
I20260709 12:14:25.252486 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.252688 11250 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:25.276669 11328 pending_rounds.cc:85] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 3
I20260709 12:14:25.277007 11328 pending_rounds.cc:107] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 10.4
I20260709 12:14:25.277245 11328 raft_consensus.cc:2889] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:25.256188 11255 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
W20260709 12:14:25.258661 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 200099us
I20260709 12:14:25.272195 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.280074 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:25.273176 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.273938 11252 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:25.273999 11324 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:25.281944 11253 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:14:25.254968 16251 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.004s
I20260709 12:14:25.283502 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:25.283473 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:25.274394 16242 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:25.280421 15986 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:25.290391 15986 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.293365 15986 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.296403 16388 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:25.296850 16388 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:25.297986 16312 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:25.298424 16312 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:25.299331 16312 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.301169 16312 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:25.280706 16248 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:25.302718 16308 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:25.303349 16308 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:25.304332 16308 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.312693 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:25.270767 16234 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:25.314297 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:25.273552 16057 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:25.324889 16233 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:25.329077 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:25.326300 16402 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:25.320374 16057 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.332705 16057 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.333854 16349 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:25.315886 16248 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:25.339752 16057 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:25.334566 16280 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:14:25.340446 16057 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.341744 16382 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:25.318540 16268 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:25.342558 16234 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.343816 16234 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:25.344272 16057 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.334347 16349 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.355111 16349 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:25.363423 16402 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:25.364446 16304 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:25.364576 16402 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.273263 16397 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:25.391438 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.393388 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:25.394948 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:25.399015 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:25.403570 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74576.4us
I20260709 12:14:25.403312 16233 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:25.407238 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.410274 11249 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:25.338492 16248 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:25.424331 11179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:25.425166 11179 raft_consensus.cc:606] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:25.404758 16334 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:25.405419 16382 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:25.432655 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.433539 11255 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:25.317651 16388 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:25.426760 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 155596us
I20260709 12:14:25.407802 16057 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:25.410274 16233 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:25.416808 16248 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:25.436443 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:25.413934 16367 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:25.424840 16202 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.441320 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.445488 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.439399 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.449852 11252 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:25.412423 16280 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:14:25.413425 16389 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:14:25.468256 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:25.468107 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:25.469816 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:25.478277 11325 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:25.478662 11330 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:25.442634 16233 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:25.469971 11253 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:25.438478 16390 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:25.484440 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.468966 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.423032 16314 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:25.472324 11255 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:25.440382 16375 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.483688 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:25.495384 11329 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:25.491128 11255 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:14:25.489277 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:25.449103 16433 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.503229 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:25.473281 16382 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.505991 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:25.508832 11330 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:25.513885 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.521727 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:25.475068 16202 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.529042 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:25.440426 16244 consensus_queue.cc:1175] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 17 status { last_received { term: 17 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 17 index: 9 } }
I20260709 12:14:25.511921 16433 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.514856 16299 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:25.531067 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.483765 16428 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:25.536118 16202 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:25.536649 11324 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:25.502508 16375 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:25.539532 16426 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:14:25.547047 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.518679 16367 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:25.519997 16209 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:25.519806 16308 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:14:25.537460 16423 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:14:25.561592 11330 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:25.561290 16367 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:25.554634 15992 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.004s
I20260709 12:14:25.570885 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.573892 11330 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:25.521057 16382 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.607245 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:25.608194 11330 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:14:25.602339 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.609125 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:25.613919 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 177523us
I20260709 12:14:25.614466 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 699) took 1666 ms. Trace:
I20260709 12:14:25.614718 11330 raft_consensus.cc:2410] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:25.615010 11257 rpcz_store.cc:276] 0709 12:14:23.948379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.950083 (+  1704us) service_pool.cc:224] Handling call
0709 12:14:23.967765 (+ 17682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.193226 (+225461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.614176 (+1420950us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54001ce8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:25.614388 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413338880}
I20260709 12:14:25.561436 16314 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:25.575524 16367 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:25.577761 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.624375 11324 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:25.573348 16300 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:25.637598 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.611245 16202 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.638765 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:25.568492 16308 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.643977 11928 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:14:25.644567 16429 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:14:25.644816 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:25.646016 16375 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:25.651777 16329 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.654800 11177 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
W20260709 12:14:25.634537 16440 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:14:25.552587 16242 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.561929 16428 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.674197 11330 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:25.636556 16397 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:25.677347 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.685814 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.686805 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.688431 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.680034 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.624396 15966 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:25.631554 16346 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:25.608549 15992 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.669669 16308 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:25.690274 11257 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:25.699635 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.632052 16314 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.733160 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.740211 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:25.681519 16276 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:25.682987 16329 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.674710 16202 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:25.747189 16386 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.005s
I20260709 12:14:25.728042 16437 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:25.797541 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:25.727428 15966 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:25.748760 16148 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:25.720026 16312 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:25.658955 16300 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.746585 16304 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:14:25.775616 16367 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:25.790489 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.791859 16245 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.796124 16329 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:25.798296 11933 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:25.799458 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:25.810441 16439 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 12:14:25.811535 16332 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:25.813796 16436 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 12:14:25.801158 16386 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.767396 16299 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.816068 15966 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.770231 16276 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.746620 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:25.869048 11257 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:25.720309 16346 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.721431 16397 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:25.874933 16397 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.856192 16437 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:25.866575 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 126482us
I20260709 12:14:25.884863 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2570) took 2540 ms. Trace:
I20260709 12:14:25.878737 11174 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:25.860117 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:25.889147 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.885460 11176 rpcz_store.cc:276] 0709 12:14:23.332335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.634403 (+302068us) service_pool.cc:224] Handling call
0709 12:14:23.634725 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.133939 (+499214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.872758 (+1738819us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b5400364554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:25.872994 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146110336}
I20260709 12:14:25.804229 16234 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:14:25.888165 16280 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:25.901465 16280 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:25.901949 16280 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:25.902315 16428 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:25.884322 16437 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:25.904570 16437 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:25.904867 11256 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:14:25.905745 16285 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:25.898806 16382 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.907065 11256 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:25.869230 16299 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.909142 16299 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.871230 16332 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:25.911254 16332 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:25.911844 11928 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:25.914660 16313 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:25.915098 16313 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:25.915989 16313 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:25.918118 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2541) took 2312 ms. Trace:
I20260709 12:14:25.918665 11331 rpcz_store.cc:276] 0709 12:14:23.605953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.636084 (+ 30131us) service_pool.cc:224] Handling call
0709 12:14:23.636385 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.306240 (+669855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.917837 (+1611597us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b5400459cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:25.918060 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514716544}
I20260709 12:14:25.896395 16442 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:14:25.918251 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 815) took 1093 ms. Trace:
I20260709 12:14:25.900090 15992 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:25.869789 16433 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:25.877578 16209 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:25.925347 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:25.926468 11325 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:25.927417 11257 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Deduplicated request from leader. Original: 9.8->[11.9-13.10]   Dedup: 11.9->[13.10-13.10]
I20260709 12:14:25.932300 11257 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 12:14:25.924660 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:25.929327 11930 rpcz_store.cc:276] 0709 12:14:24.824507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.824654 (+   147us) service_pool.cc:224] Handling call
0709 12:14:25.918055 (+1093401us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b5400673ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:25.918164 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88628992}
I20260709 12:14:26.003327 16202 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.004863 16300 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.004992 16234 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:25.897383 16312 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:26.006795 16367 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.007234 16314 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.009333 15966 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.013172 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:25.917430 16313 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:26.018055 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92771.3us
I20260709 12:14:26.033087 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:26.035423 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:25.914673 16397 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.039976 11077 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 8 to 12, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 12 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.031703 15900 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:26.030974 16202 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:14:26.060127 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 126656us
I20260709 12:14:26.057201 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:26.065071 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.065762 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.067703 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.068439 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.069346 11933 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:26.070248 11077 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 7 to 14, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.071107 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:26.066085 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:26.079339 11256 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:26.081420 11254 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:14:26.055212 15900 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:26.136636 11079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 243) took 36601 ms (client timeout 15000 ms). Trace:
I20260709 12:14:26.147230 16386 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.153789 16242 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.155120 16049 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.158594 16399 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.113138 16314 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:14:26.159478 11933 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:26.112703 15992 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:26.160437 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:26.160974 15992 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.161550 15992 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:26.162694 16395 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:26.164081 16148 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:26.164546 16148 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:26.170094 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:26.170899 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:26.171891 16367 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.172573 16367 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:26.174333 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.174929 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.175724 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2229) took 1106 ms. Trace:
I20260709 12:14:26.176255 11249 rpcz_store.cc:276] 0709 12:14:25.069491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.069729 (+   238us) service_pool.cc:224] Handling call
0709 12:14:26.175435 (+1105706us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400108954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:26.175643 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716157184}
I20260709 12:14:26.177989 11251 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:26.070881 16402 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:26.181133 16402 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.184295 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.186447 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 890) took 2487 ms. Trace:
I20260709 12:14:26.186771 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.187368 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2026) took 2150 ms. Trace:
I20260709 12:14:26.188217 11932 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:14:26.187764 11322 rpcz_store.cc:276] 0709 12:14:24.037181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.503081 (+465900us) service_pool.cc:224] Handling call
0709 12:14:24.527319 (+ 24238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.577110 (+ 49791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.577144 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.577182 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:24.577268 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.187156 (+1609888us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b54000c7bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:26.187200 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.187313 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113982336}
I20260709 12:14:26.083256 16273 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:26.189767 16273 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:26.190276 16273 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:26.185366 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:26.192780 16346 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.193583 16346 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:26.186837 11326 rpcz_store.cc:276] 0709 12:14:23.699096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.977227 (+278131us) service_pool.cc:224] Handling call
0709 12:14:23.977474 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.530891 (+553417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.186178 (+1655287us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54000c71d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:26.186378 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264341888}
I20260709 12:14:26.108618 16202 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:26.199970 11928 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:26.175927 11933 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
W20260709 12:14:26.137079 11079 rpcz_store.cc:269] 0709 12:13:49.535201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.535438 (+   237us) service_pool.cc:224] Handling call
0709 12:14:26.136380 (+36600942us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:15.752487 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:15.753217 (+   730us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:15.753276 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:14:15.753298 (+    22us) tablet.cc:662] Decoding operations
0709 12:14:16.055758 (+302460us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:16.103421 (+ 47663us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:16.103455 (+    34us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:14:16.374340 (+270885us) tablet.cc:701] Row locks acquired
0709 12:14:16.388835 (+ 14495us) write_op.cc:260] PREPARE: finished
0709 12:14:16.389051 (+   216us) write_op.cc:270] Start()
0709 12:14:16.389223 (+   172us) write_op.cc:276] Timestamp: P: 1783599256388973 usec, L: 0
0709 12:14:16.389266 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:14:16.595214 (+205948us) log.cc:844] Serialized 6328 byte log entry
0709 12:14:17.457577 (+862363us) op_driver.cc:464] REPLICATION: finished
0709 12:14:17.667090 (+209513us) write_op.cc:301] APPLY: starting
0709 12:14:17.667240 (+   150us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:17.778745 (+111505us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:17.793876 (+ 15131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:19.049956 (+1256080us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:19.175797 (+125841us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:14:19.175911 (+   114us) write_op.cc:312] APPLY: finished
0709 12:14:19.225620 (+ 49709us) log.cc:844] Serialized 148 byte log entry
0709 12:14:19.309401 (+ 83781us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:19.309967 (+   566us) write_op.cc:454] Released schema lock
0709 12:14:19.327417 (+ 17450us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:19.366045 (+ 38628us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:19.366074 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":491975,"reported_tablets":50,"spinlock_wait_cycles":15104,"child_traces":[["op",{"apply.queue_time_us":208353,"apply.run_cpu_time_us":91429,"apply.run_wall_time_us":1699256,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71135,"num_ops":16,"prepare.queue_time_us":615795,"prepare.run_cpu_time_us":42283,"prepare.run_wall_time_us":1493659,"raft.queue_time_us":153217,"raft.run_cpu_time_us":18545,"raft.run_wall_time_us":129724,"replication_time_us":1068126,"spinlock_wait_cycles":4352,"thread_start_us":1306213,"threads_started":5,"wal-append.queue_time_us":723607,"wal-append.run_cpu_time_us":21172,"wal-append.run_wall_time_us":2239836}]]}
I20260709 12:14:26.211651 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:26.212083 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:26.212647 11326 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:26.213322 11322 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:26.134900 11077 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 6 to 12, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 12 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.217095 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:26.134093 16248 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.002s
I20260709 12:14:26.221480 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.223838 11256 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
W20260709 12:14:26.227631 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.235837 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:26.236922 11322 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:26.210685 16121 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:26.195227 16202 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:26.224951 16349 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.869s	user 0.012s	sys 0.005s
I20260709 12:14:26.252138 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:26.255836 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 198752us
W20260709 12:14:26.259603 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.260069 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:26.261209 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:26.264317 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:26.264199 11077 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 5 to 10, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 10 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.219238 16399 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:14:26.165521 16148 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.245932 16459 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:26.286767 11323 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:14:26.287494 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.239777 16028 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:14:26.285074 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:26.290671 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.234920 16248 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.280164 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:26.300791 11251 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:26.282564 11077 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 10 to 13, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 13 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.277655 16390 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:26.306021 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.301720 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:26.245587 15900 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:26.297516 16334 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.862s	user 0.012s	sys 0.002s
I20260709 12:14:26.289920 16242 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:26.317878 16242 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.318122 16332 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:26.318193 16367 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.255955 16349 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.323035 11926 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:26.322793 11178 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:14:26.322906 11077 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 10, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 10 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.302436 16148 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:26.301978 16319 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:26.307377 16390 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.310811 16386 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.352782 11077 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.355064 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:26.329852 15949 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.294433 16457 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:14:26.363013 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:26.364871 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:26.366331 11077 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 9 to 12, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 12 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.320525 16242 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.370924 11251 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:26.372444 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.373215 11173 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:14:26.334153 16334 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:26.299145 16459 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.374342 11077 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 5 to 12, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 12 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.349085 16402 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.356086 16390 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:26.373342 16233 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.879s	user 0.013s	sys 0.003s
I20260709 12:14:26.378304 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.378734 11077 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 2 to 13, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 13 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.379740 11182 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:26.388585 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.406101 11173 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:26.353615 15900 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:26.400426 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.407579 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.407583 11077 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 11 to 12, leader changed from <none> to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 12 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.387948 11179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47"
dest_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:45566
I20260709 12:14:26.417189 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.365468 16367 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:14:26.424937 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.417980 11175 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:26.351325 16462 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 12:14:26.427066 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:26.405089 16308 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.004s
I20260709 12:14:26.422777 11326 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:26.329226 16202 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.356854 15949 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.427826 11176 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:26.364264 16386 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.434599 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.430017 11077 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.428697 16209 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:26.430861 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.357378 16319 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.403770 16234 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:26.434242 16275 raft_consensus.cc:2764] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:26.438450 16248 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.389213 16334 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.401847 16437 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.442694 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.427505 16465 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:26.410472 16233 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.452209 16367 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:26.467368 11182 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:26.487576 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.466720 11077 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.471482 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.489677 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.490528 11175 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 12:14:26.490705 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2558) took 1019 ms. Trace:
I20260709 12:14:26.491427 16319 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.491668 11175 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:26.491218 11330 rpcz_store.cc:276] 0709 12:14:25.471280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.620584 (+149304us) service_pool.cc:224] Handling call
0709 12:14:25.620895 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:25.681738 (+ 60843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.490467 (+808729us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b5400323cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:26.490643 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85128448}
I20260709 12:14:26.479326 16465 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.493510 16465 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:26.477341 11176 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:26.453393 16402 raft_consensus.cc:993] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:14:26.439951 16308 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.426200 11173 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 12:14:26.481832 16248 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:26.494092 16334 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:26.507118 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 142296us
I20260709 12:14:26.508939 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:26.496973 16367 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:26.510427 11182 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:26.509974 11077 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.513672 16233 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.517843 16308 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.517650 11077 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.519363 11077 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.520064 16437 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.500922 16402 raft_consensus.cc:1081] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
W20260709 12:14:26.527431 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 103152us
I20260709 12:14:26.529361 16334 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:26.535462 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.520817 11077 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 9 to 12, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 12 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:14:26.518190 16464 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:14:26.505030 16466 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:26.581552 11077 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 10 to 11, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 11 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.576617 16004 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:26.533319 16415 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.592675 11077 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.540964 16402 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:26.571672 15900 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.577242 16437 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.505329 16248 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.537266 16461 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:26.582111 16395 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:26.581804 16349 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.589834 16004 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.594990 16459 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.622192 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.626874 11256 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:26.624745 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:26.632292 11256 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:26.634835 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.635648 11249 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:26.639402 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.597014 16233 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:14:26.642076 11251 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
W20260709 12:14:26.662021 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.662518 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.644495 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:26.610234 15900 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.646698 16432 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:14:26.649295 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.620697 16395 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.649890 16386 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:26.650467 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.654320 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2251) took 1977 ms. Trace:
I20260709 12:14:26.659320 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:26.680171 11326 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:26.610802 16437 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.670543 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.587421 16466 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.674147 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:26.629417 16202 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.676271 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.689459 11254 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:14:26.682304 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:26.690177 11251 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:26.671592 16280 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.769s	user 0.016s	sys 0.002s
I20260709 12:14:26.596565 16415 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.604810 16402 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.687107 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.702955 11256 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:26.668023 11077 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.678958 11181 rpcz_store.cc:276] 0709 12:14:24.676851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.731912 (+ 55061us) service_pool.cc:224] Handling call
0709 12:14:24.732229 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.941788 (+209559us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:24.981672 (+ 39884us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.982039 (+   367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.982113 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:24.982191 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.654046 (+1671855us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400363654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:26.654093 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.654251 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":73301,"raft.queue_time_us":745,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3388,"spinlock_wait_cycles":490677504}
I20260709 12:14:26.669545 16432 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:26.695235 11257 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:26.695442 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.694460 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:26.615988 16461 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.669293 16299 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.677663 16397 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.723088 11077 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:26.727385 11250 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 11: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:26.695456 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:26.729105 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:26.732596 11251 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:26.740954 11257 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:26.669014 16457 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:14:26.746146 11252 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:26.751946 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.753891 16475 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:14:26.755164 16461 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.755760 16461 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:26.756510 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.641463 16367 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.761161 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:26.764319 11249 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:26.756242 16461 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.767263 11077 catalog_manager.cc:5697] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 1 to 8, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 8 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.775116 11077 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 10 to 13, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 13 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.707645 16280 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.783795 11077 catalog_manager.cc:5697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 3 to 10, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 10 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.777724 16265 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:26.793023 16265 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.795693 16265 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:26.797333 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.798143 16457 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:26.798744 16457 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.801116 16457 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.801776 16432 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.803158 16432 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:26.807279 16461 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.708572 16202 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.809774 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:26.781387 15992 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.619s	user 0.014s	sys 0.000s
I20260709 12:14:26.815325 11257 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 12: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:26.749933 16390 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.751912 16397 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.824431 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.824928 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:26.748626 16299 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.825337 11077 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.829751 11077 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.831282 11077 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 10 to 12, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 12 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:26.833258 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:26.834215 11176 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:26.811443 16457 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:26.839123 11181 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:14:26.839484 16480 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:26.841277 16308 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:26.841944 16308 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.845618 16483 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.848263 16483 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.849052 16402 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.845963 16248 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.850961 16299 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.847405 16390 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.818470 16484 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:14:26.831744 16472 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:14:26.866483 16484 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.867211 16437 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.867604 16202 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.868841 16484 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.831049 16466 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.864013 16457 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.872551 11173 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:26.873226 16442 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.873764 16442 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.876058 16442 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:26.877920 16397 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.872864 16457 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:26.834431 16481 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:14:26.891018 11173 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:26.847615 16004 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.908206 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.908070 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:26.909791 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.918376 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:26.921542 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:26.919359 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:26.864861 16483 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:26.826160 15992 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.892597 16334 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:26.950400 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.898018 16466 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.897841 16397 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.898558 16484 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:26.959430 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:26.961869 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:26.962780 11181 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:26.963883 16390 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:26.965510 16390 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:26.966063 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.966612 16390 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.967602 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:26.968454 11181 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
W20260709 12:14:26.970537 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.971249 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.972478 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 348653us
I20260709 12:14:26.920122 16395 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:26.946452 16333 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:26.974830 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:26.979892 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:26.981555 11330 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:26.983120 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:26.984416 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2204) took 1540 ms. Trace:
I20260709 12:14:26.973079 16397 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:26.989825 11255 rpcz_store.cc:276] 0709 12:14:25.443893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.496573 (+ 52680us) service_pool.cc:224] Handling call
0709 12:14:25.497037 (+   464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:25.504444 (+  7407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.504495 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:25.505404 (+   909us) log.cc:844] Serialized 52 byte log entry
0709 12:14:26.660036 (+1154632us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b180034ec40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:14:26.838421 (+178385us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:26.863692 (+ 25271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.886663 (+ 22971us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.984139 (+ 97476us) raft_consensus.cc:1685] finished
0709 12:14:26.984158 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.984354 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":957,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":10205,"spinlock_wait_cycles":10218880,"thread_start_us":167216,"threads_started":1,"wal-append.queue_time_us":1396624}
I20260709 12:14:26.979434 16437 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:14:26.973505 15950 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:26.954854 16488 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:26.971544 16483 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:27.004906 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:27.007539 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:27.008701 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:27.014104 16495 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:27.014609 16495 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.014707 16349 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:27.014981 16495 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:27.011832 16457 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:14:27.016755 16349 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:26.971076 16466 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:27.017231 16121 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.763s	user 0.010s	sys 0.000s
I20260709 12:14:27.017232 16349 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:26.972421 16484 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.021100 16484 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:27.002668 16389 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:27.023075 16280 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.023384 16483 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:27.024617 16483 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.023084 16389 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:26.976740 16333 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:27.019325 16466 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:27.028530 11929 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:27.031210 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.038764 11326 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:27.039605 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:27.042055 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:27.046545 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.047425 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.064518 11326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:14:27.065214 11326 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:27.029592 16437 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:27.065585 11326 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:27.066838 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.070726 11331 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:27.067378 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:27.072124 11930 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:27.039098 16295 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:27.046535 16148 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.696s	user 0.013s	sys 0.000s
I20260709 12:14:26.980121 16465 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.077504 11322 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:27.079017 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.082391 11331 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
W20260709 12:14:27.094496 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 114676us
I20260709 12:14:27.088908 11326 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.100874 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.106817 11326 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.069487 16437 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:27.072234 16273 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.881s	user 0.012s	sys 0.001s
I20260709 12:14:27.110311 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:27.114779 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:27.112552 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.076222 16389 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:27.107082 16313 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.068s	user 0.012s	sys 0.005s
I20260709 12:14:27.126355 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.127432 11331 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:27.129707 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:27.148080 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.148814 11322 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:27.148922 11173 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:27.116185 16437 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:27.151659 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:27.096316 16295 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.168031 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:27.168789 11255 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:27.169404 16496 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:14:27.171257 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:27.175485 16313 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:27.148972 16300 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:27.118297 16273 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:27.255396 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.149755 16498 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:14:27.258859 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87656.7us
I20260709 12:14:27.119030 16479 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:14:27.107638 16333 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.272687 16333 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:27.273475 16300 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:27.274552 16300 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.275425 16349 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.282943 16273 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.284132 11180 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:27.284788 11179 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:27.285286 11179 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:27.286984 16319 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:27.287546 16319 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:27.288550 16319 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:27.290766 16500 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:14:27.292415 16273 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.293308 16273 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.167166 15992 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.306718 11930 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:27.328197 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:27.163884 16466 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:14:27.331143 11255 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:27.333109 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:27.328815 11250 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:27.334251 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:27.339728 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:27.341007 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:27.341291 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:27.342005 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:27.343235 11175 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:27.345358 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.335714 11257 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:27.271292 16499 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:27.362439 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:27.360774 11179 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:27.380008 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.383019 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:27.278862 16397 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.299116 16390 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:27.390447 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:27.307499 16490 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:27.388710 16466 consensus_queue.cc:1243] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:27.294754 16280 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:14:27.350927 16148 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.378629 16319 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:27.402069 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:27.362972 16499 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.387146 16258 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.410852 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:27.415731 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:27.400606 16466 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.401824 16390 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.417861 16466 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.418358 16465 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.420138 16466 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:27.420496 16390 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.420848 16466 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.421226 16390 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.421422 16466 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.403697 16490 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.423440 16273 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.423805 16466 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.424489 16490 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.412096 15950 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:27.426679 15950 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.428215 16258 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:27.430197 16502 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:14:27.430243 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:27.394884 16498 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:27.430754 16258 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:27.431216 16498 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.431658 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:27.410022 16148 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.432192 11930 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:27.433622 16148 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:27.441022 11930 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:27.433490 16498 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:27.400691 16349 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:27.448740 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:27.449218 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.415215 16499 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:27.449539 11322 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:27.453375 11930 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:14:27.454661 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:27.455291 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76279.5us
I20260709 12:14:27.463830 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:27.466055 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:27.429466 16495 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.470024 16495 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:27.454254 16255 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:27.480491 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:27.482012 11175 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:27.487128 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.487898 11326 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:27.488804 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:27.489002 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.442865 16390 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.490000 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:27.490520 11326 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:27.490689 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:27.491698 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.494803 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:27.480363 16349 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:27.500813 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:27.501109 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:27.499254 16349 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.483098 16258 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:27.505473 11322 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:27.494333 16505 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:27.507426 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:27.511293 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:27.517795 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:27.523065 16390 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:27.531769 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:27.532899 11331 raft_consensus.cc:1217] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[9.6-14.8]   Dedup: 11.7->[14.8-14.8]
I20260709 12:14:27.533504 11331 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: [9.6-14.8]
I20260709 12:14:27.507007 16349 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:27.536995 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:27.509908 16505 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.538820 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:27.556057 11175 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:14:27.558578 11175 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:27.563153 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:27.595113 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:27.625747 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.673768 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.697913 11326 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:27.539531 15984 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:14:27.699496 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 136473us
I20260709 12:14:27.535599 16518 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:27.700676 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:27.699528 16521 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:27.709810 16516 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:14:27.713936 15984 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.725826 16513 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:27.543118 16004 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:27.545413 16505 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:27.782183 16521 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.782238 16148 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.798990 16521 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:27.799543 16258 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:27.794705 16520 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:27.703832 16462 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.804740 11935 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:27.804795 16462 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:27.805797 16248 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.806442 16248 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.834012 16462 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:27.794377 16300 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.838810 16300 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:27.988139 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 287509us
I20260709 12:14:27.992807 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:27.998983 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.985697 16397 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.987224 16520 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.812269 15984 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:27.831033 16148 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.046672 16524 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.047508 16300 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.047327 16524 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.049088 16300 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:28.050093 16524 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:28.018057 16397 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.056703 16397 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.061892 15984 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:28.068023 16499 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
I20260709 12:14:28.018926 16520 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:28.095495 16437 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.930s	user 0.008s	sys 0.003s
I20260709 12:14:28.126896 16437 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.101831 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.128281 11326 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:28.120226 16258 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:28.121778 16004 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:28.129984 16004 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:28.130333 16004 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:28.131536 11323 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:28.131178 16004 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.132441 16004 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:28.074747 16499 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:28.135313 16319 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.725s	user 0.009s	sys 0.002s
W20260709 12:14:28.147509 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.150836 16258 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.152347 16258 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.096623 16148 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:28.127648 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.174332 16258 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:28.188158 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 203804us
I20260709 12:14:28.195109 16520 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.196771 16520 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.201512 16437 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:28.210937 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.135516 16499 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.226429 16528 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:28.227260 16528 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.231014 16528 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:28.240300 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 322304us
I20260709 12:14:28.250186 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.251235 11326 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:14:28.276298 11928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 801) took 2118 ms (client timeout 2037 ms). Trace:
W20260709 12:14:28.277176 11928 rpcz_store.cc:269] 0709 12:14:26.158066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.160188 (+  2122us) service_pool.cc:224] Handling call
0709 12:14:28.275955 (+2115767us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b54004c2554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:28.276191 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88252800}
I20260709 12:14:28.296485 16174 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:28.297142 16174 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:28.298214 16174 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:28.301561 11898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.008s
W20260709 12:14:28.301960 11898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.000s	sys 0.008s
W20260709 12:14:28.298623 11174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2257) took 2660 ms (client timeout 2873 ms). Trace:
W20260709 12:14:28.303678 11174 rpcz_store.cc:269] 0709 12:14:25.638372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.638503 (+   131us) service_pool.cc:224] Handling call
0709 12:14:28.298378 (+2659875us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540016f8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:28.298537 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500607232}
I20260709 12:14:28.303053 16308 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:28.302945 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 310211us
I20260709 12:14:28.307456 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:28.305766 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.002s	sys 0.007s
W20260709 12:14:28.308378 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.002s	sys 0.007s
I20260709 12:14:28.308571 16105 raft_consensus.cc:2764] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:14:28.309252 16105 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:14:28.309612 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:28.309690 16525 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:28.310797 16532 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:14:28.310075 16503 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.038s	user 0.005s	sys 0.028s
I20260709 12:14:28.311223 16525 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.313791 16527 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.312330 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 920) took 1696 ms. Trace:
I20260709 12:14:28.313840 16528 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.314702 16525 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.346367 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.314836 16528 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.347102 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:28.311930 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:28.342275 16203 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:28.348107 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.348678 16528 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:28.314554 11932 rpcz_store.cc:276] 0709 12:14:26.616087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.616256 (+   169us) service_pool.cc:224] Handling call
0709 12:14:26.616601 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:26.626183 (+  9582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.626215 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:26.626257 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:26.626288 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.312116 (+1685828us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b54000429d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:28.312160 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.312273 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21547264}
I20260709 12:14:28.349560 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:28.350215 11329 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.351416 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.353962 16285 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:28.354275 16105 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:28.354068 16533 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:28.354614 16285 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:28.354748 16105 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.355063 16285 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:28.355155 16105 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:28.353312 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:28.322319 16276 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.327688 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.197444 16520 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.358438 16533 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:28.365509 16509 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.092s	user 0.017s	sys 0.050s
I20260709 12:14:28.366608 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.367046 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2275) took 1411 ms. Trace:
I20260709 12:14:28.367558 11174 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:28.381580 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:28.334213 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2224) took 3338 ms. Trace:
I20260709 12:14:28.383469 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:28.324132 16349 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.777s	user 0.007s	sys 0.003s
I20260709 12:14:28.325824 16505 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.314564 16527 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.345847 16532 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.347208 11179 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.308037 16388 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:28.348846 11322 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:28.324132 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.326624 15949 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.396869 16528 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:28.397898 16525 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:28.359091 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.396885 11326 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:28.399439 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:28.330579 16518 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.593s	user 0.008s	sys 0.002s
I20260709 12:14:28.405912 16518 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:28.407292 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.407998 11252 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:28.368543 16520 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.367511 11176 rpcz_store.cc:276] 0709 12:14:26.955977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.956138 (+   161us) service_pool.cc:224] Handling call
0709 12:14:26.956592 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:26.971977 (+ 15385us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:26.972018 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.977227 (+  5209us) log.cc:844] Serialized 26 byte log entry
0709 12:14:27.162404 (+185177us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:27.162466 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.162547 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.364963 (+1202416us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:28.366760 (+  1797us) raft_consensus.cc:1685] finished
0709 12:14:28.366784 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.366990 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14931,"raft.queue_time_us":22627,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":30502,"spinlock_wait_cycles":19296128,"thread_start_us":155589,"threads_started":1,"wal-append.queue_time_us":279984}
I20260709 12:14:28.370635 15900 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:28.416673 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:28.419914 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61933.9us
I20260709 12:14:28.423926 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.424729 11254 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:28.371865 16533 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:28.330837 16471 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:28.377772 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.449118 11250 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:28.378625 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.380203 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 804) took 3647 ms. Trace:
I20260709 12:14:28.332878 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.384224 11928 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:28.449483 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.383803 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.388155 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.389902 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.452296 11253 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:28.392371 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.383734 11248 rpcz_store.cc:276] 0709 12:14:24.995924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.996884 (+   960us) service_pool.cc:224] Handling call
0709 12:14:24.997227 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.997568 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.997595 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.031256 (+1033661us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b18003d5ee0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a
0709 12:14:28.300274 (+2269018us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b2c00000e98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:28.300908 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.333829 (+ 32921us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:28.333874 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.333958 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.334000 (+    42us) raft_consensus.cc:1685] finished
0709 12:14:28.334012 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.334135 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15998,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":10057,"spinlock_wait_cycles":47420672,"thread_start_us":32527,"threads_started":1,"wal-append.queue_time_us":6319}
I20260709 12:14:28.453156 11930 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:28.453899 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:28.395222 11329 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:28.400616 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45566 (request call id 2590) took 2018 ms. Trace:
I20260709 12:14:28.394837 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:28.454557 11930 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:28.400882 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.450620 11178 rpcz_store.cc:276] 0709 12:14:24.733095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.756029 (+ 22934us) service_pool.cc:224] Handling call
0709 12:14:28.379999 (+3623970us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400347bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:28.380145 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3543302912}
I20260709 12:14:28.399205 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:28.398317 16525 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:28.410518 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.410518 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:28.455855 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56738.7us
W20260709 12:14:28.456195 11927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 924) took 1778 ms (client timeout 2116 ms). Trace:
W20260709 12:14:28.410573 11223 connection.cc:256] RPC call timeout handler was delayed by -2.3172312859999238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
I20260709 12:14:28.444828 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.456892 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.417470 16244 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.458792 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:28.453814 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.459206 11174 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:28.456293 11252 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:28.397620 16532 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.454730 11179 rpcz_store.cc:276] 0709 12:14:26.376411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.376563 (+   152us) service_pool.cc:224] Handling call
0709 12:14:28.345518 (+1968955us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54000cdd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e3a3d 00007f3ecb6b3cd0 00007f3ec4c38254 00007f3ec4c381e2 00007f3ec4c38111 00007f3ec4c3808c 00007f3ec4c37342 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:28.394465 (+ 48947us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1997876992}
I20260709 12:14:28.457506 11322 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
W20260709 12:14:28.456846 11223 connection.cc:256] RPC call timeout handler was delayed by -2.2677312859998437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
W20260709 12:14:28.468036 11223 connection.cc:256] RPC call timeout handler was delayed by -2.0427312859999347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
I20260709 12:14:28.457787 11176 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:28.469403 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.452414 11249 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:28.470388 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.472460 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.472728 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.459578 11248 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:28.473698 11251 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:28.474766 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.474928 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.452186 11932 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:28.455099 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:28.476650 11329 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16.
I20260709 12:14:28.460183 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.477804 11250 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:28.461146 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.461763 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.462603 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.479542 11331 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:28.479894 11252 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:28.480206 11252 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:28.480386 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.463465 16504 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:14:28.465010 11296 leader_election.cc:336] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.512s (SENT)
W20260709 12:14:28.456662 11927 rpcz_store.cc:269] 0709 12:14:26.678183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.332598 (+1654415us) service_pool.cc:224] Handling call
0709 12:14:28.456136 (+123538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:28.480971 11252 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.491678 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:28.443344 16295 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.408645 15986 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:28.481240 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:28.502400 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.502862 11176 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:28.503360 11176 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:28.503723 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.504302 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.504736 11249 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.475580 11248 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
W20260709 12:14:28.482914 11935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 814) took 3685 ms (client timeout 1512 ms). Trace:
I20260709 12:14:28.460162 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:28.483407 11223 connection.cc:256] RPC call timeout handler was delayed by -1.8582312859998638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 12:14:28.505860 11935 rpcz_store.cc:269] 0709 12:14:24.797821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.798351 (+   530us) service_pool.cc:224] Handling call
0709 12:14:28.482759 (+3684408us) spinlock_profiling.cc:243] Waited 3 s on lock 0x7b54000a7154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:28.482851 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6905620864}
W20260709 12:14:28.506850 11223 connection.cc:256] RPC call timeout handler was delayed by -1.57945084999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 12:14:28.507256 11223 connection.cc:256] RPC call timeout handler was delayed by -1.5524428909998278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
I20260709 12:14:28.503155 11927 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
W20260709 12:14:28.484190 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.486879 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.508654 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.509096 11257 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.509512 11248 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:28.487967 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:28.488554 11928 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:28.510644 11932 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:28.392258 16349 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.492283 11252 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.492372 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2591) took 2115 ms. Trace:
I20260709 12:14:28.443331 16505 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:28.465623 16455 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 2.334s	user 0.018s	sys 0.015s
W20260709 12:14:28.507668 11223 connection.cc:256] RPC call timeout handler was delayed by -1.5173428909999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
I20260709 12:14:28.505729 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.486769 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.504464 11176 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.483799 16441 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.498646 16451 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.534498 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.550848 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.551843 11329 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:28.534894 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.534497 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.534504 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.542335 11179 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:28.553262 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:28.542500 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72230.7us
I20260709 12:14:28.554529 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.555925 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:28.543275 11223 connection.cc:256] RPC call timeout handler was delayed by -1.4614172809999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 12:14:28.557891 11223 connection.cc:256] RPC call timeout handler was delayed by -1.4289360609998312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 12:14:28.558401 11223 connection.cc:256] RPC call timeout handler was delayed by -1.3791357229999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 12:14:28.558845 11223 connection.cc:256] RPC call timeout handler was delayed by -1.1946357229999194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
I20260709 12:14:28.506105 11174 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:28.536319 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:28.561323 11928 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.546092 11325 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:28.484025 16518 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:28.492117 16004 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:28.491453 16504 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:28.567721 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.512094 16390 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:28.543766 11177 rpcz_store.cc:276] 0709 12:14:26.376796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.376931 (+   135us) service_pool.cc:224] Handling call
0709 12:14:26.398452 (+ 21521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:26.452877 (+ 54425us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:26.452952 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.453545 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.579969 (+126424us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:26.580038 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.480880 (+1900842us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b54002e4fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:28.480927 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.480975 (+    48us) raft_consensus.cc:1685] finished
0709 12:14:28.480987 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.492282 (+ 11295us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31171,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":11662,"spinlock_wait_cycles":124026240,"thread_start_us":96909,"threads_started":1,"wal-append.queue_time_us":146426,"wal-append.run_cpu_time_us":2675,"wal-append.run_wall_time_us":1165494}
I20260709 12:14:28.533892 16385 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:28.535375 16397 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:28.562233 16337 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.565253 16518 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.568707 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.569330 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:28.569459 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.557338 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.592036 11322 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:28.592442 11174 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:28.573305 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.569988 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.482153 16244 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.568261 16505 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:28.578200 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.458930 16471 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.599015 11935 raft_consensus.cc:2410] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:14:28.577968 11253 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:28.597043 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.563410 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.554512 16441 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.588676 11930 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:28.568495 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.567425 16004 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.593526 11329 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:28.594272 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.515046 15986 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.620622 11935 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:14:28.566025 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 75749.2us
I20260709 12:14:28.600410 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.624667 11249 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:28.621616 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.625434 11322 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:28.576893 16542 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:28.601094 11177 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:28.625474 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.585196 16489 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:28.629086 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.629943 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:28.616150 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.629961 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.607410 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.633273 11179 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:14:28.596093 16518 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.634308 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.630188 11329 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:28.640918 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.636122 11932 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:28.636682 11257 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:28.598124 16390 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:28.637171 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:28.639467 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.647114 11930 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:14:28.647312 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.648885 11248 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:28.641119 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.648972 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:28.564390 16540 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 12:14:28.623248 16004 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:28.589457 16385 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.651384 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.651387 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.658769 11257 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:28.659052 11249 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:28.655853 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:28.653563 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.564465 16451 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.656140 16248 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.593329 16397 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.644053 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.653838 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.656770 16312 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:28.665907 16312 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.666556 16548 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:28.622994 16527 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.596181 16105 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.667070 16548 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.667552 16548 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:28.595368 16337 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.643285 16203 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:28.669773 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.624045 16285 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.650736 16390 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.656855 16275 raft_consensus.cc:2764] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:14:28.657887 15984 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:28.664875 11248 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:28.650122 11253 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:28.674081 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.651160 16526 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:28.676084 11329 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 10.
I20260709 12:14:28.666819 16441 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.675420 11252 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:28.687330 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.678861 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.679594 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:28.677575 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 120347us
I20260709 12:14:28.670261 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:28.661293 16540 consensus_queue.cc:1243] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:28.682603 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:28.684887 11356 heartbeater.cc:646] Failed to heartbeat to 127.9.198.254:36219 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.198.254:36219 timed out after 15.000s (SENT)
I20260709 12:14:28.685920 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.686836 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.704888 11249 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:28.643046 16489 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.681167 15984 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.666821 16312 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:28.708498 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:28.682176 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.707301 16489 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:28.712049 11248 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:28.714134 11257 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:28.706228 11329 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:28.678321 16438 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:28.675112 16203 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:28.707420 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.680922 16244 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.680544 16390 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:28.713389 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.710378 15984 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:28.707442 16545 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:28.746526 11928 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:28.748739 16548 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.749246 15986 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.751919 16548 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.725920 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.720144 16275 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:28.722054 16203 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.713850 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:28.720168 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.720244 11252 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:28.724776 11254 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:14:28.765588 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 47504.2us
I20260709 12:14:28.679670 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.726537 16285 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.710124 16532 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.735121 16397 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:28.723960 16349 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.746903 11257 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:14:28.771663 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57905.6us
I20260709 12:14:28.772445 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.708735 11253 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:28.759042 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:28.773545 11331 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:14:28.757210 11249 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:28.715758 11248 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:28.748059 16545 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.766494 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:28.740258 16471 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.723532 16543 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:28.747848 16105 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.756479 16549 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:28.776250 11248 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:28.758551 16385 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.813967 11251 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:28.770529 16457 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:28.773835 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 94226.8us
W20260709 12:14:28.810573 11148 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.658s (SENT) [suppressed 7 similar messages]
I20260709 12:14:28.804862 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.817432 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:28.818369 11928 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:28.821005 11928 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:28.822291 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:28.821807 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 151559us
I20260709 12:14:28.826210 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.772172 16542 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:28.776849 16445 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:28.777640 16548 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:28.828907 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.813784 16532 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:28.811975 16275 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:28.826416 11248 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.805258 16545 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:28.825392 16057 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.827632 16098 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:28.826758 16457 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.839210 11248 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:28.830925 16105 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:28.843307 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:28.840644 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.812222 16004 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.842341 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.844645 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.846688 11257 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:28.845851 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.837276 16532 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.835331 16549 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:28.811206 16543 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.840739 16057 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.855317 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:28.858539 11927 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:28.841734 16457 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:28.898952 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58382.4us
I20260709 12:14:28.884298 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 819) took 2337 ms. Trace:
I20260709 12:14:28.861756 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.853953 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.862273 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.861207 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:28.883200 11325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2043) took 2715 ms (client timeout 1787 ms). Trace:
W20260709 12:14:28.904671 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44066.2us
W20260709 12:14:28.904457 11325 rpcz_store.cc:269] 0709 12:14:26.167353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.169715 (+  2362us) service_pool.cc:224] Handling call
0709 12:14:28.882969 (+2713254us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b5400322dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:28.883084 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5177559296}
I20260709 12:14:28.879781 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2595) took 2500 ms. Trace:
I20260709 12:14:28.884820 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 916) took 3084 ms. Trace:
I20260709 12:14:28.905550 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:28.853315 11249 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:28.842790 16388 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:28.908144 11181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:28.919761 11181 raft_consensus.cc:606] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.905763 11926 rpcz_store.cc:276] 0709 12:14:25.800234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.800638 (+   404us) service_pool.cc:224] Handling call
0709 12:14:25.818621 (+ 17983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:26.334327 (+515706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.884588 (+2550261us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54003b5454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:28.884761 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159517056}
I20260709 12:14:28.920850 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:28.907261 11257 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:14:28.874118 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2599) took 2493 ms (client timeout 2866 ms). Trace:
W20260709 12:14:28.925467 11182 rpcz_store.cc:269] 0709 12:14:26.380766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.487275 (+106509us) service_pool.cc:224] Handling call
0709 12:14:28.873839 (+2386564us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b54000d8c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:28.874028 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47102208}
W20260709 12:14:28.875216 16450 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 2.729s	user 0.007s	sys 0.016s
I20260709 12:14:28.905546 11180 rpcz_store.cc:276] 0709 12:14:26.379391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.379515 (+   124us) service_pool.cc:224] Handling call
0709 12:14:26.379823 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.284629 (+904806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.879461 (+1594832us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54000cdd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:28.879711 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2079118208}
I20260709 12:14:28.903568 11178 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:28.933923 16004 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:28.924732 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 63088.3us
I20260709 12:14:28.906541 16549 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:28.928099 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.918406 16275 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:28.936028 16275 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.936376 16275 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:28.936839 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.937515 11325 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:28.849157 16550 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:28.865595 16285 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:28.902477 11931 rpcz_store.cc:276] 0709 12:14:26.547271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.547458 (+   187us) service_pool.cc:224] Handling call
0709 12:14:26.584343 (+ 36885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:26.642226 (+ 57883us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:26.828313 (+186087us) log.cc:844] Serialized 14 byte log entry
0709 12:14:27.121590 (+293277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:27.121653 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:27.121699 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.884000 (+1762301us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b540015e0d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:28.884062 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.884225 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":225863,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":2427,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4298,"spinlock_wait_cycles":132550016,"thread_start_us":444206,"threads_started":2,"wal-append.queue_time_us":467635,"wal-append.run_cpu_time_us":1466,"wal-append.run_wall_time_us":1114491}
I20260709 12:14:28.934975 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.878607 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:28.846944 16451 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:28.848048 16445 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:28.846729 16098 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:28.860638 16532 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:28.941568 11249 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:14:28.942799 16301 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:14:28.861972 16385 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:28.953087 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:28.930558 16504 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.947708 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:28.950002 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 50510.8us
I20260709 12:14:28.943126 16550 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.954924 11257 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:29.011327 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:28.961345 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.942994 16441 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:28.876075 16312 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:28.956758 11180 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:29.015161 11253 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:29.015561 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:29.018254 11325 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:28.955830 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.019951 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.946687 16533 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:29.028162 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:29.030252 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 914) took 2374 ms. Trace:
I20260709 12:14:28.949611 16445 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:29.023186 11935 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:29.036271 11331 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:29.036909 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.955832 11223 leader_election.cc:400] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16.
I20260709 12:14:28.938248 15984 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:29.025923 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:29.033170 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:28.947042 16549 raft_consensus.cc:2764] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:29.037623 11223 leader_election.cc:403] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:14:29.037961 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:28.938383 16553 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:28.891930 16518 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:28.957173 16465 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:29.020452 16550 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:28.939962 16388 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:29.016836 16441 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.057245 16441 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:29.040905 11331 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:29.039610 16445 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:28.946866 16397 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:14:29.028669 16504 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:29.028069 16334 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:29.053332 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:29.022451 16202 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:28.952900 16276 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:14:29.047780 16244 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16.
I20260709 12:14:29.054193 16543 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:29.059449 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:29.075531 11331 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:29.076550 11253 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
W20260709 12:14:29.041864 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 86249.7us
I20260709 12:14:29.078538 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.078205 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:29.080896 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.063565 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:28.958418 16466 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:14:28.956389 16451 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:14:28.923360 16542 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.102109 11331 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:29.052102 16533 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:29.083045 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:29.111588 11253 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:29.112571 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:29.083817 11249 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:14:29.062186 11248 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:14:29.115423 11248 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:14:29.084805 16347 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:29.125592 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 920) took 2121 ms. Trace:
I20260709 12:14:29.126169 11330 rpcz_store.cc:276] 0709 12:14:27.003789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.005145 (+  1356us) service_pool.cc:224] Handling call
0709 12:14:27.005584 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.005855 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:14:29.038470 (+2032615us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.125269 (+ 86799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.125342 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:29.125381 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.125434 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.125540 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1959911,"prepare.queue_time_us":1956138,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":706,"thread_start_us":162937,"threads_started":2}
I20260709 12:14:29.085101 15986 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:29.084105 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.090170 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:29.134979 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50955.6us
I20260709 12:14:29.135344 11255 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 11.
I20260709 12:14:29.091145 16397 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:29.081584 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.099469 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:29.061228 16438 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:29.054682 16459 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:14:29.104455 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:29.065201 16465 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:29.069278 16553 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:29.067539 16334 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:29.148783 11330 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:29.132261 16533 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:29.137024 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:29.131134 16390 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.078653 16549 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:29.027491 16098 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:29.188761 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51827.8us
I20260709 12:14:29.142865 11325 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:29.110496 16415 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:29.142294 15986 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:29.141036 16203 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.150339 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:29.060474 16105 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:14:29.123075 16451 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:14:29.179966 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 98471us
I20260709 12:14:29.104264 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:29.203079 11331 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:29.204016 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:29.204777 11331 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:14:29.191069 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:29.150975 16528 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:29.215175 16528 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:29.215269 16203 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:29.216051 16416 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:29.154314 16334 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:29.195662 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:29.199194 11252 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:29.216950 11324 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:29.146800 16397 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.188617 16549 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:29.198503 15986 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:29.146627 16244 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:29.191784 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:29.246165 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54459.2us
I20260709 12:14:29.208946 11329 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:29.249281 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:29.249929 11174 raft_consensus.cc:606] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:29.250262 11174 raft_consensus.cc:614] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:29.250289 16258 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:29.251298 16258 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:29.251940 11174 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 12:14:29.252216 16258 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:29.210053 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:29.259104 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:29.259627 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49634.3us
I20260709 12:14:29.260555 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:29.210304 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:29.201903 16505 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.000s	sys 0.012s
I20260709 12:14:29.194322 16295 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:29.227023 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.134646 16347 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.231024 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.268203 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:29.200276 16466 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:29.249468 16525 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:29.228358 16397 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:29.257994 11324 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:14:29.288551 11324 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
W20260709 12:14:29.289139 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.189046 16438 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.260360 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.289958 16438 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.290470 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:29.291320 16349 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:29.291713 16349 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.292095 16349 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:29.292555 16438 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:29.150135 16548 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:29.282518 11252 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:29.295375 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:29.273411 16563 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:29.293051 16004 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:29.296911 11252 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:29.266870 16505 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.000s	sys 0.012s
I20260709 12:14:29.261013 16244 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.304514 16571 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:29.265311 16041 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:29.328931 11249 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:29.305042 16571 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.558007 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2566) took 1175 ms. Trace:
I20260709 12:14:29.558588 11325 rpcz_store.cc:276] 0709 12:14:28.382448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.284205 (+901757us) service_pool.cc:224] Handling call
0709 12:14:29.557348 (+273143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.557570 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.557596 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.557664 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:29.557711 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.557786 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.557934 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:29.330760 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:29.560598 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 229977us
I20260709 12:14:29.280447 16148 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:29.275350 15984 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:29.555382 16571 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:29.260306 16367 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:14:29.294111 16105 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:29.283952 16203 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:29.295702 16466 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:29.216250 16528 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:29.606051 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:29.607024 11326 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
W20260709 12:14:29.607717 11326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2574) took 1211 ms (client timeout 1514 ms). Trace:
W20260709 12:14:29.608171 11326 rpcz_store.cc:269] 0709 12:14:28.396639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.408206 (+ 11567us) service_pool.cc:224] Handling call
0709 12:14:29.605838 (+1197632us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:14:29.607641 (+  1803us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4006144}
I20260709 12:14:29.311949 16438 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:29.306551 16244 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:29.308192 16505 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:29.591657 11174 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:14:29.556404 16301 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.294891 16241 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.293s
I20260709 12:14:29.565948 16280 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:29.564240 16525 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:29.704039 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2291) took 1235 ms. Trace:
I20260709 12:14:29.739802 11174 rpcz_store.cc:276] 0709 12:14:28.468916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.299904 (+830988us) service_pool.cc:224] Handling call
0709 12:14:29.567482 (+267578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.703579 (+136097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.703942 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:29.692026 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 465042us
W20260709 12:14:29.743125 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:29.743630 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.746939 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:29.748637 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:29.286274 16465 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:29.641263 16312 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:29.692227 16091 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.012s	sys 0.053s
I20260709 12:14:29.673280 16388 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:29.776882 11933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 925) took 3087 ms (client timeout 1916 ms). Trace:
I20260709 12:14:29.780814 11324 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:29.777493 11933 rpcz_store.cc:269] 0709 12:14:26.689798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.776609 (+3086811us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:29.776789 (+   180us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:29.770915 16203 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:14:29.773785 16091 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.012s	sys 0.054s
I20260709 12:14:29.754421 16280 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:29.721669 16301 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.772523 16415 raft_consensus.cc:2764] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:14:29.711316 16438 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:29.758793 15900 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:14:29.784147 16552 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.528s
I20260709 12:14:29.565315 16533 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:29.807673 16445 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.728s	user 0.008s	sys 0.005s
I20260709 12:14:29.782115 16388 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:29.807673 16275 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.871s	user 0.006s	sys 0.002s
I20260709 12:14:29.565233 16549 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.798171 16390 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:29.822216 16275 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:29.829916 16532 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.878s	user 0.011s	sys 0.001s
I20260709 12:14:29.793251 16466 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.825991 16445 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.843170 16532 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:29.943742 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:29.945120 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.840238 16347 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:29.819221 16091 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:29.952731 11248 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:29.953136 16091 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:29.954732 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.001s	sys 0.028s
W20260709 12:14:29.955519 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.001s	sys 0.028s
W20260709 12:14:29.956032 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.051s	sys 0.095s
W20260709 12:14:29.956875 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.051s	sys 0.095s
W20260709 12:14:29.957577 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 665092us
W20260709 12:14:29.958072 16385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.006s	sys 0.011s
W20260709 12:14:29.958688 16385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.006s	sys 0.011s
I20260709 12:14:29.843377 16280 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:29.956686 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5940690190000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 12:14:29.964349 16457 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.026s	user 0.005s	sys 0.008s
I20260709 12:14:29.819875 16505 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:29.991375 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5436690190001627s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 12:14:29.994292 11222 connection.cc:256] RPC call timeout handler was delayed by -1.508569019000106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 12:14:29.997699 11222 connection.cc:256] RPC call timeout handler was delayed by -1.4536690190000172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 12:14:29.999665 16098 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.000s	sys 0.010s
W20260709 12:14:30.000361 16098 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.000s	sys 0.010s
I20260709 12:14:29.825819 16275 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:30.009038 16301 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:30.010174 11222 connection.cc:256] RPC call timeout handler was delayed by -1.4356690190002155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 12:14:29.831000 16388 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:29.955933 16091 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:30.015687 11222 connection.cc:256] RPC call timeout handler was delayed by -1.4239690190001966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 12:14:30.019110 11222 connection.cc:256] RPC call timeout handler was delayed by -1.2736690190001809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
I20260709 12:14:30.023073 16445 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:29.965375 16457 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:30.027060 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 278365us
W20260709 12:14:30.033306 11222 connection.cc:256] RPC call timeout handler was delayed by -1.2295690190001096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 12:14:30.033830 11222 connection.cc:256] RPC call timeout handler was delayed by -1.2205690190000951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 12:14:30.034091 11222 connection.cc:256] RPC call timeout handler was delayed by -1.1467690190002031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
I20260709 12:14:30.039096 16388 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:30.044585 16545 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.199s	user 0.008s	sys 0.005s
I20260709 12:14:30.055732 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:30.058199 16574 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:30.058914 16574 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:30.058319 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61680.2us
I20260709 12:14:30.045158 16545 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.060539 16574 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.079237 16532 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.081378 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:30.089708 16438 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.095688 16457 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:30.099615 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.103925 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2282) took 1723 ms. Trace:
W20260709 12:14:30.108951 16334 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.847s	user 0.013s	sys 0.005s
I20260709 12:14:30.109062 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 837) took 1366 ms. Trace:
I20260709 12:14:30.109453 16244 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.126057 16483 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:30.142757 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2089) took 1322 ms. Trace:
I20260709 12:14:30.143435 11329 rpcz_store.cc:276] 0709 12:14:28.811066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.855069 (+ 44003us) service_pool.cc:224] Handling call
0709 12:14:30.132908 (+1277839us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54003241d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.133174 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706585472}
I20260709 12:14:30.137506 11182 rpcz_store.cc:276] 0709 12:14:28.742590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.106721 (+1364131us) service_pool.cc:224] Handling call
0709 12:14:30.107557 (+   836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.108152 (+   595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.108196 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.108294 (+    98us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:30.108369 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.108570 (+   201us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.108783 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.145572 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2075) took 1802 ms. Trace:
I20260709 12:14:30.146049 16483 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.146808 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 831) took 1679 ms. Trace:
W20260709 12:14:30.146989 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 65920.4us
I20260709 12:14:30.147303 11927 rpcz_store.cc:276] 0709 12:14:28.467074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.600130 (+133056us) service_pool.cc:224] Handling call
0709 12:14:30.146235 (+1546105us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540059a6d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.146505 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541323648}
I20260709 12:14:30.136596 11180 rpcz_store.cc:276] 0709 12:14:28.380124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.042585 (+662461us) service_pool.cc:224] Handling call
0709 12:14:29.043052 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.065440 (+ 22388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.065489 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.065550 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:29.065605 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.103218 (+1037613us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b54003480d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.103312 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.103656 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50887936}
W20260709 12:14:30.151831 11934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 923) took 3508 ms (client timeout 2093 ms). Trace:
W20260709 12:14:30.152390 11934 rpcz_store.cc:269] 0709 12:14:26.643041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.151354 (+3508313us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:30.151577 (+   223us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:30.152907 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:30.153378 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:30.154327 16388 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:30.146078 11330 rpcz_store.cc:276] 0709 12:14:28.343146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.128429 (+785283us) service_pool.cc:224] Handling call
0709 12:14:29.128842 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.149422 (+ 20580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.144926 (+995504us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.145278 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40210304}
I20260709 12:14:30.152607 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2298) took 1309 ms. Trace:
I20260709 12:14:30.157708 11180 rpcz_store.cc:276] 0709 12:14:28.843494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.151046 (+1307552us) service_pool.cc:224] Handling call
0709 12:14:30.151676 (+   630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.152135 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.152174 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.152249 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:30.152288 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.152370 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.152505 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.158797 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2258) took 1135 ms. Trace:
W20260709 12:14:30.139108 16234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.201s	user 0.032s	sys 0.029s
I20260709 12:14:30.144465 16244 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.160281 11248 rpcz_store.cc:276] 0709 12:14:29.023140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.116419 (+ 93279us) service_pool.cc:224] Handling call
0709 12:14:29.116802 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.953692 (+836890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.154939 (+201247us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b5400069854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.158728 (+  3789us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920268160}
W20260709 12:14:30.156936 16550 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.046s	user 0.010s	sys 0.005s
I20260709 12:14:30.175030 11182 raft_consensus.cc:606] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:30.180932 16234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.380s	user 0.034s	sys 0.029s
W20260709 12:14:30.187752 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54674.5us
I20260709 12:14:30.183871 16483 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:30.204402 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.207942 16174 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:30.197599 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44946.1us
I20260709 12:14:30.198467 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 838) took 1304 ms. Trace:
I20260709 12:14:30.156065 16334 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.212031 11182 rpcz_store.cc:276] 0709 12:14:28.894271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.176960 (+1282689us) service_pool.cc:224] Handling call
0709 12:14:30.177583 (+   623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.198019 (+ 20436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.198049 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.198114 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:30.198153 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.198209 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.198373 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:30.200377 16285 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.252s	user 0.005s	sys 0.012s
I20260709 12:14:30.213881 16285 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:30.201085 16244 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:30.210813 16545 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:30.192420 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 99637.6us
W20260709 12:14:30.219245 11935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.005s	sys 0.015s
I20260709 12:14:30.223464 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:30.224308 11182 raft_consensus.cc:606] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:30.224860 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:30.228730 16337 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:30.224855 11182 raft_consensus.cc:614] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:30.229275 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:30.235530 11935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.006s	sys 0.015s
I20260709 12:14:30.237648 16334 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.239501 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:30.241231 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:30.187705 16550 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.241910 16244 raft_consensus.cc:2764] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:14:30.251792 11926 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.985s	user 0.006s	sys 0.011s
I20260709 12:14:30.254227 11926 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:30.255956 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 945) took 1051 ms. Trace:
I20260709 12:14:30.264494 16518 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:30.258782 11926 rpcz_store.cc:276] 0709 12:14:29.204274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.204425 (+   151us) service_pool.cc:224] Handling call
0709 12:14:30.255865 (+1051440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.257421 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:30.267025 16520 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:30.267545 16388 raft_consensus.cc:2764] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:14:30.279579 11929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 921) took 3653 ms (client timeout 2100 ms). Trace:
I20260709 12:14:30.279881 11174 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: []
W20260709 12:14:30.280117 11929 rpcz_store.cc:269] 0709 12:14:26.624100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.624389 (+   289us) service_pool.cc:224] Handling call
0709 12:14:30.277384 (+3652995us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b5400650ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.277608 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921159424}
W20260709 12:14:30.280728 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.267575 16520 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:30.287261 16520 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.281498 16454 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:30.295142 16454 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:30.301872 16349 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.008s	user 0.015s	sys 0.010s
I20260709 12:14:30.302074 16451 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:30.302772 16349 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.302932 16451 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.298099 16545 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.304746 16545 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.305721 16451 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:30.309327 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 80238.5us
I20260709 12:14:30.299247 16454 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.312887 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:29.032292 11328 rpcz_store.cc:276] 0709 12:14:26.655786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.657768 (+  1982us) service_pool.cc:224] Handling call
0709 12:14:26.658128 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:26.658509 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:26.773956 (+115447us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.909039 (+135083us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.911175 (+  2136us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.911307 (+   132us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:28.937166 (+2025859us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b180024a600. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:14:28.937395 (+   229us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.020960 (+ 83565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.021033 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.030044 (+  9011us) raft_consensus.cc:1685] finished
0709 12:14:29.030062 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.030198 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1995438,"prepare.queue_time_us":2336128,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":1416,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":40051,"spinlock_wait_cycles":5853952,"thread_start_us":332833,"threads_started":3,"wal-append.queue_time_us":205370,"wal-append.run_cpu_time_us":1975,"wal-append.run_wall_time_us":1037712}
I20260709 12:14:30.350185 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:30.315632 16588 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:14:30.318619 11173 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2270) took 3413 ms (client timeout 2365 ms). Trace:
I20260709 12:14:30.319036 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2250) took 1358 ms. Trace:
I20260709 12:14:30.319941 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2238) took 1876 ms. Trace:
I20260709 12:14:30.328035 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:30.329319 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:30.332190 16518 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.330756 16545 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.335804 16121 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:30.306499 16337 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:30.352533 11926 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 16. Ops: []
W20260709 12:14:30.345760 16397 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.981s	user 0.015s	sys 0.008s
I20260709 12:14:30.353747 11250 rpcz_store.cc:276] 0709 12:14:28.443629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.478639 (+ 35010us) service_pool.cc:224] Handling call
0709 12:14:28.479027 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.643623 (+164596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:28.643668 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:28.643719 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:28.643761 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.319482 (+1675721us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.319553 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.319770 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377961728}
W20260709 12:14:30.353435 11173 rpcz_store.cc:269] 0709 12:14:26.905442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.907755 (+  2313us) service_pool.cc:224] Handling call
0709 12:14:30.318156 (+3410401us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54002576d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.318365 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545261312}
I20260709 12:14:30.353519 11254 rpcz_store.cc:276] 0709 12:14:28.960152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.961494 (+  1342us) service_pool.cc:224] Handling call
0709 12:14:28.961850 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.078618 (+116768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.078655 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.078706 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:29.078739 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.318514 (+1239775us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540010bb54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.318592 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.318843 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267999104}
I20260709 12:14:30.365396 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:30.366569 11931 raft_consensus.cc:2368] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16.
I20260709 12:14:30.366451 16314 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:30.367051 16314 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:30.367199 16513 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.367928 16513 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.368844 16244 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:30.369678 11175 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:30.370185 16319 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:30.370857 16319 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:30.358188 16451 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:30.356585 16337 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:30.371833 11175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2284) took 1988 ms (client timeout 1645 ms). Trace:
I20260709 12:14:30.372094 16337 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:30.372394 11175 rpcz_store.cc:269] 0709 12:14:28.383083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.399205 (+ 16122us) service_pool.cc:224] Handling call
0709 12:14:30.371561 (+1972356us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54000cd854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.371737 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4505448192}
I20260709 12:14:30.373926 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:30.374287 11323 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2056) took 3302 ms (client timeout 1658 ms). Trace:
I20260709 12:14:30.374439 11327 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:30.374750 11323 rpcz_store.cc:269] 0709 12:14:27.071586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.075546 (+  3960us) service_pool.cc:224] Handling call
0709 12:14:30.373707 (+3298161us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001c52d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.374034 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2376862464}
I20260709 12:14:30.378398 11929 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:30.379045 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:30.380081 11933 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:30.379494 16121 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.382761 16121 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.383327 16285 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.383869 16285 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.385499 11934 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 15. Ops: []
W20260709 12:14:30.386343 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58474.6us
I20260709 12:14:30.368507 16314 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.388674 16314 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:30.363525 16397 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.372076 16319 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.389586 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:30.399295 11934 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:30.399821 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:30.400333 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:30.438405 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.410336 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2300) took 1566 ms. Trace:
I20260709 12:14:30.400817 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:30.425681 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:30.390524 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61360us
W20260709 12:14:30.405243 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55265.4us
I20260709 12:14:30.417280 11255 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 16. Ops: []
I20260709 12:14:30.417692 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2281) took 2037 ms. Trace:
I20260709 12:14:30.412070 16513 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.398371 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:30.403020 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2076) took 2059 ms. Trace:
I20260709 12:14:30.420985 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:30.452306 11933 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:30.441129 11180 rpcz_store.cc:276] 0709 12:14:28.844073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.178228 (+1334155us) service_pool.cc:224] Handling call
0709 12:14:30.191885 (+ 13657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.192369 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:30.286420 (+ 94051us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.359582 (+ 73162us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.409885 (+ 50303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.409947 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:30.410049 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.410115 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.410249 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73520,"prepare.queue_time_us":155238,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":102,"raft.queue_time_us":104621,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":1772,"thread_start_us":100418,"threads_started":2,"wal-append.queue_time_us":10940}
I20260709 12:14:30.449163 11177 rpcz_store.cc:276] 0709 12:14:28.379864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.663269 (+283405us) service_pool.cc:224] Handling call
0709 12:14:30.417291 (+1754022us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001f3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.417498 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2888532864}
W20260709 12:14:30.437434 16571 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.844s	user 0.014s	sys 0.006s
I20260709 12:14:30.392542 16349 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.419831 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:30.454877 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:30.407878 16441 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.350s	user 0.010s	sys 0.008s
I20260709 12:14:30.390957 15949 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:30.394325 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:30.420295 15986 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.153s	user 0.005s	sys 0.013s
I20260709 12:14:30.450002 11331 rpcz_store.cc:276] 0709 12:14:28.343485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.218041 (+874556us) service_pool.cc:224] Handling call
0709 12:14:29.218344 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.306836 (+ 88492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.306884 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.306949 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:29.306995 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.402280 (+1095285us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b540042a9d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.402406 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.402752 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203066624}
W20260709 12:14:30.418377 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2235) took 1992 ms (client timeout 2646 ms). Trace:
I20260709 12:14:30.458993 11257 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:30.416471 16596 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:14:30.462034 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.459856 11179 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:30.409241 16091 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:30.408159 16057 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.463024 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:30.466892 11251 rpcz_store.cc:269] 0709 12:14:28.426270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.474613 (+ 48343us) service_pool.cc:224] Handling call
0709 12:14:30.417968 (+1943355us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001cf2d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.418151 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777756032}
I20260709 12:14:30.419404 16397 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.481681 11927 raft_consensus.cc:2368] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19.
I20260709 12:14:30.479380 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.481063 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.483444 16209 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:30.475756 16441 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.396605 16518 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.488250 11181 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:14:30.489656 16441 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.483467 11928 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:30.490339 11252 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:30.478789 15986 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.498970 15986 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.490420 15949 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:30.501348 15949 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.504014 11253 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:30.504982 16572 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:30.505811 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:30.505936 16572 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.508724 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:30.509248 16583 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:14:30.509662 15986 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.510978 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 944) took 1307 ms. Trace:
I20260709 12:14:30.510795 15986 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.511447 11935 rpcz_store.cc:276] 0709 12:14:29.203583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.203743 (+   160us) service_pool.cc:224] Handling call
0709 12:14:29.258515 (+ 54772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.271958 (+ 13443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.272007 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.272663 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.292876 (+1020213us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:30.300832 (+  7956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.305857 (+  5025us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.510692 (+204835us) raft_consensus.cc:1685] finished
0709 12:14:30.510716 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.510899 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3938,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":2870,"thread_start_us":970327,"threads_started":1,"wal-append.queue_time_us":915824}
I20260709 12:14:30.473536 16571 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.515749 16571 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.516505 15986 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.518603 16550 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.492812 16457 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.520208 16457 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.523423 16457 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.526036 16518 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.531924 16454 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.534121 16057 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:30.534595 16057 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:30.535020 16057 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:30.484457 16520 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.543057 16513 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:30.544728 16520 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.546177 16300 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:30.547489 16300 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.553864 16234 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:30.554391 16234 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:30.555537 16234 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.556809 16234 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:30.508726 16572 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.558084 16300 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.560751 16300 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.545953 16571 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:14:30.528115 16504 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.285s	user 0.015s	sys 0.006s
I20260709 12:14:30.572925 11331 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:14:30.573028 11326 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:14:30.578543 11257 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
W20260709 12:14:30.583478 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 193835us
I20260709 12:14:30.589533 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.590495 11328 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:30.593387 16209 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:30.581700 16334 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.594540 16334 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.594432 16209 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:30.597975 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.598160 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.598021 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:30.600255 11256 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:30.603113 16209 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.605721 16057 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.608542 16057 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.614354 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.615537 11178 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:30.616950 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.618141 11179 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:14:30.620715 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2297) took 2075 ms. Trace:
I20260709 12:14:30.624159 16549 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.626946 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.629278 11254 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:14:30.621244 11174 rpcz_store.cc:276] 0709 12:14:28.545655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.951349 (+1405694us) service_pool.cc:224] Handling call
0709 12:14:30.024265 (+ 72916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.280274 (+256009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.620344 (+340070us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54000d7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.620607 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583511424}
I20260709 12:14:30.593896 16105 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:30.647358 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:30.653709 11181 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:14:30.654050 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 932) took 2327 ms. Trace:
I20260709 12:14:30.652439 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:30.658139 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 149600us
I20260709 12:14:30.646054 16105 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:30.654532 11327 rpcz_store.cc:276] 0709 12:14:28.322796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.334570 (+ 11774us) service_pool.cc:224] Handling call
0709 12:14:30.373992 (+2039422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.374929 (+   937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.648088 (+273159us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54001c52d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:30.650232 (+  2144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4689156992}
W20260709 12:14:30.648655 16525 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.720s	user 0.016s	sys 0.005s
I20260709 12:14:30.662088 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:30.663105 16504 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.665941 16553 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:30.666903 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:30.667317 16594 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:14:30.667976 16479 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.651728 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.680907 11179 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:30.659986 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:30.679813 11181 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:14:30.677511 11323 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:30.669953 16454 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:30.687330 11182 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.676832 16349 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.688738 16349 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.689558 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:30.690332 11250 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:30.691398 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.692189 16209 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.696673 11325 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:30.676416 16553 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:30.679998 16285 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.700235 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.703907 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:30.704145 11327 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:30.714623 11328 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-14.11]   Dedup: 11.9->[12.10-14.11]
I20260709 12:14:30.718428 11328 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: [11.9-14.11]
I20260709 12:14:30.719946 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:30.723757 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72360us
I20260709 12:14:30.697649 16209 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.726405 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:30.722877 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.696237 16479 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.699357 16504 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.712563 16572 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.733173 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:30.707249 16057 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:14:30.738029 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:30.737600 11323 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:30.730907 16525 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.741840 16525 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.743259 11256 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.738935 11326 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:30.745325 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.744664 16572 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.753257 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:30.754734 11252 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:30.759909 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2254) took 1794 ms. Trace:
I20260709 12:14:30.760406 11253 rpcz_store.cc:276] 0709 12:14:28.965222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.113367 (+148145us) service_pool.cc:224] Handling call
0709 12:14:29.113687 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.504796 (+1391109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.759598 (+254802us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400109854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.759843 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3196192512}
I20260709 12:14:30.761682 16533 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:30.761965 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2259) took 1732 ms. Trace:
I20260709 12:14:30.762310 11255 rpcz_store.cc:276] 0709 12:14:29.029209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.136125 (+106916us) service_pool.cc:224] Handling call
0709 12:14:29.148009 (+ 11884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.464436 (+1316427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.761667 (+297231us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400379e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:30.761887 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2918179200}
I20260709 12:14:30.762377 16533 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.763682 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:30.764307 11255 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:30.767071 11325 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.738957 16209 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:30.768924 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:30.770514 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:30.771131 11249 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:30.772061 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:30.774277 11251 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:30.778198 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:30.778544 11927 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:30.778882 11935 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.779386 11928 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:30.773902 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:30.782080 11256 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:30.781970 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:30.782954 11252 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:30.785908 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.785027 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:30.783896 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.791574 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:30.787626 11331 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:30.795226 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.795976 11253 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:14:30.780344 16432 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:30.803167 11254 raft_consensus.cc:2410] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:14:30.804450 16545 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:30.805725 16432 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:30.807206 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49754.2us
W20260709 12:14:30.806253 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60730us
I20260709 12:14:30.809049 16432 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.808936 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:30.815224 11257 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.821346 11331 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:30.823163 11256 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:30.803826 16533 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.826617 11176 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:30.830991 11255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
mode: GRACEFUL
new_leader_uuid: "016411114c6d42039bd04025378dfc98"
 from {username='slave'} at 127.0.0.1:55516
I20260709 12:14:30.831645 11255 raft_consensus.cc:606] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Received request to transfer leadership to TS 016411114c6d42039bd04025378dfc98
I20260709 12:14:30.831733 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.832028 11255 raft_consensus.cc:614] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:14:30.833403 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 130686us
I20260709 12:14:30.834694 11935 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:30.830055 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.836586 16525 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.836910 11174 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:30.837500 11253 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:30.766532 16553 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.837925 11253 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:30.843580 16504 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.844753 16572 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.848496 16337 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.850543 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.851243 11175 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:30.856179 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:30.858548 16572 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.859195 16334 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.859366 16572 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.865483 11251 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:30.869254 11326 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:30.841661 16553 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:30.863488 11330 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:30.866822 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:30.873487 11331 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:30.877893 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:14:30.879670 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.854691 16209 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:30.880904 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:30.873018 16571 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:30.885998 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:30.904551 11182 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
W20260709 12:14:30.906143 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 97383.9us
I20260709 12:14:30.908404 11326 raft_consensus.cc:2410] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:30.880530 16057 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.910665 11257 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:30.884755 16209 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:30.913865 16209 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:30.916194 16057 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.918434 16057 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.919381 16349 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.925045 16571 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:30.926177 16571 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:30.926564 16571 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:30.926815 11253 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:30.927943 11253 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:30.927690 16571 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:30.930666 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:30.931504 16349 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:30.932008 16349 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:30.932469 16349 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:30.934223 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:30.935086 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.931713 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:30.938490 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.938052 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 81944.4us
I20260709 12:14:30.944343 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:30.899631 16525 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:14:30.929613 16479 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:30.950425 11176 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:30.939716 16285 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:14:30.951148 11250 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:30.913051 16121 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:14:30.954644 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:30.955757 11176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2286) took 2488 ms (client timeout 1752 ms). Trace:
I20260709 12:14:30.941395 16572 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:30.979700 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.981092 11327 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:14:30.956782 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:30.955216 16533 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:14:30.982005 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:30.945255 16571 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:30.978930 11251 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:14:30.991255 16603 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.196s	user 0.001s	sys 0.001s
I20260709 12:14:30.992197 11256 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:30.940403 16091 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:14:30.994493 11326 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 11.
W20260709 12:14:30.986683 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.998096 11329 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:14:30.989557 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:30.983426 11176 rpcz_store.cc:269] 0709 12:14:28.467798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.481010 (+ 13212us) service_pool.cc:224] Handling call
0709 12:14:30.398514 (+1917504us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b1800218400. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6727646 00007f3ec666158e 00007f3ec66e2f9e 00007f3ec66e47de 00007f3ec66f027c 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c
0709 12:14:30.826215 (+427701us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b58008f75d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66480aa 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66f027c 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c
0709 12:14:30.955695 (+129480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3176506112}
I20260709 12:14:30.994441 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:30.954533 16574 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:30.999634 11255 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:30.998786 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:31.006187 11174 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:14:31.005874 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:31.005015 11330 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:31.010730 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:31.012710 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:31.013738 16459 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.013969 16454 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:31.014498 16454 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.014339 16459 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.014892 16454 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.017843 16202 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
W20260709 12:14:30.934526 16397 consensus_queue.cc:1175] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 18 status { last_received { term: 13 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:31.021808 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.022563 11322 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:31.023520 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.023330 16105 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:31.023990 16526 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:31.024359 11322 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:31.010795 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:31.025832 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:31.026620 11931 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:31.027530 11248 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
W20260709 12:14:31.033232 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 103147us
I20260709 12:14:31.033912 16337 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:31.034192 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:31.034560 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.002831 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.038837 11175 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:30.996083 16479 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:31.041231 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 50592.4us
I20260709 12:14:31.007712 11174 pending_rounds.cc:85] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 7
W20260709 12:14:31.072422 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69922us
I20260709 12:14:31.074221 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:31.074354 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:31.017433 16459 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:31.085063 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.087127 11325 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:31.035449 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:31.087630 11331 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:31.091827 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:31.072667 11174 pending_rounds.cc:107] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:14:31.100152 11174 raft_consensus.cc:2889] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:31.101401 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.104218 11326 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:31.024369 16526 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:31.100040 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:31.037464 16432 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:31.086107 16518 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:31.119485 11331 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:14:31.126533 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:31.127192 11252 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:31.125380 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.130702 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.112233 16057 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:14:31.121500 16459 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:14:31.142104 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 185621us
I20260709 12:14:31.143327 11147 leader_election.cc:400] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16.
I20260709 12:14:31.143689 11147 leader_election.cc:403] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:31.144392 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 70165.2us
I20260709 12:14:31.146356 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:31.147862 16057 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:31.148787 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:31.150655 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:31.153754 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:31.115046 16397 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:31.153376 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:31.157948 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.119385 16337 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:31.105684 16549 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:14:31.168658 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.169799 11257 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:14:31.171399 11325 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:31.159309 15986 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:31.172829 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:31.153502 16105 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:31.129472 16518 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:31.175637 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.130739 16605 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:31.177482 11255 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:31.178737 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.183990 16397 raft_consensus.cc:2764] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:31.185984 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.186584 11173 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.156278 16459 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:31.190624 16459 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.193747 16495 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:31.194411 16495 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.194860 16495 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:31.199724 16234 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.005s
I20260709 12:14:31.178611 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.202951 11326 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:31.185210 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:31.204304 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 132315us
I20260709 12:14:31.205555 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:31.206601 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:31.207788 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:31.208631 11256 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:31.209539 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:31.210639 11251 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.211736 16479 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:31.170360 16574 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:31.203163 16337 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:31.215025 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.210000 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:31.227461 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.229876 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53636.9us
I20260709 12:14:31.231000 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:31.169327 16504 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16.
I20260709 12:14:31.235560 11250 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:31.224709 11327 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:31.225822 11176 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:31.232926 11253 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:14:31.233189 11329 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
W20260709 12:14:31.237897 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.171926 16091 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:31.238693 11253 pending_rounds.cc:85] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 11
I20260709 12:14:31.241130 11253 pending_rounds.cc:107] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:14:31.241425 11253 raft_consensus.cc:2889] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:31.241775 11253 pending_rounds.cc:107] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:14:31.242093 11253 raft_consensus.cc:2889] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:31.244066 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.245955 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.247310 11326 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:31.250011 11175 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 12:14:31.222182 11251 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:31.254412 11182 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:31.259074 11931 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:31.260676 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.259800 16525 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:31.270422 16525 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:31.271945 16525 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.273797 16148 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:31.278828 16518 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.258180 16605 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:31.283346 16605 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:31.283042 16545 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:31.284468 16605 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.245725 16091 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.254441 16572 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:31.291244 11252 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
W20260709 12:14:31.291760 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 148667us
I20260709 12:14:31.233955 16591 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:14:31.245805 16105 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.256600 16608 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:31.305061 11147 leader_election.cc:400] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19.
I20260709 12:14:31.305609 11147 leader_election.cc:403] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:31.310887 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:31.314049 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 83189.7us
I20260709 12:14:31.292925 16057 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:31.326718 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:31.329883 11331 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:31.337890 11173 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:31.345772 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:31.346338 11935 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.351378 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.302225 16525 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:31.354365 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:31.249584 16574 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.358788 11177 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
W20260709 12:14:31.360242 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 61703.3us
I20260709 12:14:31.361070 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:31.362303 16533 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:31.362726 16533 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:31.363099 16533 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:31.371915 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.372908 16572 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:31.373359 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.374272 11327 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:31.375241 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.375568 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:31.376029 11324 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:31.377303 11329 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:31.377698 11329 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:31.377930 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.378528 11329 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:31.379633 11329 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:31.374004 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.350590 16608 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:31.299666 16105 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.289942 16526 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.388352 16105 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:31.393442 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:31.385753 16608 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.394734 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:31.359804 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 237564us
I20260709 12:14:31.402832 11931 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:31.403837 16520 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:31.405010 16479 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:31.405683 16479 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:31.406502 16479 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:31.407898 16479 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.407948 16209 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:31.410421 16337 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:31.345677 16504 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19.
I20260709 12:14:31.418105 16454 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.420363 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:31.404675 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:31.428190 11331 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:31.429108 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:31.425611 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.396166 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:31.431988 11325 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:31.433585 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.433882 11330 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:14:31.432194 11931 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:31.430390 11935 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:31.438112 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:31.441948 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.442668 11181 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:31.451231 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.453306 11177 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:31.451588 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.452131 11173 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:31.456054 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.458860 11323 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:31.456578 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.450585 11329 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:14:31.456677 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.457345 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:31.456272 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:31.460645 11179 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:31.451082 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:31.464803 11250 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:31.461359 11933 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:31.463873 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:31.464761 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 257204us
I20260709 12:14:31.468497 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.469981 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:31.471236 11929 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:31.450775 16571 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:31.472205 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.395761 16533 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.474674 11182 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:31.474629 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:31.480448 11252 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:31.482097 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.483145 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:31.471796 16349 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
W20260709 12:14:31.484654 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.473686 16591 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:31.500530 11325 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:14:31.466310 16105 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:31.498762 11253 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
W20260709 12:14:31.501597 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.503638 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.474098 16571 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.510829 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.511564 11176 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.496798 15986 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:31.517087 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.514436 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.517213 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:31.515169 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1629219340002237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
I20260709 12:14:31.523200 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.518576 11179 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:31.502159 16591 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:31.522471 11296 connection.cc:256] RPC call timeout handler was delayed by -1.066621934000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 12:14:31.474187 16518 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:14:31.526615 11248 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:31.529491 11182 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:31.531720 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:31.532366 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.508749 16571 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:31.502017 16209 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:31.483909 16549 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:31.536401 11182 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:14:31.554075 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.537979 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.545212 11181 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:31.556463 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.558372 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:31.500259 16349 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.565836 16533 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:31.565958 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.583360 11257 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:31.583437 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:31.534618 16105 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.585381 11929 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:31.567958 11255 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:31.568449 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:31.569276 11324 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:14:31.573256 16607 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:31.573468 16334 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:31.587334 16607 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:31.574723 11328 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:31.551664 16611 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.589329 16611 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.590627 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:31.591609 16611 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:31.592307 16611 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:31.592727 16611 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:31.593251 16611 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:31.534504 16591 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:31.582258 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:31.602304 16553 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:31.602839 16553 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:31.567654 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:31.604019 11250 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:31.603845 16553 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.532469 16609 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.606295 11175 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:31.606896 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.628165 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.609338 11928 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 14. Ops: []
W20260709 12:14:31.633239 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 118391us
I20260709 12:14:31.589356 16526 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.640458 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:31.652354 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.654644 16518 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:31.659986 16610 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 12:14:31.609210 16202 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:14:31.645728 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.648896 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72051.8us
I20260709 12:14:31.632066 16334 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 12:14:31.662143 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 85658.2us
I20260709 12:14:31.628350 16285 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:14:31.645238 11176 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:31.631631 16432 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:14:31.656155 16454 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:31.650485 16618 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:31.717439 11180 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:31.565696 11075 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 13, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 13 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:31.732468 11175 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:31.693295 16459 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:14:31.740989 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 100585us
I20260709 12:14:31.705010 16334 consensus_queue.cc:1243] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:14:31.750737 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 159152us
I20260709 12:14:31.646332 16609 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.762710 11175 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:31.725471 16618 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.734298 16209 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.766999 16618 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:31.767242 16209 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:31.770951 16209 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:31.732889 16300 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:14:31.748849 16495 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.001s
I20260709 12:14:31.737725 16479 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:31.743816 15986 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:31.752177 16459 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:31.781368 16614 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:14:31.787639 16533 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:31.755168 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:31.791327 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54920.5us
W20260709 12:14:31.795166 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.795820 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 123092us
I20260709 12:14:31.798229 16105 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:31.815886 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:31.823282 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:31.824265 11180 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:31.831039 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:31.832023 11253 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:31.779284 16300 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:31.779711 16495 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:31.842741 11176 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:31.811272 16605 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:14:31.789731 16479 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:31.891358 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 110176us
I20260709 12:14:31.807209 16628 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:31.834600 16349 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:31.886169 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:31.911895 11175 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:31.904481 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:31.896788 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.912920 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.838131 16572 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:31.905198 11326 pending_rounds.cc:85] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:14:31.920195 11326 pending_rounds.cc:107] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 12:14:31.920522 11326 raft_consensus.cc:2889] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:31.892475 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 76729us
I20260709 12:14:31.823112 16314 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:14:31.815342 16612 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:14:31.944519 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:31.944370 16635 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:31.914249 16628 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.945096 16627 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:14:31.927045 16608 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:14:31.966936 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54018.2us
I20260709 12:14:31.955906 16572 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.997836 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:31.998651 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 161074us
I20260709 12:14:31.945813 16459 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:31.977779 16625 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:14:31.978160 16300 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:31.926892 11174 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:31.988991 11175 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
W20260709 12:14:31.991524 16633 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:14:31.992446 16621 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:14:31.992802 16629 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:31.951062 15986 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:31.983552 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 154720us
I20260709 12:14:31.968276 16617 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.189s
I20260709 12:14:31.994973 16628 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:32.027025 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70344.4us
I20260709 12:14:32.028604 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:32.032454 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 218125us
I20260709 12:14:32.035046 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:32.018030 16209 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.037300 16643 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:32.037859 11324 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:32.039886 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:32.023164 16300 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:14:32.031100 16639 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:14:32.003048 16572 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.051438 16636 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.011756 16349 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:14:32.072101 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:32.078516 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:32.035372 16609 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:32.082758 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:32.089262 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53524.7us
I20260709 12:14:32.094123 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:32.077891 16300 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:32.076393 16495 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:32.019753 16607 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.042404 16209 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.113674 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:32.091917 16637 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:14:32.032696 16629 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.087929 16202 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:32.077031 16643 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:32.104071 16525 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.000s
I20260709 12:14:32.078223 16636 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.099195 16549 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:32.095783 16148 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:14:32.124279 16479 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:32.141062 16617 consensus_queue.cc:1175] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 15 status { last_received { term: 14 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:32.095872 16609 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:32.127089 16631 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.179s	user 0.000s	sys 0.003s
I20260709 12:14:32.113371 16618 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.151463 11326 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-14.11]   Dedup: 11.9->[12.10-14.11]
W20260709 12:14:32.153123 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50422.6us
I20260709 12:14:32.114259 16623 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:32.125236 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:32.178164 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53026.8us
I20260709 12:14:32.179363 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:32.158587 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:32.138255 16611 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.002s
I20260709 12:14:32.159713 16202 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:32.161720 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 217294us
I20260709 12:14:32.189388 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:32.190201 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 76133.9us
I20260709 12:14:32.190501 16479 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:32.191100 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:32.125957 16454 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:14:32.138197 16571 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.004s
I20260709 12:14:32.174934 16609 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.202669 16609 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:32.202828 16397 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:32.203325 16397 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:32.203717 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:32.204326 16397 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.183495 16654 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:32.185648 11326 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: [11.9-14.11]
I20260709 12:14:32.180968 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:32.126883 16607 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:32.133246 16591 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.001s
I20260709 12:14:32.187419 11180 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:14:32.188146 11248 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:14:32.191074 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.161846 16647 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:32.193811 16658 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:32.192130 16454 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:32.199892 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:32.201761 16571 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.217716 15986 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.201776 16648 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:32.218369 16629 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:32.218763 16300 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.198134 16643 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:32.184425 16623 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.202554 16572 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:32.260910 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69881.5us
W20260709 12:14:32.312654 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:32.243285 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52330.7us
W20260709 12:14:32.239825 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.352783 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.354597 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:32.357520 16432 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:32.359931 16432 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.360364 16432 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:32.360261 16234 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:32.261245 16202 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.366020 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2305) took 1205 ms. Trace:
I20260709 12:14:32.215343 16618 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:32.366503 11174 rpcz_store.cc:276] 0709 12:14:31.160750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.165308 (+  4558us) service_pool.cc:224] Handling call
0709 12:14:32.365799 (+1200491us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54002b6454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:32.365955 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695571328}
I20260709 12:14:32.367033 16618 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.367666 16618 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.231415 16591 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.195735 16611 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.375510 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.218402 16654 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:32.232998 16647 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.378355 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.382941 11252 raft_consensus.cc:2368] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:32.383301 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:32.231190 16397 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:32.288530 16613 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.690s	user 0.001s	sys 0.000s
W20260709 12:14:32.242563 16631 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:32.261149 16644 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:32.318058 16640 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:32.398571 11180 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:14:32.356596 16623 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:32.398005 11928 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:14:32.388725 11253 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:14:32.371289 16629 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:32.264384 16636 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:32.302488 16209 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:32.260880 16479 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:32.393340 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:32.407431 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 226518us
I20260709 12:14:32.408121 11174 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:32.395946 16658 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:32.397230 11251 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:32.397854 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:32.397989 11931 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:32.355999 16628 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.398510 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.395033 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:32.400498 11929 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:32.403141 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.415035 11933 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:32.382879 16654 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.405194 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.387513 16647 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:32.382237 16432 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.389927 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.415256 11935 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:32.403985 16644 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:32.417836 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:32.370647 15986 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.419353 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.433360 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.433418 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:32.422519 16611 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:32.398523 16640 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:32.424376 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:32.389441 16337 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:14:32.426039 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:32.426486 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:32.383342 16545 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:14:32.430438 11248 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:32.392970 16574 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:14:32.407860 16479 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.420394 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:32.428491 16654 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:32.414645 16618 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:32.438720 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.430676 16209 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:32.439719 11933 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:32.443264 16549 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:32.457525 16646 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:14:32.494884 16646 consensus_queue.cc:1243] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:14:32.494983 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:32.445200 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.500710 16571 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:32.501694 16640 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.503010 16640 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:32.506759 16630 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:32.507246 16630 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:32.416672 16533 raft_consensus.cc:993] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:14:32.453199 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.439276 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:32.443867 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:32.494308 16572 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.443929 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.508051 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2279) took 1072 ms. Trace:
I20260709 12:14:32.523736 11253 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:32.449908 16479 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:32.528100 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:32.530287 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:32.529279 11182 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:14:32.450450 16553 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:32.520046 11256 rpcz_store.cc:276] 0709 12:14:31.435848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.436030 (+   182us) service_pool.cc:224] Handling call
0709 12:14:31.456026 (+ 19996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:31.497890 (+ 41864us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:31.592209 (+ 94319us) log.cc:844] Serialized 14 byte log entry
0709 12:14:31.735227 (+143018us) log.cc:844] Serialized 14 byte log entry
0709 12:14:31.743176 (+  7949us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:31.777886 (+ 34710us) log.cc:844] Serialized 26 byte log entry
0709 12:14:31.778105 (+   219us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:31.778360 (+   255us) log.cc:844] Serialized 14 byte log entry
0709 12:14:31.780583 (+  2223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.780675 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:32.370302 (+589627us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b180033ade0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:32.506800 (+136498us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b180033ade0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:32.507835 (+  1035us) raft_consensus.cc:1685] finished
0709 12:14:32.507862 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:32.507996 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7026,"prepare.queue_time_us":1435380,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":98,"raft.queue_time_us":45920,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":26203,"spinlock_wait_cycles":57064448,"thread_start_us":219218,"threads_started":2,"wal-append.queue_time_us":246913}
I20260709 12:14:32.441416 16202 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:32.449467 16648 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:32.531569 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:32.533443 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.533670 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:32.434466 16636 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:14:32.536389 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:32.551775 11328 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:32.539165 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:32.552909 11330 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:32.540076 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:32.543587 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:32.545338 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:32.547945 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.549044 11329 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:32.550381 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:32.555160 11931 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:32.453475 16091 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:32.426676 16454 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.523928 16209 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:32.557798 11326 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:32.562026 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:32.560907 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:32.565668 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 126437us
I20260709 12:14:32.558872 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:32.525508 16572 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.530592 16533 raft_consensus.cc:1081] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:14:32.440070 16609 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.567826 11253 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:32.566722 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.572674 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:32.580289 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.582114 11934 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:14:32.582252 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:32.582960 11254 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:32.583428 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.584692 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:32.585469 11256 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:32.587570 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:32.587361 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:32.616349 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:32.557564 16665 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:32.592870 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.619297 11250 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:14:32.561870 16571 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:32.575538 16202 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.620213 11927 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:32.585856 16656 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 12:14:32.679996 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51550.3us
I20260709 12:14:32.680156 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.698501 11251 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:32.638437 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.701261 11253 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:32.625478 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:32.689220 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.702610 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.703549 11256 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:32.652266 11330 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:32.579355 16591 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:32.711336 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 119492us
I20260709 12:14:32.712266 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:32.713846 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.679912 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:32.614799 16648 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:32.740487 16648 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.740906 16648 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:32.743492 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.737816 16202 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.607645 16234 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.131s
I20260709 12:14:32.738677 16572 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:32.672823 16495 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:32.747246 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 44738.1us
I20260709 12:14:32.716328 16571 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.739504 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:32.556521 16644 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.690583 16334 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:32.764261 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.735337 16209 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.751722 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.772517 11256 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:32.760576 11254 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:32.761161 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2287) took 1261 ms. Trace:
I20260709 12:14:32.708446 16533 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:32.744405 16665 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:32.781333 11249 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:32.774025 11252 rpcz_store.cc:276] 0709 12:14:31.500234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.506965 (+  6731us) service_pool.cc:224] Handling call
0709 12:14:32.760894 (+1253929us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54001ce8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:32.761082 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974989184}
I20260709 12:14:32.790002 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:32.749375 16495 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.805523 11251 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:32.766762 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:32.690459 16504 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:14:32.834710 16632 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:32.763948 16334 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:32.754832 16628 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:32.779706 16643 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:32.742565 16057 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:14:32.875454 16670 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:14:32.832796 16657 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:32.831955 16624 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:32.778653 16610 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 12:14:32.866611 16669 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 12:14:32.867753 16667 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
I20260709 12:14:32.871042 16202 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:32.871790 16572 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:32.797098 16495 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:32.785482 16665 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:32.885187 11251 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:14:32.894003 11256 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:32.894677 11254 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:14:32.907783 11255 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:14:32.927167 16665 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:32.932839 16674 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:14:32.936146 15986 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:32.744773 16630 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:32.949265 16609 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:32.949806 16672 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.116s	user 0.004s	sys 0.000s
I20260709 12:14:32.891885 16632 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:32.901641 16658 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.952683 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:32.959142 11174 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.001s
I20260709 12:14:32.965454 11174 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:32.966306 11173 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:14:32.963948 11255 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
W20260709 12:14:32.954607 16397 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:14:32.968863 11322 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:32.974176 16635 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:32.975602 11323 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:14:32.977232 11327 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:14:32.982833 16571 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:32.991649 16632 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:32.968041 16504 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:32.997524 16623 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.000s
I20260709 12:14:33.027238 11181 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:33.011942 16605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.002s	sys 0.012s
I20260709 12:14:33.010591 16504 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:33.016206 16647 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
I20260709 12:14:33.012820 11322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
W20260709 12:14:33.029227 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 239338us
I20260709 12:14:33.052541 11322 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.053025 11322 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:33.052950 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:33.053789 11253 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:32.969120 16658 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:33.051091 11926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.016s	sys 0.012s
W20260709 12:14:33.065722 11926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.017s	sys 0.012s
W20260709 12:14:33.077392 11328 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.000s
I20260709 12:14:33.078173 11328 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
W20260709 12:14:33.051851 16605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.002s	sys 0.012s
I20260709 12:14:33.052346 16504 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:33.086535 16658 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:33.090847 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:33.091568 11325 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:14:33.097474 11323 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:14:33.098264 11324 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:33.099009 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:33.099803 11327 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:33.113138 11327 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:33.014809 16623 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.118134 11322 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.138139 11325 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:33.143882 11322 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:33.063889 16647 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.165458 16623 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:33.133484 16495 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.192229 16495 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:33.195098 16057 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:33.195834 16057 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:33.156100 16654 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.008s
I20260709 12:14:33.197213 16057 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:33.219945 16683 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.002s
I20260709 12:14:33.231276 16647 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:33.243367 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.247452 16630 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:33.213954 16479 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.657s	user 0.017s	sys 0.000s
I20260709 12:14:33.248503 16479 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.249886 11178 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:33.251431 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 285080us
I20260709 12:14:33.235196 16057 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:33.235311 16640 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.008s
I20260709 12:14:33.213235 16654 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.215813 16648 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.265417 16314 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:33.277174 11252 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:33.284425 16314 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.294852 11256 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:14:33.295356 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 722597us
I20260709 12:14:33.296315 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.274998 16479 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.297557 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.306727 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:33.307953 11323 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:33.294279 16640 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:33.314049 16640 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.316383 16640 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:33.325730 11177 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:33.330803 16657 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:33.331346 16432 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.331811 11249 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:14:33.333745 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:33.334790 16648 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.335206 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.336083 16624 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:33.336624 16624 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:33.337596 16624 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.308076 16658 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:33.343823 16665 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.345189 16314 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.350083 16654 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:33.354672 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:33.350273 16504 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.353567 16202 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:33.353544 16623 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:33.353657 16209 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:14:33.358570 16665 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:33.366109 16658 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:33.378372 16454 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:33.379702 16454 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:33.380133 16454 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:33.382645 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.377701 16504 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:33.394224 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.394850 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:33.395581 11174 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:33.395749 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:33.396484 11181 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:33.398916 16630 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.399479 16630 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.401715 16630 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.403342 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.404084 16658 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:33.404196 11322 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:33.414805 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:33.415622 11178 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:33.417505 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2285) took 1011 ms. Trace:
I20260709 12:14:33.417959 11252 rpcz_store.cc:276] 0709 12:14:32.406186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.795789 (+389603us) service_pool.cc:224] Handling call
0709 12:14:32.848125 (+ 52336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:33.277810 (+429685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.417259 (+139449us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:33.417441 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986003712}
I20260709 12:14:33.418717 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:33.421653 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.424540 16647 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.425196 16647 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.427501 16647 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.430058 16652 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.430665 16652 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.434409 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:33.434780 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2293) took 1012 ms. Trace:
I20260709 12:14:33.435165 11928 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:33.435715 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:33.437012 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:33.437386 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.438126 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 955) took 1063 ms. Trace:
I20260709 12:14:33.403437 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:33.438519 11927 rpcz_store.cc:276] 0709 12:14:32.375076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.375215 (+   139us) service_pool.cc:224] Handling call
0709 12:14:33.437952 (+1062737us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54005afd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:33.438052 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558622592}
I20260709 12:14:33.440459 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:33.444159 11928 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:33.449492 16658 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:33.450398 16658 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:33.450656 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.451473 16658 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.452011 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:33.453605 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.454470 16202 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:33.455029 16202 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:33.455442 16202 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:33.455737 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.457086 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.457835 16504 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:33.458154 16629 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:33.460731 16629 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:33.461826 16209 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:33.462428 16209 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:33.462990 16209 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:33.435241 11251 rpcz_store.cc:276] 0709 12:14:32.421938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.806803 (+384865us) service_pool.cc:224] Handling call
0709 12:14:32.856225 (+ 49422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:32.917920 (+ 61695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.434518 (+516598us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b5400088ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:33.434716 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65759616}
I20260709 12:14:33.466233 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2290) took 1045 ms. Trace:
I20260709 12:14:33.466679 11257 rpcz_store.cc:276] 0709 12:14:32.420997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.803944 (+382947us) service_pool.cc:224] Handling call
0709 12:14:32.854194 (+ 50250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:32.854455 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:32.872496 (+ 18041us) log.cc:844] Serialized 14 byte log entry
0709 12:14:33.433825 (+561329us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b18002983c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:33.465843 (+ 32018us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:33.465914 (+    71us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:33.465963 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.466037 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.466167 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82794,"prepare.queue_time_us":19455,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"spinlock_wait_cycles":17858816,"thread_start_us":94876,"threads_started":1,"wal-append.queue_time_us":177850}
I20260709 12:14:33.471254 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:33.472506 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:33.472746 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:33.458719 16623 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:33.473320 11179 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:33.474278 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.460870 16334 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.475835 16334 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:33.476028 16057 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.476850 16057 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.479859 16057 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.480324 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:33.466246 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 413359us
W20260709 12:14:33.487123 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.487488 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.487758 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:33.488003 16571 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.488626 16665 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:33.488555 16571 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.474190 16609 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:33.489809 16665 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:33.491904 16665 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:33.492805 16665 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.486902 16397 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.494100 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.494418 16665 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:33.494699 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:33.495395 11256 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:33.496317 16057 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:33.487802 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2298) took 1064 ms. Trace:
I20260709 12:14:33.489785 16648 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:33.511207 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:33.490536 11249 pending_rounds.cc:85] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 12
I20260709 12:14:33.513062 11249 pending_rounds.cc:107] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 12:14:33.513375 11249 raft_consensus.cc:2889] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:33.497521 11254 rpcz_store.cc:276] 0709 12:14:32.423519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.930586 (+507067us) service_pool.cc:224] Handling call
0709 12:14:32.933274 (+  2688us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:32.934916 (+  1642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.934960 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:32.936227 (+  1267us) log.cc:844] Serialized 52 byte log entry
0709 12:14:33.018155 (+ 81928us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:33.029429 (+ 11274us) log.cc:844] Serialized 14 byte log entry
0709 12:14:33.346264 (+316835us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b18001c5b80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:33.450894 (+104630us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b18001c5b80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:14:33.451133 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 12:14:33.487443 (+ 36310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.487535 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.487587 (+    52us) raft_consensus.cc:1685] finished
0709 12:14:33.487600 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.487745 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":580181,"prepare.queue_time_us":446190,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":59812,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":9907,"spinlock_wait_cycles":47688320,"thread_start_us":81425,"threads_started":1,"wal-append.queue_time_us":280696}
I20260709 12:14:33.500746 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.501818 11174 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:33.508862 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:33.514812 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:33.512532 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:33.517127 11182 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15.
I20260709 12:14:33.498293 16518 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:14:33.520368 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:33.521225 11178 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:33.494428 16397 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.510955 16657 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.520403 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.522774 16657 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.522049 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:33.524207 16397 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.525228 16657 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.525753 16617 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:33.489991 16526 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:14:33.570197 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:33.571002 11255 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 16
W20260709 12:14:33.571552 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51688.2us
I20260709 12:14:33.574395 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:33.575146 11253 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:33.574522 11222 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15.
I20260709 12:14:33.527751 16571 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.528646 16624 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:33.529397 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.530251 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:33.593243 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63088.9us
I20260709 12:14:33.593528 11173 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:33.594684 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:33.491031 16627 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:33.530910 16121 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:33.491122 16609 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:33.530891 16652 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:33.533324 11256 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:33.533715 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2320) took 1014 ms. Trace:
I20260709 12:14:33.608851 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:33.609424 11177 rpcz_store.cc:276] 0709 12:14:32.519802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.519942 (+   140us) service_pool.cc:224] Handling call
0709 12:14:33.533502 (+1013560us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54003d22d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:33.533651 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634678016}
I20260709 12:14:33.610181 11180 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:14:33.534825 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.536377 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.538249 16549 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.539089 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:33.539186 16654 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.539287 11248 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:33.539567 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.539839 16644 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:14:33.539947 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53501.6us
I20260709 12:14:33.592656 16624 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:33.530825 16202 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.542276 11325 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:33.525763 16314 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:33.545087 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:33.545437 11075 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:33.526360 11330 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:33.578413 11222 leader_election.cc:403] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:14:33.677623 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 103339us
I20260709 12:14:33.596318 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.596957 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:33.599174 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:33.678526 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.599689 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:33.601161 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:33.617394 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:33.679684 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.680171 11326 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:14:33.611933 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 154908us
I20260709 12:14:33.620167 16617 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:33.621013 16609 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.622435 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:33.693470 11256 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:33.694053 11251 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:33.693517 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:33.623752 16121 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:33.624166 16526 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:33.624577 16627 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:33.617169 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:33.612668 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48960.3us
I20260709 12:14:33.640257 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:33.661149 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.711158 11322 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:33.608529 16545 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:33.665387 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 765) took 1244 ms. Trace:
I20260709 12:14:33.711794 11248 rpcz_store.cc:276] 0709 12:14:32.421259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.438467 (+ 17208us) service_pool.cc:224] Handling call
0709 12:14:33.665164 (+1226697us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540007ec54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:33.665315 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2492584832}
I20260709 12:14:33.666600 11252 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:33.668125 11181 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:33.618180 16234 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:33.718830 11256 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:33.595666 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:33.682125 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.687644 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.721071 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.623164 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:33.662757 16654 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.540020 16632 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:33.715242 11324 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
W20260709 12:14:33.718730 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57660.9us
I20260709 12:14:33.658546 16549 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.640070 16209 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.682823 16504 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:33.727145 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:33.730583 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 48584.3us
I20260709 12:14:33.693003 16571 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15.
I20260709 12:14:33.672971 11253 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:33.752333 11251 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:14:33.755707 11250 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:14:33.763139 11255 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
W20260709 12:14:33.733654 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 280084us
I20260709 12:14:33.710457 16057 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:33.708848 16657 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:33.708788 16526 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.736907 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.694957 16454 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.741801 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:33.784504 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47683.5us
I20260709 12:14:33.785145 11930 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15.
W20260709 12:14:33.742905 11077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 239) took 19988 ms (client timeout 15000 ms). Trace:
W20260709 12:14:33.743567 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.786332 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.731051 16644 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:33.731082 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:33.733003 16652 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:33.746903 11075 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 4 to 14, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 14 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:33.726065 16686 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:33.737476 16648 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:33.729465 16623 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:33.756569 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:33.807420 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50957.7us
I20260709 12:14:33.768584 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:33.810360 11329 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:33.697113 16397 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:14:33.739611 16632 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.731050 16545 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:33.743515 16624 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.789376 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:33.782044 16571 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:33.782105 16526 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:33.797465 16495 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:33.736537 16285 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:33.806497 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:33.838071 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 48767.5us
I20260709 12:14:33.811456 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:33.803503 16504 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:33.806907 16644 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:33.845081 16647 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:33.825174 16300 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:33.825240 16646 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:33.862252 16545 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.829000 16234 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:14:33.828827 16314 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:14:33.863099 11075 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 10, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 10 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:33.865337 16209 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:33.836556 16057 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.816756 16623 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.843706 16571 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:33.847784 16533 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.871098 16553 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:33.837796 16648 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:14:33.869428 16646 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:33.901289 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.802716 16121 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.891767 16628 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:14:33.881145 16624 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:33.954216 16454 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:33.955827 16712 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:14:33.856799 16285 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.958065 16630 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:33.958699 16630 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:33.959803 16711 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:14:33.960980 16630 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:33.962057 16202 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:33.962646 16708 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:14:33.965758 16648 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.966432 16648 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.967155 11075 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:33.879529 16652 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:14:33.890187 16057 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:33.979665 16334 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:33.830950 16627 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.980211 16334 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:33.847390 16605 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 12:14:33.997684 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 157102us
I20260709 12:14:33.968917 16648 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:34.002501 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 163421us
I20260709 12:14:33.985909 16646 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:33.980230 16057 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:34.027931 16647 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:34.028455 11324 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:34.030827 16647 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.034713 16640 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:34.039112 16640 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:34.040510 16234 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:34.041445 16640 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:34.042727 16520 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:34.043190 16520 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:34.044227 16520 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.045487 16520 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:34.051847 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.055518 16234 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:33.981256 16334 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.910135 16533 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:33.944833 16571 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:34.018672 16665 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.001s
I20260709 12:14:34.007985 16605 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:34.068472 16533 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:33.785658 11077 rpcz_store.cc:269] 0709 12:14:13.754929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.755284 (+   355us) service_pool.cc:224] Handling call
0709 12:14:13.879559 (+124275us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b5400260110. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c8640 00007f3ecc49f846 00007f3ecc5a990e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:33.742632 (+19863073us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:27.507177 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:27.712194 (+205017us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:27.712257 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:14:27.712276 (+    19us) tablet.cc:662] Decoding operations
0709 12:14:28.005171 (+292895us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:28.005320 (+   149us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:28.005363 (+    43us) tablet.cc:685] Acquiring locks for 30 operations
0709 12:14:28.009335 (+  3972us) tablet.cc:701] Row locks acquired
0709 12:14:28.009399 (+    64us) write_op.cc:260] PREPARE: finished
0709 12:14:28.009685 (+   286us) write_op.cc:270] Start()
0709 12:14:28.009885 (+   200us) write_op.cc:276] Timestamp: P: 1783599268009568 usec, L: 0
0709 12:14:28.009926 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:14:28.011000 (+  1074us) log.cc:844] Serialized 11919 byte log entry
0709 12:14:28.320735 (+309735us) op_driver.cc:464] REPLICATION: finished
0709 12:14:28.328138 (+  7403us) write_op.cc:301] APPLY: starting
0709 12:14:28.328244 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:28.346096 (+ 17852us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:28.356232 (+ 10136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:29.802934 (+1446702us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:29.841808 (+ 38874us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=60,mrs_lookups=0
0709 12:14:29.847842 (+  6034us) write_op.cc:312] APPLY: finished
0709 12:14:29.889875 (+ 42033us) log.cc:844] Serialized 260 byte log entry
0709 12:14:29.903546 (+ 13671us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:29.903888 (+   342us) write_op.cc:454] Released schema lock
0709 12:14:29.904005 (+   117us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:29.904587 (+   582us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:29.904628 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4903635,"reported_tablets":49,"spinlock_wait_cycles":5309184,"child_traces":[["op",{"apply.queue_time_us":7109,"apply.run_cpu_time_us":56500,"apply.run_wall_time_us":1576406,"cfile_cache_hit":122,"cfile_cache_hit_bytes":129767,"num_ops":30,"prepare.queue_time_us":334620,"prepare.run_cpu_time_us":10867,"prepare.run_wall_time_us":819514,"raft.queue_time_us":1083,"raft.run_cpu_time_us":7087,"raft.run_wall_time_us":13452,"replication_time_us":310607,"thread_start_us":541479,"threads_started":5,"wal-append.queue_time_us":323665,"wal-append.run_cpu_time_us":5443,"wal-append.run_wall_time_us":2247580}]]}
I20260709 12:14:34.058504 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.075186 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:34.096786 16553 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:34.097163 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:34.097498 16553 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:34.098564 16553 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.099824 16520 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:34.100513 16520 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:34.072042 16605 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:34.101567 16520 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.105885 16571 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:34.106767 16533 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:34.006953 16314 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:14:34.114177 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:34.144083 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:34.147657 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:34.148468 11249 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
W20260709 12:14:34.137768 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 355783us
W20260709 12:14:34.116954 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 389882us
I20260709 12:14:34.168897 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:34.172066 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:34.179104 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:34.183249 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:34.185779 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:34.206456 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.969283 11075 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 11, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 11 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:34.196842 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:34.219414 16654 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:34.141575 16533 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:34.223843 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:34.225359 11254 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:34.226922 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 58134.2us
I20260709 12:14:34.324100 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:34.004611 16644 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.102011 16605 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:34.250792 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 251138us
I20260709 12:14:34.256217 11328 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:14:34.186566 16658 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.164034 16234 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:14:34.220611 16121 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:34.117678 16640 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 12:14:34.232014 16057 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:34.228469 16632 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:34.293881 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:34.298035 16736 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:14:34.251204 16726 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:14:34.214797 11296 leader_election.cc:400] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
I20260709 12:14:34.328075 11222 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15.
I20260709 12:14:34.359457 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:34.335813 16654 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:14:34.338431 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.341131 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:34.340829 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.367084 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.349527 16495 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 12:14:34.381083 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:34.338019 16533 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:34.356132 16549 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:34.360512 11296 leader_election.cc:403] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:34.389424 16734 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:14:34.390442 16728 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:34.391333 16729 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:14:34.391822 16314 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.368471 16617 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:14:34.394308 16623 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:34.396711 16432 raft_consensus.cc:2764] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:14:34.405840 16728 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:34.408521 11179 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:14:34.418565 16728 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:34.387283 16605 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:34.426494 16723 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:34.427063 16723 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:34.426704 16314 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.386926 11075 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:34.428045 16723 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.408828 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:34.434258 11254 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:34.377604 16495 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:34.417192 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:34.398674 16533 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.349826 16658 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.365260 16121 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.442466 11180 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:34.429482 16723 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:34.428931 11075 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 13, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 13 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:34.442479 16623 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13.
W20260709 12:14:34.478253 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 137229us
I20260709 12:14:34.484606 16334 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:34.468364 16743 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.001s
W20260709 12:14:34.512784 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 345541us
I20260709 12:14:34.517776 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.522516 16202 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:14:34.526021 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.510871 16285 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:34.529250 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.535689 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:34.540293 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:34.545600 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.549065 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:34.561551 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.532845 16202 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:34.515193 16646 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.568346 16624 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:14:34.582091 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:34.586517 16202 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:34.537043 16703 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:34.553226 16285 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:34.652920 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 85769.2us
I20260709 12:14:34.653192 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:34.533079 16454 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 12:14:34.571122 16526 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.729s	user 0.014s	sys 0.000s
W20260709 12:14:34.683048 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 121614us
I20260709 12:14:34.652690 16749 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.588493 16652 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:34.698873 11179 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:34.666282 16285 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:34.588070 16629 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
W20260709 12:14:34.704396 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45943us
I20260709 12:14:34.665104 16202 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:34.714371 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:34.715296 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:34.717049 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:34.717917 11323 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:34.722998 11327 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:34.730144 11182 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:34.730623 11180 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:34.732638 11928 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:34.747171 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:34.749096 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:34.586833 16718 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 12:14:34.750450 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:34.757527 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:34.757790 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:34.751185 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:34.759786 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:34.760488 11322 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:34.761328 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:34.762478 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:34.753662 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:34.708129 16285 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:34.707576 16652 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:34.683333 16526 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.768425 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:34.786770 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52547us
W20260709 12:14:34.814687 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 455297us
I20260709 12:14:34.697049 16623 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:14:34.819576 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:34.825752 11253 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:34.818017 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:34.830253 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:34.830888 11177 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:34.746129 16658 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:34.791429 11931 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:34.709084 16545 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.089s
W20260709 12:14:34.799185 16755 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:14:34.798957 16703 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:34.701663 16749 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.852926 11926 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:34.799646 16741 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:34.786703 16718 consensus_queue.cc:1243] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:34.747416 16533 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:14:34.805097 11178 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:34.810633 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:34.815874 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.701269 16728 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.818315 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:34.882441 11252 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:34.817284 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.818585 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:34.834182 16630 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.886551 11255 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:34.837464 16121 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:34.826961 11249 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:34.833724 11254 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:34.846462 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:34.891317 11253 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:34.866688 11180 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:34.813573 16652 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:34.901126 11182 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
W20260709 12:14:34.874009 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63617.3us
I20260709 12:14:34.833271 16632 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:34.849932 16658 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:34.819281 11251 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:14:34.920096 11251 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 10
I20260709 12:14:34.920396 11251 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:14:34.920650 11251 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:34.882556 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:34.899235 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:34.900293 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 83092.9us
I20260709 12:14:34.922127 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:34.849681 16609 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:34.901966 11254 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:34.902460 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 86918us
I20260709 12:14:34.907215 11255 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:34.847182 16657 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:34.852125 16648 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.927862 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:34.886600 16721 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.950006 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:34.950506 11252 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:34.951558 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:34.909205 16057 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:34.874783 16703 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.909713 16737 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:34.958148 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:34.912730 16750 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:34.919006 16741 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:34.917912 16718 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:34.966849 11252 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:34.968693 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:34.946461 16397 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:34.964634 16721 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.973341 16057 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:34.976900 16760 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:14:34.978618 16397 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.943701 16658 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.986047 11075 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 11, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 11 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:34.957126 16657 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.957080 16648 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.946727 16609 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:34.975893 16737 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:34.911862 16630 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:34.944156 16624 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:34.998936 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:34.960280 16571 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.037379 16571 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.038177 16397 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.039579 16571 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.040728 16757 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:34.959314 16665 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:35.014989 11934 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:14:35.015053 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.006734 16647 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.042176 11928 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:34.886808 16202 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.042598 16647 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.044826 16745 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:34.975170 16549 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:35.047232 16057 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.055107 16630 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:35.022472 16761 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:14:35.067058 11256 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:35.017489 16334 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.000s
I20260709 12:14:35.041690 16665 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.031013 11328 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:35.033514 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:34.996922 16646 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.029956 16737 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:34.974713 16750 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:35.067764 11250 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:35.088604 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 120033us
I20260709 12:14:35.082670 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.015098 16718 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:35.030570 16763 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:14:35.024900 16495 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:35.044914 16647 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:35.069011 16549 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.081210 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:35.090847 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.088066 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2154) took 1104 ms. Trace:
I20260709 12:14:35.172324 11330 rpcz_store.cc:276] 0709 12:14:33.983274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.984933 (+  1659us) service_pool.cc:224] Handling call
0709 12:14:34.001310 (+ 16377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.001749 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.001779 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:34.403326 (+401547us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b18003b5ba0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a
0709 12:14:34.435263 (+ 31937us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b2c00126d18. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:34.448839 (+ 13576us) log.cc:844] Serialized 26 byte log entry
0709 12:14:34.569293 (+120454us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:34.569448 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.569866 (+   418us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.087840 (+517974us) raft_consensus.cc:1685] finished
0709 12:14:35.087865 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.088008 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4306,"raft.queue_time_us":128,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":12830,"spinlock_wait_cycles":5590528,"thread_start_us":75113,"threads_started":1,"wal-append.queue_time_us":452408}
I20260709 12:14:35.095052 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:35.175094 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 80131.6us
W20260709 12:14:35.136296 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 186364us
I20260709 12:14:35.178593 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:35.134809 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.079385 16665 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:35.087157 16735 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.697s	user 0.005s	sys 0.000s
I20260709 12:14:35.085273 16648 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.083688 16703 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.079120 16721 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.189989 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.075573 16764 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:35.127173 16718 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.091485 16630 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 12:14:35.135501 16495 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:35.178453 16300 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.175063 16057 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:35.082099 16652 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.210384 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.179584 16571 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:35.202945 11250 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
W20260709 12:14:35.202836 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 125654us
I20260709 12:14:35.213927 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.222422 11328 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:14:35.219974 11322 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:35.201594 16718 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:35.227150 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.228667 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.228145 16721 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:35.209523 16764 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:35.231001 16721 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.180099 16549 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:35.235466 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 45562.7us
I20260709 12:14:35.212349 16526 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.202993 16703 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:35.197302 16397 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:14:35.239893 11173 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:35.220602 16772 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:35.223143 16749 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.228722 16571 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.247406 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:35.244908 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:35.226109 16728 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:35.213655 16648 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:35.255669 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.231266 16665 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:35.292472 16285 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.240312 16721 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:35.221055 16300 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.299221 11328 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
W20260709 12:14:35.270053 16723 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.801s	user 0.008s	sys 0.005s
I20260709 12:14:35.267633 16533 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.208768 16750 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.296514 16057 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:14:35.301685 16605 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.880s	user 0.011s	sys 0.002s
I20260709 12:14:35.284457 16722 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.282454 16526 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.322135 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:35.324257 11330 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:35.325366 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:35.328063 11179 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
W20260709 12:14:35.330740 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.331157 16300 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.308980 16658 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:35.251217 16778 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 12:14:35.321506 16057 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.322598 16731 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:35.305541 16648 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.349287 16769 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.356631 16778 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:35.385927 16571 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.337742 16647 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.240289 16764 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.268311 16772 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.335273 16722 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.361644 16741 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:35.348232 16750 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:35.362370 16775 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:14:35.363191 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.366211 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:35.411952 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.412348 11253 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:35.412745 11252 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:35.283660 16495 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.336047 16657 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.381256 16397 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:35.319123 11075 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 13, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 13 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:35.377960 16658 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.386135 16778 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:35.358368 16652 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.371235 16057 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:35.321527 16533 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.395205 11330 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
W20260709 12:14:35.388566 16777 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:14:35.335788 16526 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.406955 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.407825 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.418550 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.420569 11931 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:14:35.394523 16647 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.426354 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.410148 16749 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:35.458339 11075 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 9 to 12, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 12 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:35.463204 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:35.473436 11327 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:14:35.475252 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.463685 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.478330 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.479276 11330 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:35.479616 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.480309 11174 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:35.480482 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.469591 16764 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.481030 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.481720 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.481907 11931 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:35.482458 11327 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:35.465471 16533 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.468766 16526 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:35.473205 16722 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.481132 11181 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:35.393983 16769 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.486076 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.486712 11174 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:14:35.487599 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.488863 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.489574 16769 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.490085 16749 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.490912 16314 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.493259 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.493935 11180 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:35.494004 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.494808 11178 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:35.495842 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.496081 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.496450 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:35.496622 11178 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:35.497620 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.497857 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.498605 11178 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:14:35.471370 16549 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.499764 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.499900 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.501582 16647 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:35.501771 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.501955 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.503472 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.504122 11181 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:35.509104 16762 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:35.509724 16762 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:35.512188 16723 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:35.512684 16723 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.513067 16723 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:35.515216 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.515894 11249 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:35.517064 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:35.517329 16648 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.504072 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.519025 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:35.519642 11250 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:35.468051 16778 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.522457 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.522590 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.524272 11251 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:35.525568 11251 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:35.533506 16518 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:14:35.541929 16654 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:35.544824 11251 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: [14.9-16.10]
W20260709 12:14:35.545032 16740 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.145s	user 0.007s	sys 0.005s
I20260709 12:14:35.489431 11253 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:35.533223 16640 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.546736 16640 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.547262 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:35.547351 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:35.413301 16656 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:35.549116 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70884.6us
I20260709 12:14:35.518646 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:35.550671 11254 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:35.537509 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:35.552114 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:35.526042 16730 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:35.555399 16549 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:35.555739 16730 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:35.555940 16549 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.556455 16549 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:35.556849 16640 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.565956 11250 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:35.568161 11250 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:14:35.568993 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:35.568231 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:35.533030 16314 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.575475 16314 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.577080 16526 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:35.557576 16628 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:35.577807 16526 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.578413 16526 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:35.574522 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:35.557677 16769 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:14:35.542174 16285 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.558233 16518 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:35.554988 16495 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:35.462863 16397 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.471827 16772 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.613469 11181 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:14:35.582651 16334 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:35.462605 16658 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.556661 16730 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.629417 16730 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:35.634452 11175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:35.634852 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.635571 11175 raft_consensus.cc:606] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:35.581329 16314 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.607019 16750 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.636821 16750 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.639539 16750 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.643860 16658 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.645259 16454 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:35.645812 16454 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:35.646641 16454 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.560882 16648 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:14:35.648048 16454 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:35.651099 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:35.653482 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.653585 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:35.654364 11175 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:35.584156 16628 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:35.655273 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.655918 11180 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:35.656791 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.606101 16518 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:35.627947 16786 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:14:35.557528 16656 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.628449 16334 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:35.657861 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:35.674180 11177 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:35.680683 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:35.662700 16658 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.697738 11180 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:35.666554 16518 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:35.699163 11182 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:35.708570 11175 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:35.692672 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:35.531152 11075 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:35.711212 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.717602 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:35.722041 11177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4958b19b61094e1db763fdb94852f994"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:35.722959 11177 raft_consensus.cc:606] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.723433 11177 raft_consensus.cc:614] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:35.725270 11175 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:35.741185 11251 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:14:35.744633 11251 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:35.747633 11251 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:35.655934 16628 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.753747 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2332) took 1041 ms. Trace:
I20260709 12:14:35.755575 16121 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:35.743971 16750 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:14:35.759752 16778 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:35.762871 16723 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.715919 16648 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.765763 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 973) took 1435 ms. Trace:
I20260709 12:14:35.766119 16648 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.766234 11932 rpcz_store.cc:276] 0709 12:14:34.330841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.331015 (+   174us) service_pool.cc:224] Handling call
0709 12:14:34.331535 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.332014 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.332045 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:35.318173 (+986128us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b2c000a0068. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:35.334931 (+ 16758us) log.cc:844] Serialized 26 byte log entry
0709 12:14:35.474474 (+139543us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:35.474528 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:35.474595 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.765414 (+290819us) raft_consensus.cc:1685] finished
0709 12:14:35.765436 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.765684 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":503,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":4379904,"thread_start_us":137140,"threads_started":1,"wal-append.queue_time_us":282807}
I20260709 12:14:35.768599 11931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:14:35.769896 11926 raft_consensus.cc:2368] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:35.754179 11179 rpcz_store.cc:276] 0709 12:14:34.712854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.714089 (+  1235us) service_pool.cc:224] Handling call
0709 12:14:35.753560 (+1039471us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54001112d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:35.753689 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819950336}
I20260709 12:14:35.774099 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.776229 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:35.778035 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:35.710299 16721 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.783145 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:35.783239 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:35.784983 11251 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:14:35.785198 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.788069 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.790963 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.715365 16658 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.769381 16648 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.793457 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.799670 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:35.782862 16737 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.606s	user 0.000s	sys 0.012s
I20260709 12:14:35.805523 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.780609 16723 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.784754 16121 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:35.791867 16617 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:35.787321 16654 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:35.787470 16644 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:35.785689 16728 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:14:35.795879 16657 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:35.823673 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.800771 16571 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:35.804080 16769 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:35.815274 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2333) took 1100 ms. Trace:
I20260709 12:14:35.795536 16647 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:14:35.807888 16779 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:35.853708 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2311) took 1017 ms. Trace:
I20260709 12:14:35.751201 16334 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.813906 16733 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:35.782085 16762 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.848855 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:35.836877 16718 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.002s
I20260709 12:14:35.850571 11173 rpcz_store.cc:276] 0709 12:14:34.714830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.714978 (+   148us) service_pool.cc:224] Handling call
0709 12:14:35.815081 (+1100103us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54000d8754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:35.815210 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199369600}
I20260709 12:14:35.845569 11251 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:35.832688 16617 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.844355 16657 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:35.803582 16737 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.837779 16648 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:35.883891 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:35.854138 11248 rpcz_store.cc:276] 0709 12:14:34.836048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.017204 (+181156us) service_pool.cc:224] Handling call
0709 12:14:35.017477 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:35.031795 (+ 14318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:35.031829 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:35.031879 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:35.031920 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:35.853441 (+821521us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b54001cf7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:35.853502 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.853650 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32442624}
I20260709 12:14:35.820766 16057 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.850697 16571 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.900689 16571 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:35.904052 11328 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:14:35.904481 11328 pending_rounds.cc:85] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:14:35.904706 11328 pending_rounds.cc:107] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:14:35.904935 11328 raft_consensus.cc:2889] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:35.825798 16738 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:14:35.822104 16722 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:14:35.906402 16721 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.889286 16648 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.909605 16648 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:35.911029 16526 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:35.895372 16057 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.899693 16737 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:35.911659 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:35.915299 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2324) took 1004 ms. Trace:
I20260709 12:14:35.916045 16654 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:35.854671 16772 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:35.918280 11250 rpcz_store.cc:276] 0709 12:14:34.910938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.570343 (+659405us) service_pool.cc:224] Handling call
0709 12:14:35.570622 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:35.571051 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:35.571145 (+    94us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:35.582575 (+ 11430us) log.cc:844] Serialized 26 byte log entry
0709 12:14:35.785280 (+202705us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:35.785326 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:35.785400 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.915064 (+129664us) raft_consensus.cc:1685] finished
0709 12:14:35.915088 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.915236 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5287,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":122113,"threads_started":1,"wal-append.queue_time_us":299075}
I20260709 12:14:35.921783 16654 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.922359 16654 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.923543 16057 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:35.924605 16654 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.873382 16769 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:35.965909 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:35.888206 16617 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:35.888864 16657 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:35.887703 16731 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:35.891156 16762 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:35.890849 16397 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.973729 16733 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:35.992678 16656 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:35.998335 16658 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:35.998659 16797 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:14:36.004868 16772 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:35.982779 16757 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:36.006067 16526 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:36.005985 16757 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.008385 16757 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:35.887112 16718 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.018061 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2321) took 1173 ms. Trace:
I20260709 12:14:36.017928 16733 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.018555 11253 rpcz_store.cc:276] 0709 12:14:34.845113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.546369 (+701256us) service_pool.cc:224] Handling call
0709 12:14:35.546710 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:35.547114 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:35.547144 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:35.547648 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:14:35.781613 (+233965us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:35.781683 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:35.781778 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:36.017816 (+236038us) raft_consensus.cc:1685] finished
0709 12:14:36.017839 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.017993 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1590,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5005,"thread_start_us":233565,"threads_started":1,"wal-append.queue_time_us":299414}
I20260709 12:14:36.020103 16718 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.020537 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 777) took 1176 ms. Trace:
I20260709 12:14:36.020959 11256 rpcz_store.cc:276] 0709 12:14:34.844342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.542120 (+697778us) service_pool.cc:224] Handling call
0709 12:14:35.542567 (+   447us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:35.543051 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:35.543078 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:35.552136 (+  9058us) log.cc:844] Serialized 52 byte log entry
0709 12:14:35.783491 (+231355us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:35.806478 (+ 22987us) log.cc:844] Serialized 14 byte log entry
0709 12:14:35.820965 (+ 14487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:35.821083 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:36.001867 (+180784us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:36.020173 (+ 18306us) raft_consensus.cc:1685] finished
0709 12:14:36.020289 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.020457 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14080,"prepare.queue_time_us":50729,"raft.queue_time_us":18089,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":14451,"spinlock_wait_cycles":19054720,"thread_start_us":194591,"threads_started":1,"wal-append.queue_time_us":323965}
I20260709 12:14:36.025620 16571 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:35.996949 16769 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.029979 16733 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.031999 11252 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:36.038863 11931 raft_consensus.cc:606] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:36.056547 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.061245 11926 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:36.063593 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.064193 11182 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:36.066618 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.067374 11173 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:36.067551 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:36.067919 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.067961 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.069329 11249 raft_consensus.cc:2368] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17.
I20260709 12:14:36.069962 11176 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:14:36.073056 16571 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:36.074244 16802 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:14:36.055974 16772 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.038422 16787 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:36.076683 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.079552 11223 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:36.080911 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.078228 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.068755 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.082999 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.081225 11178 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:36.089984 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.092217 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.093079 11327 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:14:36.060318 16520 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:36.088936 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.037941 16549 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.099015 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.099713 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.106547 11331 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:36.080420 11249 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:36.101357 11253 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:14:36.101728 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.102110 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.102190 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.102450 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.102541 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.103693 11176 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:14:36.115267 11249 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:14:36.109097 11927 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:36.122520 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.123490 11323 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:36.126958 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.098007 16722 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:36.129051 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:36.130806 11255 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:36.131484 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.121940 16571 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:36.136200 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:36.137920 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.138258 16733 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.138875 16733 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.141243 16733 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.146014 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.101501 16495 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:36.150681 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.150651 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:36.154047 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.112659 16234 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.155392 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:36.157407 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.162077 11327 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:36.163812 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:36.170146 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:36.171037 11181 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:36.087147 16772 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.098555 16787 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.115063 16518 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.111871 16520 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.201840 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:36.202638 11174 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:36.199186 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:36.200747 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.149199 16757 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:14:36.180822 16397 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:36.213378 11181 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:36.201305 16721 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.219583 11930 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:36.219835 16721 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.222451 16721 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.209779 11932 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:36.227800 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.229087 11327 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:36.231442 11174 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:36.246701 16549 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.173163 16234 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.262742 16234 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.266161 16787 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.268229 16234 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:36.268697 16520 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.166887 16495 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.268987 16234 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.270936 16495 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.271201 16234 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.273100 16772 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.276396 16518 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:36.278439 16733 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:14:36.278515 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 59303.4us
I20260709 12:14:36.279621 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:36.280238 11182 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:36.285991 16731 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:36.287581 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:36.290561 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.165865 16648 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.296589 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:36.293987 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:36.298847 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:36.303645 11175 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
W20260709 12:14:36.299072 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:36.304978 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:36.305365 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.307325 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.307487 11322 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:36.308133 11323 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:36.314177 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.304436 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:36.323908 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 202494us
I20260709 12:14:36.329974 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.331559 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.332485 11253 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:36.334306 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.256875 16657 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.504382 11249 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:36.422875 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.506510 11327 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:36.522254 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:36.523433 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:36.528954 11935 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:36.533017 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.533880 11323 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:36.309964 16778 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:36.533006 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.303180 16762 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:36.298300 16397 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:36.402701 16623 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:36.562444 16648 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.563060 16397 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.297914 16813 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.356s
I20260709 12:14:36.567349 16609 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.568281 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2328) took 1520 ms. Trace:
I20260709 12:14:36.568746 11254 rpcz_store.cc:276] 0709 12:14:35.047492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.747083 (+699591us) service_pool.cc:224] Handling call
0709 12:14:35.780252 (+ 33169us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:35.784520 (+  4268us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:35.838533 (+ 54013us) log.cc:844] Serialized 14 byte log entry
0709 12:14:36.028077 (+189544us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b180038ec60. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff557 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:36.317606 (+289529us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:36.459007 (+141401us) log.cc:844] Serialized 26 byte log entry
0709 12:14:36.459221 (+   214us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:36.459277 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.459383 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:36.567960 (+108577us) raft_consensus.cc:1685] finished
0709 12:14:36.568078 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.568218 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51755,"prepare.queue_time_us":49738,"prepare.run_cpu_time_us":263,"prepare.run_wall_time_us":2360,"raft.queue_time_us":26896,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":187533,"spinlock_wait_cycles":2854528,"thread_start_us":415121,"threads_started":1,"wal-append.queue_time_us":695606}
I20260709 12:14:36.309705 16234 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.401701 16733 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:36.327167 16454 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:36.311050 16658 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.309762 16730 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.243s
I20260709 12:14:36.505499 16617 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.542073 16778 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:36.593415 11327 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:36.594055 11327 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:36.574302 16520 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:36.550590 11251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.002s	sys 0.019s
W20260709 12:14:36.606884 11251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.002s	sys 0.019s
I20260709 12:14:36.590026 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:36.595047 11327 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.609248 11327 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:36.611869 16632 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:36.613005 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.613719 11331 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 12.
I20260709 12:14:36.614777 16730 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:36.615484 16730 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.616166 16657 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:36.557174 16652 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:14:36.587740 16623 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:36.635861 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:36.636544 11179 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:36.637810 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:36.627682 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.621071 16658 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:36.634160 11329 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:14:36.634407 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:36.614809 16816 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:14:36.568632 16762 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.635130 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:36.600250 16778 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:36.637019 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:36.625115 16495 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:36.605990 16520 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.652285 16495 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.586741 16234 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.671417 11222 leader_election.cc:400] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:36.671972 11222 leader_election.cc:403] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:36.653786 16658 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.601778 16733 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:36.659915 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.675536 11329 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:36.675520 11256 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:14:36.664193 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 999) took 1150 ms. Trace:
I20260709 12:14:36.676906 11328 rpcz_store.cc:276] 0709 12:14:35.513407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.588471 (+ 75064us) service_pool.cc:224] Handling call
0709 12:14:35.588807 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:35.905381 (+316574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.663879 (+758498us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54000c1cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:36.664117 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724129536}
I20260709 12:14:36.664906 16504 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.665249 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2619) took 2489 ms. Trace:
W20260709 12:14:36.672861 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 354054us
I20260709 12:14:36.679307 11327 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 12:14:36.685408 11327 pending_rounds.cc:85] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 5
I20260709 12:14:36.685776 11327 pending_rounds.cc:107] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:14:36.686260 11327 raft_consensus.cc:2889] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:36.694315 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.605508 16819 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:14:36.695353 11325 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:36.694401 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.700006 11175 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:36.666616 16520 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:36.658202 16657 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:36.680588 11324 rpcz_store.cc:276] 0709 12:14:34.176095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.176328 (+   233us) service_pool.cc:224] Handling call
0709 12:14:34.176779 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.470126 (+293347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.470174 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:34.986767 (+516593us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b180038f260. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a
0709 12:14:35.400287 (+413520us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b2c00126d18. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:35.411090 (+ 10803us) log.cc:844] Serialized 26 byte log entry
0709 12:14:35.737538 (+326448us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:35.738846 (+  1308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.664892 (+926046us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54000e6854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:36.664949 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:36.665012 (+    63us) raft_consensus.cc:1685] finished
0709 12:14:36.665029 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.665182 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3809,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":2391,"spinlock_wait_cycles":724843392,"thread_start_us":188708,"threads_started":1,"wal-append.queue_time_us":389980}
I20260709 12:14:36.667246 16495 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:36.623914 16454 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:36.705312 11329 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:14:36.706060 11329 pending_rounds.cc:85] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:36.706475 11329 pending_rounds.cc:107] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:14:36.706889 11329 raft_consensus.cc:2889] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:36.709432 11179 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
W20260709 12:14:36.712314 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:36.714112 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:36.718009 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.680280 16813 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:36.649567 16623 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.704067 16657 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.721242 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.723879 11331 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:36.684768 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:36.733304 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.734467 11324 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:36.693825 16747 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:14:36.719810 16658 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:36.737011 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.720662 16057 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:36.696277 16504 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.784919 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:36.761201 11254 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:36.738637 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.770694 16730 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:36.726346 16657 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:36.782107 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.782420 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.711328 16300 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:14:36.788270 11329 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:36.798012 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:36.803982 11323 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:36.807794 16549 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:36.844746 11331 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:14:36.787397 16658 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.834438 16750 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:36.859917 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:36.764397 16733 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.842020 16617 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:36.863719 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2336) took 1387 ms. Trace:
I20260709 12:14:36.864259 11255 rpcz_store.cc:276] 0709 12:14:35.475978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.874955 (+398977us) service_pool.cc:224] Handling call
0709 12:14:35.875207 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.134734 (+259527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.863395 (+728661us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:36.863642 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587408256}
I20260709 12:14:36.868424 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1000) took 1354 ms. Trace:
W20260709 12:14:36.878523 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 163004us
I20260709 12:14:36.858119 16750 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:36.884044 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:36.884750 11256 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:36.872483 11326 rpcz_store.cc:276] 0709 12:14:35.538699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.732328 (+193629us) service_pool.cc:224] Handling call
0709 12:14:35.732818 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:35.834391 (+101573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:35.834423 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:35.845549 (+ 11126us) log.cc:844] Serialized 26 byte log entry
0709 12:14:36.059986 (+214437us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:36.060039 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.865723 (+805684us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b54002f67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:36.865792 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:36.865869 (+    77us) raft_consensus.cc:1685] finished
0709 12:14:36.865891 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.868352 (+  2461us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3266,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":2684,"spinlock_wait_cycles":201736576,"thread_start_us":169748,"threads_started":1,"wal-append.queue_time_us":283149}
W20260709 12:14:36.872257 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 187617us
I20260709 12:14:36.888339 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:36.862815 16830 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:36.940028 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:36.948786 11324 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:36.941007 11326 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:14:36.871821 16549 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:36.879698 16730 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.832057 16454 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:36.871789 16829 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:37.059120 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:37.061347 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 168665us
I20260709 12:14:37.062907 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:37.063207 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.940930 16234 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:37.073011 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 124583us
I20260709 12:14:37.074045 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:37.088455 11329 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:36.931661 16747 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.134s
W20260709 12:14:37.090015 11148 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.556s (SENT) [suppressed 12 similar messages]
I20260709 12:14:37.099979 16234 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.086568 16648 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:37.073486 16797 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:14:37.089648 16772 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:37.090015 16623 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:37.092561 16549 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:37.138787 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 775) took 2287 ms. Trace:
I20260709 12:14:37.065919 16730 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.065331 16837 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:14:37.139416 11257 rpcz_store.cc:276] 0709 12:14:34.835611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.977130 (+141519us) service_pool.cc:224] Handling call
0709 12:14:34.997061 (+ 19931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:35.087969 (+ 90908us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:14:35.175739 (+ 87770us) log.cc:844] Serialized 14 byte log entry
0709 12:14:35.301376 (+125637us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:35.301434 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:35.301479 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.009533 (+708054us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b1800368980. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:36.026682 (+ 17149us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b1800368980. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419
0709 12:14:37.122381 (+1095699us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b54002a4ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:37.122456 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.122653 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20500,"prepare.queue_time_us":93749,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":3368,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":5982,"spinlock_wait_cycles":231719424,"thread_start_us":121992,"threads_started":1,"wal-append.queue_time_us":138703,"wal-append.run_cpu_time_us":1071,"wal-append.run_wall_time_us":1016655}
W20260709 12:14:37.156373 16835 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:14:37.054914 16830 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:37.143332 16722 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.989s	user 0.014s	sys 0.000s
W20260709 12:14:37.163637 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 100839us
I20260709 12:14:37.085914 16750 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:37.188162 16831 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 12:14:37.179117 16846 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:37.179329 16722 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:37.150055 16772 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.135113 16648 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.189602 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 789) took 1644 ms. Trace:
I20260709 12:14:37.199396 16495 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.201216 16730 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.202059 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:37.203496 16495 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:37.159137 16623 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:14:37.199419 11248 rpcz_store.cc:276] 0709 12:14:35.545501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.890112 (+344611us) service_pool.cc:224] Handling call
0709 12:14:35.890552 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:36.301430 (+410878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:36.301478 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:36.502217 (+200739us) log.cc:844] Serialized 26 byte log entry
0709 12:14:36.714170 (+211953us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:36.714207 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.714293 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:37.189340 (+475047us) raft_consensus.cc:1685] finished
0709 12:14:37.189360 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.189540 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16377,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":585512,"threads_started":2,"wal-append.queue_time_us":288188}
I20260709 12:14:37.188174 16549 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:37.209002 16817 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.627s	user 0.007s	sys 0.002s
I20260709 12:14:37.211253 16727 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:37.211692 16727 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:37.212621 16727 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.163794 16504 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:37.215543 16832 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:14:37.216909 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2633) took 1879 ms. Trace:
I20260709 12:14:37.217084 16749 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.217267 11330 rpcz_store.cc:276] 0709 12:14:35.337590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.572826 (+235236us) service_pool.cc:224] Handling call
0709 12:14:35.610470 (+ 37644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:35.610966 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:35.611005 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:35.611584 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 12:14:36.609546 (+997962us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b180028ff60. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:14:36.790427 (+180881us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:36.790492 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.792722 (+  2230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:37.216616 (+423894us) raft_consensus.cc:1685] finished
0709 12:14:37.216635 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.216854 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2577,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":3175,"spinlock_wait_cycles":5536640,"thread_start_us":180466,"threads_started":1,"wal-append.queue_time_us":1317190}
W20260709 12:14:37.218659 16822 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.521s	user 0.009s	sys 0.002s
I20260709 12:14:37.220274 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2666) took 1067 ms. Trace:
I20260709 12:14:37.221024 16571 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.191089 16658 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:37.221915 16658 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:14:37.222306 11178 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2660) took 1721 ms (client timeout 1904 ms). Trace:
W20260709 12:14:37.223001 11178 rpcz_store.cc:269] 0709 12:14:35.500915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.501708 (+   793us) service_pool.cc:224] Handling call
0709 12:14:37.222105 (+1720397us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54000da054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:37.222237 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328931712}
I20260709 12:14:37.121923 16234 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.220616 11176 rpcz_store.cc:276] 0709 12:14:36.152454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.152631 (+   177us) service_pool.cc:224] Handling call
0709 12:14:36.174435 (+ 21804us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:36.174940 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:36.174993 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:36.175595 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:14:36.581578 (+405983us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:36.581670 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.581783 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:37.218027 (+636244us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:37.219997 (+  1970us) raft_consensus.cc:1685] finished
0709 12:14:37.220019 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.220216 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1835,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":4724,"spinlock_wait_cycles":7480064,"thread_start_us":372424,"threads_started":1,"wal-append.queue_time_us":486456}
I20260709 12:14:37.224880 16571 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:37.225672 16234 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.196219 16648 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:37.194410 16833 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
W20260709 12:14:37.236981 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.238332 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.239270 11927 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:37.239923 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 985) took 1128 ms. Trace:
I20260709 12:14:37.240345 11927 rpcz_store.cc:276] 0709 12:14:36.112330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.237986 (+1125656us) service_pool.cc:224] Handling call
0709 12:14:37.239844 (+  1858us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.222391 16658 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:37.229041 16623 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:14:37.257988 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 87950.2us
I20260709 12:14:37.260309 11296 leader_election.cc:400] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17.
I20260709 12:14:37.260763 11296 leader_election.cc:403] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:37.262012 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.196581 16772 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.238181 16846 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:37.277654 16846 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:37.280583 16778 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.001s
I20260709 12:14:37.283890 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:37.246161 16847 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:37.267797 16234 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:37.299707 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 211910us
I20260709 12:14:37.302299 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:37.325244 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51216.5us
W20260709 12:14:37.342073 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 782) took 2271 ms (client timeout 2006 ms). Trace:
W20260709 12:14:37.342715 11252 rpcz_store.cc:269] 0709 12:14:35.070246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.425968 (+355722us) service_pool.cc:224] Handling call
0709 12:14:37.335258 (+1909290us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:37.341939 (+  6681us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279646080}
I20260709 12:14:37.345858 16858 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17.
I20260709 12:14:37.346930 16858 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:37.347373 16858 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.347761 16858 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:37.341733 16855 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.349144 16855 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:37.350396 16520 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.001s
I20260709 12:14:37.350885 16520 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.351917 16855 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:37.351719 16658 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.353097 16520 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.353803 11324 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:37.354046 16623 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.354686 16623 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.356858 16623 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:37.335247 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 182787us
I20260709 12:14:37.289812 16730 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:14:37.281283 16778 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.334343 16839 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:37.369980 16657 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.004s
W20260709 12:14:37.379114 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 289093us
I20260709 12:14:37.333006 16234 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.390621 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.376295 16839 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.372095 16829 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:37.399879 16854 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:37.414633 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:37.408380 16623 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:37.415656 16830 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.398459 16733 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.417073 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:37.390475 16648 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:37.434859 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.423344 16849 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.432542 11176 raft_consensus.cc:606] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.423583 16772 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.423738 16722 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:37.424441 11932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 857) took 1348 ms (client timeout 1583 ms). Trace:
I20260709 12:14:37.425508 16609 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.409155 16454 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:37.298246 16847 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.426843 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:37.448575 11250 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:37.386140 16841 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:14:37.466176 16627 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:37.466801 16627 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:37.428097 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 981) took 1339 ms. Trace:
I20260709 12:14:37.401998 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:37.421378 16829 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.435676 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:37.428606 16839 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:37.432276 16854 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.444000 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:37.445434 16623 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:14:37.443529 11932 rpcz_store.cc:269] 0709 12:14:36.076511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.076673 (+   162us) service_pool.cc:224] Handling call
0709 12:14:37.424247 (+1347574us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54004c2554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:37.424380 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232557440}
I20260709 12:14:37.450064 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.427737 11328 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:14:37.443552 16733 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.452450 16609 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.443815 16648 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.456579 16549 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.457062 16830 raft_consensus.cc:2764] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:14:37.442973 16849 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.459568 16836 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:14:37.460260 16847 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:37.460990 16778 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.437826 16721 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.473502 11257 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:37.493724 16623 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:37.468058 11930 rpcz_store.cc:276] 0709 12:14:36.088418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.088825 (+   407us) service_pool.cc:224] Handling call
0709 12:14:36.111891 (+ 23066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.220109 (+108218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.427869 (+1207760us) spinlock_profiling.cc:243] Waited 98.1 ms on lock 0x7b540034f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:37.428039 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225697664}
I20260709 12:14:37.391710 16657 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:37.501377 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51377.6us
I20260709 12:14:37.507887 16846 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:37.466822 16841 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:37.488731 16757 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:37.493253 16854 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:37.474797 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72907us
I20260709 12:14:37.535848 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.537592 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:37.523612 16727 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:37.524358 16623 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.469296 16722 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.495962 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:37.448954 16772 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.533401 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.518025 16733 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.556007 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:37.557005 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:37.559019 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63131.6us
I20260709 12:14:37.530367 16841 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:37.581449 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 112854us
I20260709 12:14:37.558712 16397 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:37.588487 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:37.528311 16846 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:37.548070 16623 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:37.542034 16744 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:37.594112 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:37.529220 16721 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.512840 16549 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.551339 16749 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:37.579975 16658 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:37.554598 16750 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:37.576563 11257 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:37.599260 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.550411 16722 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.580821 16757 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:37.582819 16847 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:37.541024 16778 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:14:37.596443 16846 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:37.580624 16504 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:14:37.585904 16836 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:37.614403 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:37.586241 16853 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:37.640901 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:37.644001 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:37.609270 16828 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:37.644716 11174 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:37.579758 16454 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:14:37.609735 16749 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.644575 16830 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:37.605193 16432 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:14:37.648114 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:37.609313 16658 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:37.620434 16730 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:14:37.651278 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57307.7us
I20260709 12:14:37.617897 16757 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.653082 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:37.652274 16830 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.651386 16749 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:37.659901 11175 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:37.660856 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:37.638226 16722 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:14:37.598378 16397 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.648348 16853 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.664515 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:37.767676 11325 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
W20260709 12:14:37.663445 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4568750220000766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
I20260709 12:14:37.663318 16856 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.663599 16854 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:37.663285 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52378us
I20260709 12:14:37.656627 16658 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:37.708076 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.618105 16571 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:37.694214 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:37.796748 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 88805.7us
I20260709 12:14:37.797819 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:37.799324 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:37.800822 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.656888 16285 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:37.642576 16841 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.706511 16875 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.772694 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:37.803874 16285 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.805411 16285 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:37.774299 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.770285 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:37.809700 16840 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:37.810333 16840 consensus_queue.cc:1243] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:37.810590 16878 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:37.810995 16878 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.811360 16878 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:37.812959 16836 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:37.781988 16849 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.650403 16828 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.787189 16762 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:37.789605 16397 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.790189 16549 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:37.790861 16721 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.791220 16876 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.815155 16876 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.817304 16876 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.704264 16609 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.715178 16757 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:37.818567 16880 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:37.819051 16880 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.819454 16880 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:37.715559 16830 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:37.650565 16858 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.823282 16858 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.804363 16730 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:14:37.824579 16878 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.825676 16609 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.825868 16571 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.826229 16609 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:37.827870 16826 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.975s	user 0.001s	sys 0.003s
I20260709 12:14:37.828454 16609 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:37.830307 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 980) took 1746 ms. Trace:
I20260709 12:14:37.830682 11933 rpcz_store.cc:276] 0709 12:14:36.084440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.084591 (+   151us) service_pool.cc:224] Handling call
0709 12:14:36.086893 (+  2302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.087141 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:36.206202 (+119061us) log.cc:844] Serialized 14 byte log entry
0709 12:14:36.556191 (+349989us) log.cc:844] Serialized 14 byte log entry
0709 12:14:36.775055 (+218864us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b1800295a20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219
0709 12:14:37.521899 (+746844us) spinlock_profiling.cc:243] Waited 5.88 ms on lock 0x7b1800295a20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:37.829831 (+307932us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b1800295a20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:37.829982 (+   151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.830032 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:37.830083 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.830144 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.830253 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1280957,"prepare.queue_time_us":1416599,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":6576,"raft.queue_time_us":1295000,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":164,"spinlock_wait_cycles":62578048,"thread_start_us":421179,"threads_started":2,"wal-append.queue_time_us":422590,"wal-append.run_cpu_time_us":1516,"wal-append.run_wall_time_us":1001542}
I20260709 12:14:37.833703 16334 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:37.834064 16648 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.834920 16841 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:37.839046 16769 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:37.839221 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.839650 16769 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.840452 16875 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.841001 11932 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:37.838824 16876 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:37.842100 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:37.842767 11927 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:37.834143 16334 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.843504 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.683074 16772 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:37.844745 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72133.3us
I20260709 12:14:37.847980 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:37.855198 11248 raft_consensus.cc:2368] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15.
I20260709 12:14:37.849747 11323 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:14:37.855675 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.803767 16495 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:37.853106 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:37.863853 11928 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:37.864017 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:37.864869 11325 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:37.860461 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:37.853991 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50537.3us
I20260709 12:14:37.850150 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:37.855926 11323 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:37.867774 11935 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:37.858901 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:37.870370 11328 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:37.794876 16855 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.873571 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.881287 11929 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:37.874562 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:37.879267 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.884397 11928 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:14:37.879663 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:37.886590 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:37.888432 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.889387 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.890398 11248 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:37.893882 11175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:37.894490 11175 raft_consensus.cc:606] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:37.868201 11323 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.895488 11323 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:37.880235 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:37.896390 11255 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:37.897770 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.898236 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:37.893343 16741 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:37.901409 16504 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:14:37.904393 16769 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:37.910779 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.917164 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 988) took 1591 ms. Trace:
I20260709 12:14:37.917578 11935 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:37.918298 16875 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:37.897396 16855 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:37.932631 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.802894 16856 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:37.843127 16853 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:37.857781 16334 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:37.938316 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.948378 11933 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:14:37.952301 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55802.8us
I20260709 12:14:37.962549 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:37.966567 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.004650 11325 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:37.898403 11930 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
W20260709 12:14:37.976380 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 89922.9us
I20260709 12:14:37.977021 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:37.976447 11254 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:37.986980 11177 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:14:38.045504 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.046190 11254 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:38.047312 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:38.048623 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.020673 16632 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:37.991253 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:38.055051 11255 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:38.038199 16495 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:38.062219 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.065086 11325 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:37.971730 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.007190 11175 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:38.007181 11257 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:14:37.913625 16057 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:38.008976 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.009977 11248 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:38.012787 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:37.882140 16727 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 12:14:37.974993 16807 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.014766 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.012928 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:38.130999 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 124248us
I20260709 12:14:37.975404 16870 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:38.016567 11179 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:38.018532 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:37.996213 16780 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.017256 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.000144 16840 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:14:38.028614 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.006117 16722 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:37.947402 16658 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:38.018031 16839 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.001s
I20260709 12:14:37.980265 16757 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.040596 11323 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:14:38.141695 11323 pending_rounds.cc:85] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 11
I20260709 12:14:38.141968 11323 pending_rounds.cc:107] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:14:38.142210 11323 raft_consensus.cc:2889] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:38.020500 16741 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:37.944118 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:38.153051 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 209038us
I20260709 12:14:38.025835 16879 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.998312 11178 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:14:38.070969 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.055801 16854 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.101356 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:38.160032 11257 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:38.160171 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.101929 11249 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:38.102480 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 221929us
I20260709 12:14:38.161101 11178 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:14:38.104058 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.056551 16632 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:38.165196 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.058455 16856 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.104733 11256 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:38.137164 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:38.140291 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 278276us
I20260709 12:14:38.062492 16836 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:38.052677 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.170240 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:38.170428 11254 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:38.143064 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.146145 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:38.149756 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.171736 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2186) took 1478 ms. Trace:
I20260709 12:14:38.151005 11325 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:38.172201 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.167157 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:38.139669 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 236034us
I20260709 12:14:38.152133 11926 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:38.101924 16057 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:38.180369 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.185010 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:38.185786 11249 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:38.172904 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.075119 16830 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.162274 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.139191 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:38.102246 16454 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:38.060729 16495 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.222357 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.165776 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.132053 16504 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:38.140691 16750 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.004s
I20260709 12:14:38.169621 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:38.187934 11325 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:38.180835 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1010) took 1152 ms. Trace:
I20260709 12:14:38.230444 11175 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:38.230355 11326 rpcz_store.cc:276] 0709 12:14:37.028901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.073891 (+ 44990us) service_pool.cc:224] Handling call
0709 12:14:37.074314 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.206652 (+132338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.206693 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.206738 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:37.206771 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:38.180513 (+973742us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003237d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:38.180580 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:38.180751 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303804160}
I20260709 12:14:38.182873 11329 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:38.232223 11927 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:38.172852 11180 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
W20260709 12:14:38.174644 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 103796us
I20260709 12:14:38.234295 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:38.237926 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:38.238332 11325 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:38.137709 16807 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.250572 11934 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: [11.9-15.11]
I20260709 12:14:38.251813 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.252444 11182 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:14:38.172112 11327 rpcz_store.cc:276] 0709 12:14:36.692985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.730624 (+ 37639us) service_pool.cc:224] Handling call
0709 12:14:36.731011 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.862213 (+131202us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:37.058175 (+195962us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.428655 (+370480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.428712 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:37.428756 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:38.171533 (+742777us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400331dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:38.171575 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:38.171685 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5634,"prepare.queue_time_us":256144,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":2826,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":6464,"spinlock_wait_cycles":301129088,"thread_start_us":559853,"threads_started":2,"wal-append.queue_time_us":383640}
I20260709 12:14:38.177866 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.152328 16752 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.194581 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:38.162153 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.193327 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.250092 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:38.188411 16829 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:38.162904 16772 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:38.285454 16772 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:38.285956 16772 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:38.292296 16828 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.259090 16629 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:38.297219 16629 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:38.172937 16879 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:38.188431 16623 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.003s
I20260709 12:14:38.201108 16454 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.201560 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.201933 16884 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:38.188198 16876 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.120s
I20260709 12:14:38.156107 16839 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.373569 16876 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:38.374045 16876 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:38.374617 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:38.203195 16881 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:14:38.375324 16839 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.374908 16876 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.204129 16880 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.206035 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:38.207499 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:38.307929 16829 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.377117 16829 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:38.192276 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:38.377803 11173 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:38.209271 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.187358 16851 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.210784 16300 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:38.210964 16757 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.218161 16854 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.132582 16780 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.219637 11251 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:38.220700 16658 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.167676 16520 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:38.187777 16730 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.188611 16553 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:38.257875 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:38.193357 16846 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
W20260709 12:14:38.198246 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 59223.2us
I20260709 12:14:38.260586 11325 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:38.255751 16640 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:38.265875 16571 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:38.266299 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.267470 16495 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:38.273813 16873 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.585s	user 0.003s	sys 0.010s
I20260709 12:14:38.434120 16571 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:38.257364 16504 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.274857 11322 raft_consensus.cc:1217] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[9.6-16.9]   Dedup: 11.7->[14.8-16.9]
I20260709 12:14:38.435144 16571 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.435828 11322 pending_rounds.cc:85] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 7
I20260709 12:14:38.436139 11322 pending_rounds.cc:107] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:14:38.436372 11322 raft_consensus.cc:2889] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:38.276175 16807 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.276898 11932 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:14:38.276870 16285 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:38.438762 16749 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.002s
I20260709 12:14:38.439275 16749 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.440090 16285 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.277362 16752 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.442976 16752 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.278613 16057 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.444655 16334 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.445273 16057 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.278667 16617 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:38.279609 16644 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:38.447229 16644 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:38.448159 16644 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.138396 16870 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.414979 16520 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.451437 16870 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.154088 16722 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.451768 16520 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.340057 16830 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.453118 11248 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:38.453565 16722 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:37.917932 11931 rpcz_store.cc:276] 0709 12:14:36.326113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.326301 (+   188us) service_pool.cc:224] Handling call
0709 12:14:37.425733 (+1099432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.425976 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:37.509536 (+ 83560us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.868848 (+359312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.868900 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:37.868944 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.869005 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.916895 (+ 47890us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54004c2550. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:37.917068 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217641,"prepare.queue_time_us":96001,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":223828,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":15559,"spinlock_wait_cycles":2527612672,"thread_start_us":218132,"threads_started":2,"wal-append.queue_time_us":258171}
I20260709 12:14:38.299762 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.456104 11324 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:38.456375 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.457049 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.303821 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.192904 16856 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:38.199821 11254 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-15.11]   Dedup: 11.9->[13.10-15.11]
I20260709 12:14:38.461916 11254 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: [11.9-15.11]
I20260709 12:14:38.309150 11329 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:38.168195 16648 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
W20260709 12:14:38.202310 16883 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 12:14:38.316959 16882 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 12:14:38.346851 16827 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.576s	user 0.010s	sys 0.019s
I20260709 12:14:38.192750 16836 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.346570 16632 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.359431 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.361430 16627 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.348882 16657 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.353086 16855 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.364073 16747 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.341123 16629 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.361196 16772 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.501219 11179 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:38.380916 16884 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.338068 16741 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.382578 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.385535 11178 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:38.388520 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.488097 16057 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:38.388110 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.388998 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.396440 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:38.514098 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:38.396288 16300 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:38.332630 16889 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:14:38.402436 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:38.353253 16841 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.003s
I20260709 12:14:38.517501 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.518211 11178 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:38.395309 11249 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:38.519376 11252 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:38.519794 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:38.520387 11249 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:38.413976 16819 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:38.377732 16880 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.385675 16234 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.356469 16454 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.419601 11182 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:38.420538 16658 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.422544 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.529968 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:38.422719 11935 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:38.530745 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.423380 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:38.401553 16876 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:38.398103 16851 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.433123 16780 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.446123 16730 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.459316 11175 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:38.351941 16879 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.460819 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:38.463215 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:38.463816 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 269349us
I20260709 12:14:38.465274 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.546434 11324 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:38.547483 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.528640 16853 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.462584 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:38.526921 16640 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:38.553086 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 90590.3us
I20260709 12:14:38.486861 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:38.489110 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 224243us
I20260709 12:14:38.503473 16747 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.371587 16878 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.404181 16854 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.563356 11900 leader_election.cc:400] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15.
I20260709 12:14:38.563679 11900 leader_election.cc:403] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:14:38.501748 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.455816 16846 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.471714 16856 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.503774 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:38.509382 11926 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:38.379566 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.522789 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.570490 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:38.524112 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.572224 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.572947 11178 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:38.524302 11331 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:14:38.523710 16520 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:14:38.508857 16627 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:38.628118 16829 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.630964 16829 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.423058 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:38.497309 16830 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:14:38.533896 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2369) took 1082 ms. Trace:
I20260709 12:14:38.534996 11257 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:38.647209 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.647637 11250 rpcz_store.cc:276] 0709 12:14:37.451281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.451434 (+   153us) service_pool.cc:224] Handling call
0709 12:14:37.466374 (+ 14940us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:37.466817 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.466862 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:37.467519 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:14:37.772859 (+305340us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:37.772902 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.772969 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:38.533574 (+760605us) raft_consensus.cc:1685] finished
0709 12:14:38.533599 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:38.533823 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37565,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":14306,"thread_start_us":304926,"threads_started":1,"wal-append.queue_time_us":553230}
I20260709 12:14:38.646836 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:38.632308 11173 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:38.632633 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69385.5us
I20260709 12:14:38.652109 11251 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:38.655481 16617 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:38.660130 16846 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.526027 16553 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.661386 16632 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.662173 11326 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:38.661922 16553 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.662389 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:38.533847 16658 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.663183 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:38.663743 11182 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:38.664215 16553 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.521991 16057 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.664988 16658 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.665160 16854 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.666153 16057 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.548084 11324 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:38.531226 16819 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.473045 11181 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:14:38.550807 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.470105 16722 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:38.529867 16841 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:38.556736 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69986.9us
I20260709 12:14:38.686200 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:38.559867 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:38.687439 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 127652us
I20260709 12:14:38.566877 11325 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:38.535955 16749 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.573799 11331 pending_rounds.cc:85] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:38.693787 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:38.576402 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.694762 11178 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
W20260709 12:14:38.616103 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.539074 16870 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:38.621531 16856 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:38.622406 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 118709us
I20260709 12:14:38.623170 16893 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:38.626411 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:38.566298 16657 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:38.667927 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.541240 11249 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:38.711382 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.712030 11178 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:38.712247 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.472141 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.562136 16640 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:38.670953 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.656225 16334 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:38.717388 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46508.2us
I20260709 12:14:38.718477 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.671645 11177 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:14:38.673686 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.723086 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.723790 11178 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:38.724706 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:38.725265 11178 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:38.675225 11251 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:38.560966 16648 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:38.678176 11324 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:38.732069 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.734746 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.735466 11324 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:38.692708 11182 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:38.694121 11331 pending_rounds.cc:107] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:14:38.740949 11331 raft_consensus.cc:2889] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:38.508404 16884 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:38.698311 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:38.750039 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51847.9us
I20260709 12:14:38.709932 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:38.716423 11173 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:14:38.729250 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.707036 16878 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.759969 16855 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:38.760497 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:38.761194 16855 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:38.761519 11930 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:38.562484 16853 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.764355 16853 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.713341 16870 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.767567 16870 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.743734 16234 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:38.733232 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.777563 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.778640 11250 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:38.779137 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.779986 11252 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:38.780335 11250 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
W20260709 12:14:38.754714 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44864.7us
I20260709 12:14:38.783344 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:38.755859 11182 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:14:38.790149 11252 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:14:38.792479 11256 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:14:38.753697 11251 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:38.693409 16894 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:14:38.759931 16057 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:38.793848 16057 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.793911 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:38.794243 16894 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:38.794855 16894 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:38.707445 16632 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:38.796283 16632 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:38.795977 16894 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.796831 16632 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:38.706638 16893 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:38.797851 16893 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:38.705997 16609 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.802487 16836 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.761649 16855 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:38.760918 16878 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.802826 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.803612 16855 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.804044 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.804699 16878 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.804881 16855 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:38.805274 16894 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:38.746568 16880 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:38.802096 16609 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.794292 16057 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:38.805994 16730 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.807335 16609 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.707878 16841 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:38.808843 16829 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:38.711078 16657 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.810001 16657 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:38.810283 11325 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:38.811371 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.812264 11329 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:38.813095 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.814636 16747 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.716662 16875 raft_consensus.cc:2764] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:14:38.816924 11178 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:38.818207 16819 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.819697 16772 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:38.696151 16722 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.809367 16829 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:38.818068 16057 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.827261 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.827451 16851 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.755208 16846 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:38.828590 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.748571 16658 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.829716 16658 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:38.832485 16658 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:38.832741 16504 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:38.834162 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74670.5us
I20260709 12:14:38.837646 11179 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:14:38.829407 11326 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:38.844295 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.845284 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.845664 11250 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:14:38.838997 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55797.9us
I20260709 12:14:38.849506 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:38.730620 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:38.851310 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 120761us
I20260709 12:14:38.852304 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:38.853042 11326 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:38.858307 16772 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:38.858930 16628 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:38.842353 16648 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:38.859547 16772 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:38.859856 16648 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:38.860010 16772 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:38.860308 16648 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:38.860419 16772 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:38.861810 16875 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:38.862234 16875 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:38.864046 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.864601 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.848120 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:38.867969 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.877461 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.878683 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:38.876132 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:38.881418 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:38.809197 16841 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:38.882552 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 307833us
I20260709 12:14:38.882043 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:38.889796 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:38.904682 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:38.840170 16846 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:14:38.865612 16057 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:38.877569 16658 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:38.876893 16808 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:38.883220 16628 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:14:38.881229 16730 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:14:38.883376 16856 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:38.893666 16769 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:38.883553 16778 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:39.029619 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.031175 11330 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
W20260709 12:14:39.045606 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 152674us
I20260709 12:14:38.857482 11324 raft_consensus.cc:2410] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:14:39.048369 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.049696 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.050022 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:39.050156 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:39.050557 11324 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:39.051478 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:39.052634 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:39.053341 11248 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:39.053838 16854 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:39.054291 16854 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.054679 16854 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:39.057818 16808 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:39.058861 16808 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.067591 11325 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:39.069311 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.070904 11324 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:39.073031 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.077010 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.078897 11324 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:39.080907 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:39.090117 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:39.089532 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 213451us
I20260709 12:14:39.099511 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:39.048930 16909 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:14:39.237288 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.238344 11322 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:39.241329 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2663) took 1375 ms. Trace:
W20260709 12:14:39.243549 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.231781 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2671) took 1047 ms. Trace:
I20260709 12:14:39.246485 11324 rpcz_store.cc:276] 0709 12:14:38.184164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.229985 (+1045821us) service_pool.cc:224] Handling call
0709 12:14:39.230950 (+   965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.231319 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:39.231364 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:39.231425 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:39.231472 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.231547 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.231703 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:39.242981 11331 rpcz_store.cc:276] 0709 12:14:37.866218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.868960 (+  2742us) service_pool.cc:224] Handling call
0709 12:14:37.959314 (+ 90354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:38.741738 (+782424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.232993 (+491255us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54000c08d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:39.241250 (+  8257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298451328}
I20260709 12:14:39.261360 16837 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.262024 16837 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:39.264513 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 165116us
I20260709 12:14:39.265260 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:39.266623 16334 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:39.267143 16334 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:39.268103 16504 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:39.267969 16334 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:39.091619 16870 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:39.235911 16749 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:39.303660 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2684) took 1106 ms. Trace:
I20260709 12:14:39.304049 11175 rpcz_store.cc:276] 0709 12:14:38.197768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.231087 (+ 33319us) service_pool.cc:224] Handling call
0709 12:14:39.303426 (+1072339us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b54002b7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:39.303600 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57421056}
I20260709 12:14:39.305912 11175 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:39.307314 11175 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:14:39.315270 16617 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.000s
I20260709 12:14:39.316012 16617 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15.
I20260709 12:14:39.317129 16617 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:39.283306 16870 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.296048 16749 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.302382 16893 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:39.303463 16627 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.000s
W20260709 12:14:39.368712 11256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.006s	sys 0.034s
W20260709 12:14:39.373888 11256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.007s	sys 0.035s
I20260709 12:14:39.317965 16617 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:39.368018 16571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.034s	sys 0.060s
I20260709 12:14:39.321843 16722 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:39.346377 16876 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.810s	user 0.008s	sys 0.003s
I20260709 12:14:39.378187 11329 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 13: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:39.332577 16870 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:39.356822 16629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.029s	sys 0.070s
W20260709 12:14:39.371690 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 481944us
W20260709 12:14:39.378233 16234 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.001s
W20260709 12:14:39.378634 16741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.085s	sys 0.137s
I20260709 12:14:39.380919 16617 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.330744 16749 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:39.426563 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.017s
W20260709 12:14:39.446384 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.005s	sys 0.020s
W20260709 12:14:39.446818 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 556190us
I20260709 12:14:39.448096 11901 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:39.378547 16837 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:39.426339 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 376339us
I20260709 12:14:39.361824 16609 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:14:39.464778 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:39.403182 16752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.008s	sys 0.037s
W20260709 12:14:39.396472 16629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.030s	sys 0.070s
W20260709 12:14:39.449201 16741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.093s	sys 0.150s
W20260709 12:14:39.468401 16640 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.721s	user 0.009s	sys 0.001s
W20260709 12:14:39.386044 16571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.034s	sys 0.060s
I20260709 12:14:39.471624 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:39.482826 11326 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.000s
W20260709 12:14:39.464143 16632 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.000s
I20260709 12:14:39.410418 16234 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:14:39.409818 16903 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:14:39.498793 11326 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:39.499748 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:39.485879 16752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.008s	sys 0.038s
I20260709 12:14:39.502650 11176 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:14:39.472074 16780 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.016s
I20260709 12:14:39.518121 11179 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:39.524636 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:39.499554 16640 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:39.528707 16912 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:39.501952 16632 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.532578 16632 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:39.534685 16829 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.535802 16829 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.535595 16875 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:39.536664 16875 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:39.537653 16741 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:14:39.538074 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 656668us
I20260709 12:14:39.538467 16741 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:39.538923 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:39.539268 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 894) took 1392 ms. Trace:
I20260709 12:14:39.539674 11181 rpcz_store.cc:276] 0709 12:14:38.147156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.147321 (+   165us) service_pool.cc:224] Handling call
0709 12:14:38.147743 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:38.670739 (+522996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.539018 (+868279us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002e4ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:39.539209 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747506048}
I20260709 12:14:39.540318 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:39.539716 16741 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:39.546825 16854 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.498387 16908 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:14:39.548025 16841 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.500504 16840 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:39.496416 16837 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:39.525905 16432 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:39.547459 16658 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:39.552632 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2204) took 1475 ms. Trace:
W20260709 12:14:39.517078 16780 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.005s	sys 0.016s
I20260709 12:14:39.557683 11180 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:39.549379 16840 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:39.573843 11898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.002s	sys 0.042s
W20260709 12:14:39.495326 16862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.010s	sys 0.009s
W20260709 12:14:39.574385 11898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.002s	sys 0.042s
I20260709 12:14:39.549419 16854 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:39.553220 16856 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:14:39.554572 16772 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.004s
W20260709 12:14:39.565083 16657 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.755s	user 0.007s	sys 0.004s
I20260709 12:14:39.554059 16730 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:39.564917 16658 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:39.565892 16752 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:39.564812 16855 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.613921 16855 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:39.614624 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.616735 16855 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:39.617151 11174 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:39.562055 16234 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:39.564585 16432 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:39.618606 16432 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:39.594014 16730 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:39.619416 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2215) took 1091 ms. Trace:
I20260709 12:14:39.619736 16730 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:39.620689 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.621186 16730 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.569527 16908 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:39.575598 16840 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:39.576736 16862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.011s	sys 0.009s
I20260709 12:14:39.606354 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 897) took 1456 ms. Trace:
I20260709 12:14:39.641458 11180 rpcz_store.cc:276] 0709 12:14:38.149748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.257585 (+107837us) service_pool.cc:224] Handling call
0709 12:14:39.606146 (+1348561us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b5400364554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:39.606283 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2177949568}
W20260709 12:14:39.608351 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.610755 16893 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:39.617918 16234 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:39.620002 11325 rpcz_store.cc:276] 0709 12:14:38.527524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.803773 (+276249us) service_pool.cc:224] Handling call
0709 12:14:39.619192 (+815419us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b54000c71d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:39.619339 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12908288}
I20260709 12:14:39.631217 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:39.634379 11257 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:39.635208 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:39.649818 11252 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:39.565254 11328 rpcz_store.cc:276] 0709 12:14:38.076830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.458291 (+381461us) service_pool.cc:224] Handling call
0709 12:14:38.458601 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:38.558608 (+100007us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:38.633424 (+ 74816us) log.cc:844] Serialized 14 byte log entry
0709 12:14:38.846674 (+213250us) log.cc:844] Serialized 14 byte log entry
0709 12:14:38.875975 (+ 29301us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:38.884336 (+  8361us) log.cc:844] Serialized 26 byte log entry
0709 12:14:38.884553 (+   217us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:38.884858 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 12:14:39.549838 (+664980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.552237 (+  2399us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b54003318d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:39.552360 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:39.552405 (+    45us) raft_consensus.cc:1685] finished
0709 12:14:39.552418 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.552565 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":710635,"prepare.queue_time_us":100898,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":126,"raft.queue_time_us":692293,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":23111,"spinlock_wait_cycles":239680640,"thread_start_us":209418,"threads_started":1,"wal-append.queue_time_us":224731}
W20260709 12:14:39.642803 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.642861 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:39.655925 11328 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:39.661559 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 445816us
W20260709 12:14:39.643983 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 376030us
I20260709 12:14:39.664328 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.664813 11329 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:39.602451 16658 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:39.671161 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:39.672542 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:39.673154 11180 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:39.601977 16752 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:39.669034 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.675021 11248 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:14:39.580273 16769 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.012s	sys 0.009s
W20260709 12:14:39.674605 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.678210 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:39.638264 16908 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:39.680831 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:39.705421 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.683718 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:39.627842 16772 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:39.708894 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47492.5us
I20260709 12:14:39.723766 11252 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:39.707195 11329 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.595878 16657 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:39.708544 16632 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:39.735682 16300 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.739872 16285 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:39.740028 16300 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.740469 16285 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.742543 16908 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:39.725368 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.008s	sys 0.014s
I20260709 12:14:39.726529 11249 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:39.743777 16908 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.735586 16772 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:39.744323 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2383) took 1327 ms. Trace:
I20260709 12:14:39.736210 11251 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:39.743765 16878 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:14:39.745371 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.009s	sys 0.014s
I20260709 12:14:39.745500 16878 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:39.746090 16878 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:39.744755 11253 rpcz_store.cc:276] 0709 12:14:38.417195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.656490 (+239295us) service_pool.cc:224] Handling call
0709 12:14:38.656789 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:38.821836 (+165047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:38.821867 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:38.821908 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:38.821941 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.744085 (+922144us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54001cf2d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:39.744132 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.744261 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379132160}
I20260709 12:14:39.758684 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.753434 11257 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:39.760427 11180 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:39.762776 11330 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
W20260709 12:14:39.753955 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 306269us
I20260709 12:14:39.763236 11330 pending_rounds.cc:85] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:39.763618 11330 pending_rounds.cc:107] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:14:39.763867 11330 raft_consensus.cc:2889] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:39.765770 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54372us
I20260709 12:14:39.766616 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:39.707016 16769 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.014s	sys 0.009s
I20260709 12:14:39.772642 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:39.774178 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:39.680183 16752 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:39.673933 16919 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:14:39.807649 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 43871.6us
I20260709 12:14:39.810295 11329 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:39.825291 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2683) took 1658 ms. Trace:
I20260709 12:14:39.826094 11177 rpcz_store.cc:276] 0709 12:14:38.166981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.818922 (+651941us) service_pool.cc:224] Handling call
0709 12:14:38.819241 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.625943 (+806702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:39.625975 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:39.626018 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:39.626051 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.824917 (+198866us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54002e45d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:39.824987 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.825190 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854770944}
W20260709 12:14:39.851212 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 84682.4us
I20260709 12:14:39.853295 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:39.820612 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 93414.9us
I20260709 12:14:39.869576 16285 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:39.873118 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 99064.3us
I20260709 12:14:39.873817 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:39.874280 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64605.4us
I20260709 12:14:39.875051 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:39.875919 11180 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:39.873972 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:39.879061 11329 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:39.882871 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.884631 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:39.885571 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:39.887864 11324 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:39.895309 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:39.895996 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.869746 16057 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:39.798264 16769 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:39.911036 16769 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.669826 16749 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.899533 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:39.899703 11326 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:14:39.869602 16571 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:14:39.857577 16632 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:39.950017 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:39.951514 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77801us
I20260709 12:14:39.770502 16870 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:39.810833 16234 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:39.716866 16880 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.141s
I20260709 12:14:39.905725 16854 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.954226 11326 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:14:39.857204 16851 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.013270 16640 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:39.797272 16772 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 12:14:39.922967 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.032258 11257 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
W20260709 12:14:40.003676 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.872282 16657 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:39.898747 16285 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:39.977451 16722 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:14:39.914708 16894 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.109s	user 0.007s	sys 0.005s
I20260709 12:14:39.900596 16837 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:40.009027 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.009600 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:40.139811 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:40.083932 16875 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.004s
I20260709 12:14:40.140591 11326 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:40.137089 16640 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:39.977350 16830 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:40.010644 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2203) took 1934 ms. Trace:
W20260709 12:14:40.015678 16917 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:14:40.016630 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.158563 11177 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:40.026893 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:39.956894 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.015s	sys 0.009s
I20260709 12:14:40.028362 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.161047 11926 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:40.036635 16234 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:40.029836 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:39.971277 16808 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:14:40.038482 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:40.136402 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.015s	sys 0.028s
W20260709 12:14:40.166234 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.016s	sys 0.028s
I20260709 12:14:39.871703 16658 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.037747 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.167269 11935 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:40.046177 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.059621 16870 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.033334 16851 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:40.066864 16884 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.245s	user 0.003s	sys 0.008s
I20260709 12:14:40.004010 16854 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:40.068266 16648 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.207s	user 0.007s	sys 0.006s
I20260709 12:14:40.219010 16648 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.030253 16915 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:40.070600 16749 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:40.071880 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.072961 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.222028 16915 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:40.085376 11322 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.039s
W20260709 12:14:40.147984 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.223210 16658 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:40.223233 11322 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.000s	sys 0.039s
W20260709 12:14:40.224623 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.224853 16915 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:40.143942 11182 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
I20260709 12:14:40.225366 16851 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.225966 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 907) took 1521 ms. Trace:
I20260709 12:14:40.226747 16884 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:40.227275 11330 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:14:40.227789 11330 pending_rounds.cc:85] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 11
I20260709 12:14:40.228040 11330 pending_rounds.cc:107] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:14:40.228317 11330 raft_consensus.cc:2889] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:40.229645 11254 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:40.226430 11182 rpcz_store.cc:276] 0709 12:14:38.704216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.143348 (+1439132us) service_pool.cc:224] Handling call
0709 12:14:40.143726 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:40.225605 (+ 81879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.225890 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:40.234426 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.151245 11327 rpcz_store.cc:276] 0709 12:14:38.076543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.363886 (+287343us) service_pool.cc:224] Handling call
0709 12:14:38.364348 (+   462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:38.364836 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:38.508984 (+144148us) log.cc:844] Serialized 14 byte log entry
0709 12:14:38.850846 (+341862us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:38.878406 (+ 27560us) log.cc:844] Serialized 52 byte log entry
0709 12:14:38.878613 (+   207us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:38.879251 (+   638us) log.cc:844] Serialized 14 byte log entry
0709 12:14:39.530003 (+650752us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b18003f88c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:14:39.971665 (+441662us) log.cc:844] Serialized 14 byte log entry
0709 12:14:39.974389 (+  2724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.010286 (+ 35897us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:40.010361 (+    75us) raft_consensus.cc:1685] finished
0709 12:14:40.010381 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:40.010566 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2816,"prepare.queue_time_us":142136,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":101,"raft.queue_time_us":663907,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":27318,"spinlock_wait_cycles":4456704,"thread_start_us":904495,"threads_started":3,"wal-append.queue_time_us":731566}
I20260709 12:14:40.238211 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:40.238960 11327 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:40.154561 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.029008 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:40.138223 16853 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.013s	sys 0.016s
W20260709 12:14:40.258447 11323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.018s	sys 0.014s
W20260709 12:14:40.258906 11323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.018s	sys 0.014s
I20260709 12:14:40.259673 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2199) took 2298 ms. Trace:
I20260709 12:14:40.162385 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.260028 11323 rpcz_store.cc:276] 0709 12:14:37.961683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.155479 (+193796us) service_pool.cc:224] Handling call
0709 12:14:38.155762 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:38.155961 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:38.498197 (+342236us) log.cc:844] Serialized 14 byte log entry
0709 12:14:40.259210 (+1761013us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:40.259275 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:40.259325 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.259417 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:40.259600 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":651447,"prepare.queue_time_us":348196,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":659146,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":132,"thread_start_us":1420163,"threads_started":3,"wal-append.queue_time_us":776272}
I20260709 12:14:40.163535 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.069589 16878 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.271854 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.272806 11934 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
W20260709 12:14:40.165217 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.015s	sys 0.010s
I20260709 12:14:40.176752 11180 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:40.180011 11326 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:40.279062 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 904) took 1588 ms. Trace:
I20260709 12:14:40.279929 11326 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:40.141878 16722 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:40.183013 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 642683us
I20260709 12:14:40.184880 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.185331 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2362) took 2180 ms. Trace:
I20260709 12:14:40.285928 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:40.286319 11176 rpcz_store.cc:276] 0709 12:14:37.988990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.012071 (+ 23081us) service_pool.cc:224] Handling call
0709 12:14:38.022993 (+ 10922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:39.516842 (+1493849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.169070 (+652228us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400363654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:40.169277 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3399481088}
I20260709 12:14:40.206161 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.289373 11931 raft_consensus.cc:2410] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:14:40.168768 16840 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 12:14:40.218158 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:40.221554 16617 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.000s
W20260709 12:14:40.254647 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.257129 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.282706 11326 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:40.291728 11328 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.022s	sys 0.022s
W20260709 12:14:40.315593 11328 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.023s	sys 0.022s
I20260709 12:14:40.296169 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:40.300558 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.328042 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.329101 11931 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:40.331591 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.334064 11930 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:40.162473 16285 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:40.283661 16911 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.536s	user 0.004s	sys 0.000s
I20260709 12:14:40.164878 16830 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:40.257898 16853 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.013s	sys 0.016s
I20260709 12:14:40.323254 11929 raft_consensus.cc:2376] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16. Re-sending same reply.
W20260709 12:14:40.385761 11331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.020s	sys 0.042s
W20260709 12:14:40.386247 11331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.020s	sys 0.042s
W20260709 12:14:40.289253 16841 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.741s	user 0.010s	sys 0.001s
W20260709 12:14:40.388453 11325 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.018s	sys 0.026s
W20260709 12:14:40.388913 11325 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.018s	sys 0.026s
I20260709 12:14:40.313424 16658 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:40.392303 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:40.202881 16887 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.020s	sys 0.029s
W20260709 12:14:40.412317 16887 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.020s	sys 0.029s
W20260709 12:14:40.157687 16839 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.021s	sys 0.025s
W20260709 12:14:40.413710 16839 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.022s	sys 0.025s
I20260709 12:14:40.333828 16829 raft_consensus.cc:2764] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:14:40.402709 11928 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:40.406749 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.288813 16609 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:40.418766 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.314021 16772 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:40.291451 16752 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.427388 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.334412 16840 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:40.430691 16931 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260709 12:14:40.409246 16841 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:40.429347 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.401042 11935 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:40.333467 16617 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.313007 16851 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:14:40.333781 16876 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:14:40.467429 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:40.431710 16840 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.470674 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:40.474546 16915 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:40.476231 16915 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.475850 16722 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.476467 16854 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:40.432617 16658 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:40.432575 16609 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:40.476783 16915 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:40.468945 16829 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:14:40.478382 16644 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:40.466197 16772 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:40.487103 16741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.011s	sys 0.020s
I20260709 12:14:40.512138 11930 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:40.481189 16895 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:14:40.479710 16894 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:14:40.494418 16644 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.494418 16908 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:14:40.583369 11145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.004s	sys 0.007s
W20260709 12:14:40.583851 11174 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.010s	sys 0.039s
W20260709 12:14:40.584584 11174 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.010s	sys 0.039s
W20260709 12:14:40.584717 11175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.010s	sys 0.023s
W20260709 12:14:40.585237 11175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.010s	sys 0.023s
I20260709 12:14:40.150453 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2367) took 1923 ms. Trace:
I20260709 12:14:40.586151 11179 rpcz_store.cc:276] 0709 12:14:38.227532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.877970 (+650438us) service_pool.cc:224] Handling call
0709 12:14:38.903704 (+ 25734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.518580 (+614876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.150115 (+631535us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54002323d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:40.150373 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412606720}
I20260709 12:14:40.587131 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2695) took 2169 ms. Trace:
I20260709 12:14:40.587694 11175 rpcz_store.cc:276] 0709 12:14:38.417477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.307952 (+890475us) service_pool.cc:224] Handling call
0709 12:14:39.308183 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.308360 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:39.724797 (+416437us) log.cc:844] Serialized 14 byte log entry
0709 12:14:40.586162 (+861365us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:40.586287 (+   125us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:40.586421 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.586590 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:40.586933 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":710951,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"thread_start_us":1276617,"threads_started":2,"wal-append.queue_time_us":761842}
W20260709 12:14:40.589675 11181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.009s	sys 0.022s
W20260709 12:14:40.590464 11181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.010s	sys 0.022s
I20260709 12:14:40.279572 11180 rpcz_store.cc:276] 0709 12:14:38.690764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.146606 (+1455842us) service_pool.cc:224] Handling call
0709 12:14:40.176507 (+ 29901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:40.278685 (+102178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.278973 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:40.591976 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2696) took 2131 ms. Trace:
W20260709 12:14:40.594156 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.001s	sys 0.010s
W20260709 12:14:40.594732 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.001s	sys 0.010s
W20260709 12:14:40.584362 11145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.005s	sys 0.007s
I20260709 12:14:40.600193 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:40.600773 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:40.597115 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 743834us
I20260709 12:14:40.592410 11181 rpcz_store.cc:276] 0709 12:14:38.460940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.540961 (+1080021us) service_pool.cc:224] Handling call
0709 12:14:39.541270 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.541488 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:39.915235 (+373747us) log.cc:844] Serialized 14 byte log entry
0709 12:14:40.591247 (+676012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:40.591368 (+   121us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:40.591541 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.591660 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:40.591902 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1456,"prepare.queue_time_us":619322,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"thread_start_us":1024020,"threads_started":2,"wal-append.queue_time_us":584221}
W20260709 12:14:40.601440 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.603399 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.615075 11327 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:40.615898 11252 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:40.616732 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:40.603411 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:40.618443 11255 raft_consensus.cc:2368] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15.
I20260709 12:14:40.619158 11327 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:40.608395 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.603452 11248 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:40.620110 11256 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:40.610401 11257 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:40.604231 11250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.004s	sys 0.002s
W20260709 12:14:40.621963 11250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.005s	sys 0.002s
I20260709 12:14:40.622385 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:40.622465 11329 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
W20260709 12:14:40.614305 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:40.614907 11324 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.020s	sys 0.010s
W20260709 12:14:40.625530 11324 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.021s	sys 0.011s
I20260709 12:14:40.625393 11175 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:14:40.626157 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.618448 11326 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:40.627444 11257 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:40.606869 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.628268 11253 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:40.602869 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:40.631435 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:40.621341 11327 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:14:40.617528 11252 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:40.634375 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:40.637063 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:40.639147 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.640996 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.641360 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.642112 11253 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:40.642886 11252 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
W20260709 12:14:40.642946 11898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.024s	sys 0.042s
W20260709 12:14:40.643675 11898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.025s	sys 0.042s
I20260709 12:14:40.644372 11181 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:40.639554 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.645830 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.646742 11327 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:40.647876 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 157731us
I20260709 12:14:40.648800 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.650027 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.651141 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
W20260709 12:14:40.652118 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.653070 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.655592 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:40.656906 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.657689 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:40.658152 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.659107 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:40.659437 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:40.659667 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.660315 11176 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:40.660889 11179 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:40.662359 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.663488 11179 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:40.663707 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.664759 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.662420 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:40.666142 11179 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:40.666857 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.665347 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.667917 11327 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:40.668443 11177 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:40.672008 11327 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:40.680759 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2339) took 2210 ms (client timeout 2200 ms). Trace:
W20260709 12:14:40.692618 11254 rpcz_store.cc:269] 0709 12:14:38.469771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.485210 (+ 15439us) service_pool.cc:224] Handling call
0709 12:14:40.680129 (+2194919us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54002a58d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:40.680495 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3829532032}
I20260709 12:14:40.694689 16657 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:40.524628 16644 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:40.694506 16658 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.695654 16657 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:40.683604 16571 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:40.682154 11935 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: [11.9-15.11]
I20260709 12:14:40.697682 16657 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:40.698096 16571 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.685489 16285 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.700214 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:40.683894 16057 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:40.690066 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:40.685981 16617 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:40.691977 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:40.694818 16640 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.703660 16640 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:40.691061 16432 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.072s	user 0.012s	sys 0.000s
I20260709 12:14:40.705982 16432 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.702517 16617 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:40.707432 11173 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:40.708185 16617 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:40.701336 11252 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:40.708730 16432 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.708760 16640 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.711593 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 903) took 2033 ms. Trace:
I20260709 12:14:40.712018 11173 rpcz_store.cc:276] 0709 12:14:38.678708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.626520 (+947812us) service_pool.cc:224] Handling call
0709 12:14:39.626812 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:40.707956 (+1081144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.709512 (+  1556us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400256cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:40.711527 (+  2015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2484211072}
I20260709 12:14:40.732409 16646 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:14:40.744344 16300 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.004s	user 0.018s	sys 0.001s
I20260709 12:14:40.745509 11257 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:40.747493 11179 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:40.747337 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.748693 16627 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.749554 16627 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.751621 11178 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:40.755475 16646 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:14:40.759238 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:40.511685 16741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.020s
I20260709 12:14:40.791301 11256 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:40.818248 16808 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:40.800210 16609 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:40.819468 16808 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:40.819902 16609 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:40.820134 16808 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:40.801750 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:40.799973 16648 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.823362 16808 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.760141 16627 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.811533 11180 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:40.759284 16454 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.812168 16769 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
W20260709 12:14:40.813439 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.806546 16571 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.810564 16752 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:40.831830 16752 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:40.832227 16752 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:40.807855 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:40.833384 16752 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.809561 16300 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:40.835840 16752 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:40.835471 16300 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.834434 11330 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:40.838176 16300 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:40.833393 16730 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:40.827667 16454 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:40.839424 16730 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:40.841531 16454 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:40.844650 16730 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:40.855170 16300 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.855929 16300 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:40.858384 16300 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:40.665385 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.880323 16886 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:40.880930 16886 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:40.882843 16887 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:40.883491 16887 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:40.884819 16887 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:40.817512 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.663873 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:40.886320 16894 consensus_queue.cc:1243] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:14:40.886042 16888 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:14:40.912864 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.759461 16644 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:40.917827 11176 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:14:40.939026 11177 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:40.936394 16888 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:40.993713 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 328734us
W20260709 12:14:40.997046 11323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.006s
I20260709 12:14:40.942299 16617 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
W20260709 12:14:41.014933 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 348272us
W20260709 12:14:41.016214 11323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.007s	sys 0.007s
I20260709 12:14:41.033583 16888 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.035261 16888 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.036947 16851 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.037912 11330 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:41.037887 16851 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:41.040428 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.041769 16935 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.042694 16935 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.044565 16752 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:14:41.045094 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.000s	sys 0.005s
W20260709 12:14:41.045603 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.000s	sys 0.005s
I20260709 12:14:41.034395 16644 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:41.048930 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.025033 16057 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:41.046272 16853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.013s	sys 0.004s
I20260709 12:14:41.098529 16876 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:41.130298 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.078889 16454 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.097259 16730 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.134665 11327 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:41.125439 16853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.014s	sys 0.005s
W20260709 12:14:41.155550 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.154361 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:41.099388 16876 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.157136 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2406) took 1505 ms. Trace:
W20260709 12:14:41.169756 11325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.010s
I20260709 12:14:41.201211 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1018) took 1299 ms. Trace:
I20260709 12:14:41.210137 16876 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:41.211570 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2210) took 2842 ms. Trace:
W20260709 12:14:41.214316 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.147927 16851 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:41.197192 11328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.000s	sys 0.024s
W20260709 12:14:41.215636 11325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.003s	sys 0.011s
W20260709 12:14:41.206712 11322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.006s	sys 0.008s
I20260709 12:14:41.224239 11249 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:41.225060 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:41.226339 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 199116us
I20260709 12:14:41.228813 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:41.230189 16722 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:41.230739 16722 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:41.231240 16722 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:41.214911 16886 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.237005 16886 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.206076 16454 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.238221 16057 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.240170 16454 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:41.240769 11322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.006s	sys 0.008s
W20260709 12:14:41.241791 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 107798us
I20260709 12:14:41.242000 16875 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:41.243886 16644 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:41.244257 16895 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:14:41.213980 16934 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.247033 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:41.247426 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:41.247817 11330 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:41.249953 16571 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:41.249752 16730 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.250425 16571 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.250896 16571 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:41.251529 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.252243 16730 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.174584 16808 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.246703 16934 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.245719 16875 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.212059 11331 rpcz_store.cc:276] 0709 12:14:38.368828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.270846 (+902018us) service_pool.cc:224] Handling call
0709 12:14:39.286019 (+ 15173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:39.563222 (+277203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:39.563257 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:39.563737 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:14:40.007508 (+443771us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:14:40.386720 (+379212us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:41.087735 (+701015us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.210847 (+123112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.210998 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:41.211087 (+    89us) raft_consensus.cc:1685] finished
0709 12:14:41.211185 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.211384 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276805,"raft.queue_time_us":280981,"raft.run_cpu_time_us":10679,"raft.run_wall_time_us":454126,"spinlock_wait_cycles":5022976,"thread_start_us":1218719,"threads_started":2,"wal-append.queue_time_us":954484}
I20260709 12:14:41.205506 11324 rpcz_store.cc:276] 0709 12:14:39.888953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.889070 (+   117us) service_pool.cc:224] Handling call
0709 12:14:39.889342 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.889563 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:40.626237 (+736674us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.053152 (+426915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:41.086000 (+ 32848us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:41.099975 (+ 13975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.134461 (+ 34486us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.188642 (+ 54181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40007,"prepare.queue_time_us":1307344,"raft.queue_time_us":40728,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":88350,"thread_start_us":1084022,"threads_started":2}
W20260709 12:14:41.252260 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.35271e+06us
W20260709 12:14:41.275336 11328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.001s	sys 0.024s
I20260709 12:14:41.215667 16957 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:41.295753 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.300316 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:41.294320 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.109s	user 0.007s	sys 0.019s
I20260709 12:14:41.198011 11250 rpcz_store.cc:276] 0709 12:14:39.629735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.629863 (+   128us) service_pool.cc:224] Handling call
0709 12:14:39.659219 (+ 29356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.659447 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:39.676901 (+ 17454us) log.cc:844] Serialized 14 byte log entry
0709 12:14:39.914237 (+237336us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b18002938c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:14:41.012403 (+1098166us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.047630 (+ 35227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:41.052052 (+  4422us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:41.058053 (+  6001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.090162 (+ 32109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.135405 (+ 45243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2972,"prepare.queue_time_us":17277,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":13496,"raft.queue_time_us":74788,"raft.run_cpu_time_us":430,"raft.run_wall_time_us":17176,"spinlock_wait_cycles":5980544,"thread_start_us":1091782,"threads_started":2,"wal-append.queue_time_us":862707}
I20260709 12:14:41.245658 16870 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:41.315549 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.364s	user 0.007s	sys 0.019s
I20260709 12:14:41.278404 16894 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:41.319242 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 902) took 2641 ms. Trace:
I20260709 12:14:41.274912 16644 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:14:41.331784 16923 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.332182 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2685) took 1650 ms. Trace:
I20260709 12:14:41.332834 16964 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.332669 16923 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.333516 16964 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.335423 16923 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.335566 16830 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:41.336055 16939 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 12:14:41.337231 16953 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:14:41.337673 16722 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.333128 16935 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:41.340699 16722 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.345456 16957 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:41.348100 16926 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.443s	user 0.014s	sys 0.032s
I20260709 12:14:41.350440 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:41.352288 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:41.354182 16958 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:41.354728 16958 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:41.355201 16958 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:41.329478 16780 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:41.358364 16884 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.359028 16937 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.359774 16937 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.360488 16948 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:41.361151 16922 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:41.358536 16780 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.324395 16841 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.306900 16454 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.335848 16964 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:41.372506 16780 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.362340 16922 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.379439 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:41.380450 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2228) took 1727 ms. Trace:
I20260709 12:14:41.381131 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:41.383347 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:41.372534 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:41.391602 11330 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:41.387997 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:41.379498 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:41.398890 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:41.323463 11174 rpcz_store.cc:276] 0709 12:14:38.677801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.623549 (+945748us) service_pool.cc:224] Handling call
0709 12:14:39.623978 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.624231 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:40.585493 (+961262us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.223784 (+638291us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.275152 (+ 51368us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:41.284216 (+  9064us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:41.308016 (+ 23800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.314252 (+  6236us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.319175 (+  4923us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46713,"prepare.queue_time_us":558913,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":53935,"raft.run_cpu_time_us":557,"raft.run_wall_time_us":79145,"thread_start_us":1488256,"threads_started":3}
W20260709 12:14:41.399955 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47839.2us
I20260709 12:14:41.281128 16808 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.323138 16894 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.419801 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:41.417836 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.060s	sys 0.071s
I20260709 12:14:41.422957 11173 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:41.415879 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:41.430003 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.061s	sys 0.071s
W20260709 12:14:41.430677 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 938664us
W20260709 12:14:41.442154 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.391685 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2688) took 1437 ms. Trace:
I20260709 12:14:41.440271 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.332687 11328 rpcz_store.cc:276] 0709 12:14:39.682214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.685395 (+  3181us) service_pool.cc:224] Handling call
0709 12:14:39.685854 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.686157 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:40.316182 (+630025us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.329533 (+1013351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:41.331771 (+  2238us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:41.331848 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.331947 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.332124 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100059,"prepare.queue_time_us":1437692,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":27408,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":36814,"thread_start_us":1599582,"threads_started":2,"wal-append.queue_time_us":859526}
I20260709 12:14:41.384824 16658 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:41.481240 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:41.488673 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.389474 16957 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.460886 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:41.430087 16454 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.504127 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:41.483369 11324 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:41.433507 11331 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:14:41.418298 16730 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:41.503247 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.407554 11325 rpcz_store.cc:276] 0709 12:14:39.652918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.653068 (+   150us) service_pool.cc:224] Handling call
0709 12:14:39.653487 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.653824 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:40.407234 (+753410us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.361858 (+954624us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.380042 (+ 18184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:41.380108 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:41.380166 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.380261 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.380397 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":143030,"prepare.queue_time_us":1438839,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":104,"raft.queue_time_us":55234,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":10905,"thread_start_us":1546147,"threads_started":2}
I20260709 12:14:41.517138 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:41.517676 16879 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.511137 11323 rpcz_store.cc:276] 0709 12:14:39.954669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.436318 (+481649us) service_pool.cc:224] Handling call
0709 12:14:40.436720 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:41.031954 (+595234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:41.032007 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:41.032781 (+   774us) log.cc:844] Serialized 26 byte log entry
0709 12:14:41.148628 (+115847us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:41.148667 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.194860 (+ 46193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:41.365721 (+170861us) raft_consensus.cc:1685] finished
0709 12:14:41.365747 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.391565 (+ 25818us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32524,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4731,"thread_start_us":705618,"threads_started":2,"wal-append.queue_time_us":275166}
I20260709 12:14:41.369530 16875 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.531042 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:41.533398 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:41.534199 11174 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:41.534224 11329 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:41.496899 16644 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:14:41.541499 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:41.541357 16730 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.542295 11179 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:41.543902 16730 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.544358 16454 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.544600 16879 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.545858 16829 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:41.546324 16780 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:41.547115 16879 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.550534 16887 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:41.553663 11175 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.554852 16934 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:41.437703 16870 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.564769 11931 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:14:41.568248 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 135571us
W20260709 12:14:41.568143 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63134.1us
I20260709 12:14:41.569010 16875 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.569295 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:41.569729 16918 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:41.570432 16918 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.571516 16918 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.574864 16769 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:41.575366 16922 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.576403 16769 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:14:41.565469 11931 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.576817 16769 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:41.577486 16632 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.598s	user 0.016s	sys 0.008s
I20260709 12:14:41.578011 16632 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:41.577819 11931 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.578199 16769 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.578984 11931 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:41.580092 16769 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:41.581602 16957 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:41.582106 16957 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.582793 16957 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:41.580093 11251 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:41.585230 11926 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:41.586202 16929 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:41.586717 16929 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.588083 16929 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:41.588946 16915 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.091s	user 0.011s	sys 0.000s
I20260709 12:14:41.590801 16808 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:41.591379 11222 connection.cc:256] RPC call timeout handler was delayed by -1.6619118539999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 12:14:41.591814 11222 connection.cc:256] RPC call timeout handler was delayed by -1.6529118539999672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 12:14:41.592114 11222 connection.cc:256] RPC call timeout handler was delayed by -1.6025118540001131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 12:14:41.592551 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5989118540001073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
I20260709 12:14:41.593217 11175 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:41.582021 16870 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:41.539760 16978 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.006s
I20260709 12:14:41.602355 11180 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:41.576706 16922 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:41.615918 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 75874.3us
W20260709 12:14:41.597597 16888 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.530s	user 0.016s	sys 0.000s
W20260709 12:14:41.619876 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49575.6us
W20260709 12:14:41.620296 16977 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:14:41.620749 16959 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:14:41.620183 16646 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:41.620630 16982 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:41.600208 16808 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:41.622375 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 239153us
W20260709 12:14:41.627529 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.589504 16915 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.509057 16841 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.565222 16658 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.645685 11930 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:41.641177 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:41.634626 16937 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:41.657757 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 64634.5us
W20260709 12:14:41.659395 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5242118539999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 12:14:41.659866 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5107118539999647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 12:14:41.656272 16646 consensus_queue.cc:1243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:14:41.677374 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:41.677477 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:41.680143 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 97576.1us
W20260709 12:14:41.683974 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:41.684518 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:41.686633 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.688685 16772 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.688885 16984 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:14:41.690024 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:41.690583 16841 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.691344 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:41.692004 11932 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:41.690851 11928 raft_consensus.cc:2368] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16.
I20260709 12:14:41.687752 16925 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:41.690681 16908 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:14:41.695035 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:41.695297 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:41.698016 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:41.698443 16948 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:41.691380 16878 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.689544 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.706147 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2674) took 2979 ms. Trace:
I20260709 12:14:41.706633 11322 rpcz_store.cc:276] 0709 12:14:38.727112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.451540 (+724428us) service_pool.cc:224] Handling call
0709 12:14:39.496040 (+ 44500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:39.567499 (+ 71459us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:14:40.224114 (+656615us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.308272 (+1084158us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:41.375063 (+ 66791us) log.cc:844] Serialized 26 byte log entry
0709 12:14:41.375237 (+   174us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:41.375279 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.375382 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:41.705857 (+330475us) raft_consensus.cc:1685] finished
0709 12:14:41.705878 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.706063 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":312282,"prepare.queue_time_us":1444372,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":1483,"raft.queue_time_us":106683,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":29267,"thread_start_us":1658772,"threads_started":2,"wal-append.queue_time_us":1117628}
I20260709 12:14:41.708726 16958 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.686506 16808 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.710682 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.711177 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.685705 16879 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:41.712424 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.715713 16646 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:41.717767 16925 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.718006 11932 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:41.687484 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.648136 16894 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.725420 11928 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:41.732077 11931 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:14:41.735338 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:41.727730 16876 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:41.737964 11249 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:41.739326 11929 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:14:41.739470 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3347473550002178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260709 12:14:41.742842 11222 connection.cc:256] RPC call timeout handler was delayed by -1.2231473550000374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 12:14:41.743314 16876 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:41.743826 16876 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:41.744549 16772 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.744020 16925 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.745105 16772 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.747126 11256 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:14:41.747439 16772 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.750231 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:41.753032 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.753854 11252 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:41.754061 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.754918 11253 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:41.756390 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:41.743438 16924 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:14:41.757721 11254 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:41.758206 16808 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.760062 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:41.763324 16880 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:41.765903 16837 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.766553 16837 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:41.768672 11222 connection.cc:256] RPC call timeout handler was delayed by -1.0062723850001021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 12:14:41.768976 16837 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.734398 16878 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.771068 16878 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:41.774662 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.770542 16975 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.785985 16920 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:41.786465 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:41.786571 16920 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:41.787225 11256 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:14:41.787457 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:41.788471 16934 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:41.788856 16934 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:41.789049 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:41.789309 16934 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:41.784916 11252 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:41.785985 16984 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:14:41.725600 16571 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.784885 16975 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.803550 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 874) took 1158 ms. Trace:
I20260709 12:14:41.803645 16975 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.804133 11930 rpcz_store.cc:276] 0709 12:14:40.644867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.656339 (+ 11472us) service_pool.cc:224] Handling call
0709 12:14:41.803299 (+1146960us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b5400780354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:41.803454 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252523392}
I20260709 12:14:41.804657 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:41.805881 16853 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.806632 16853 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.807585 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:41.808334 16929 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.809471 16975 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:41.738471 16841 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:14:41.809937 16975 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:41.806219 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:41.799919 16571 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.813203 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:41.811038 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:41.768747 16772 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:41.801388 16986 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.179s	user 0.000s	sys 0.003s
I20260709 12:14:41.813900 11253 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:41.818676 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:41.810364 16975 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:41.820207 11252 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:41.823140 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.823624 11255 raft_consensus.cc:1217] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Deduplicated request from leader. Original: 11.11->[13.12-13.12]   Dedup: 13.12->[]
I20260709 12:14:41.823832 11248 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:41.824211 11255 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:14:41.817996 16853 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.826772 11254 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:41.826633 16984 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:14:41.830699 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51908.1us
I20260709 12:14:41.836505 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.837883 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:41.840294 11249 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:41.841921 11250 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
W20260709 12:14:41.831512 16886 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.004s
I20260709 12:14:41.841565 16982 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:14:41.845129 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.845795 16752 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:41.854673 11926 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:41.830276 16984 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:41.858228 16952 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:41.825685 16772 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:41.865103 11328 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:41.855839 16752 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.868702 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:41.875330 11331 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:41.869453 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:41.882529 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 73513us
I20260709 12:14:41.862298 16984 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:41.884775 11256 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:41.885207 11252 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:41.881600 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:41.889357 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:41.889432 16752 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.890425 11329 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 17 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:14:41.886336 11325 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:41.893972 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.895358 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:41.895957 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.898170 11257 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:41.897091 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.877324 16952 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:41.904631 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2695) took 1074 ms. Trace:
I20260709 12:14:41.901754 11328 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
W20260709 12:14:41.904959 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 84862.5us
I20260709 12:14:41.906297 11248 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:41.906041 11326 rpcz_store.cc:276] 0709 12:14:40.830500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.831887 (+  1387us) service_pool.cc:224] Handling call
0709 12:14:40.832335 (+   448us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:40.832839 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:40.832870 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:41.343355 (+510485us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b2c00126d18. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:41.344200 (+   845us) log.cc:844] Serialized 52 byte log entry
0709 12:14:41.496267 (+152067us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:41.496309 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.496387 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:41.800566 (+304179us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:41.904378 (+103812us) raft_consensus.cc:1685] finished
0709 12:14:41.904403 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.904567 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9655,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"spinlock_wait_cycles":26550400,"thread_start_us":151604,"threads_started":1,"wal-append.queue_time_us":277026}
W20260709 12:14:41.910238 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.863726 16957 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.910765 11331 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:14:41.892258 16571 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:41.916941 16658 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:41.916581 16952 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:41.918090 16994 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:14:41.918581 16888 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:41.919072 16888 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:41.919938 16996 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:14:41.920548 16970 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:41.921027 16970 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:41.922796 16888 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.924300 16888 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:41.934490 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:41.935245 11326 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:41.891709 16886 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:14:41.941049 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:41.903591 16878 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.943435 11330 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:41.898569 16908 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:14:41.932870 16958 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:41.955927 16837 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:14:41.962721 16772 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:41.963634 16970 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:41.964468 11253 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
W20260709 12:14:41.947582 16995 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:14:41.948923 16988 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:14:41.954008 16908 consensus_queue.cc:1243] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:14:41.983791 16841 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:41.984794 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.001178 11250 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:41.985581 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:41.987300 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:42.001864 11253 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:42.002300 11934 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
W20260709 12:14:41.987985 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.003572 16920 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.004354 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:42.004709 16920 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:42.005177 11182 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:41.988212 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:42.005991 11930 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:42.006196 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:41.989081 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:41.989938 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:42.007148 11929 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:42.007449 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:42.009168 11182 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:42.010166 11254 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:42.010512 11181 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:14:42.011229 16879 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:42.011761 16879 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:42.011857 16841 consensus_queue.cc:1243] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:14:42.012449 16841 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:42.012844 16841 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:42.013909 16841 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.015321 16841 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:42.016531 16976 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.516s	user 0.002s	sys 0.002s
I20260709 12:14:41.996753 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.019104 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.019869 11250 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:42.020776 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:42.022044 16997 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:14:42.021525 11250 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:42.024919 16979 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.401s	user 0.000s	sys 0.001s
W20260709 12:14:42.026532 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.024999 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.027597 11250 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:42.029036 16908 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:42.031930 16772 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:42.032006 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.033682 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.034245 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 808) took 1436 ms. Trace:
I20260709 12:14:42.034016 16974 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:41.998103 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:41.998965 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:41.999365 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:42.034672 11251 rpcz_store.cc:276] 0709 12:14:40.597518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.601955 (+  4437us) service_pool.cc:224] Handling call
0709 12:14:40.750459 (+148504us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b5800196898. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:14:40.750773 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:41.583461 (+832688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:42.034008 (+450547us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54000f87d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:42.034185 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923116160}
I20260709 12:14:42.035472 11177 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:42.036149 16918 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:42.037135 11180 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:42.034468 11330 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:42.040755 16915 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.036630 16918 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:42.042966 16918 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.044276 16918 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:42.035377 11179 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:41.995656 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:42.012036 11181 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:42.037390 16752 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:42.020962 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.056182 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:42.061390 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 52453.8us
I20260709 12:14:42.035279 16974 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:42.065698 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:42.066723 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70534.6us
I20260709 12:14:42.068245 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.069653 11251 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:42.071020 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:42.070909 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:42.058895 16969 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.435s	user 0.002s	sys 0.000s
I20260709 12:14:42.072144 11250 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:42.076336 11255 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:42.079286 16984 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.079519 16934 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.081362 16948 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:42.082115 16948 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:42.083614 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:42.083371 16948 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.087489 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.090278 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.091727 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:42.062961 16895 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:42.098979 16984 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:42.101068 11331 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:42.107769 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.108846 11253 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:42.110529 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:42.111497 11250 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:42.112596 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.099076 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.111290 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:42.116060 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.073302 16875 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.129616 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.124075 16571 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:42.130450 16571 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.130170 16875 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.130944 16571 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:42.131146 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.132572 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.132972 16875 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.133370 11180 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:42.086053 16617 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:14:42.133786 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.133944 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.134568 16617 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:42.113695 16934 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:42.135208 16617 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:42.135882 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.062881 16658 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.128064 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.137008 16658 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.134657 11175 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:42.128885 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.137985 11173 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:42.139667 16876 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.140421 16571 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.142385 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:42.143018 16996 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
I20260709 12:14:42.129726 16772 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:42.146076 11248 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:42.147101 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.134696 11178 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:42.122188 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.140029 16658 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:42.086091 16894 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:42.150595 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 173268us
I20260709 12:14:42.152650 11251 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:42.104406 16853 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:42.154762 16057 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:42.102231 16974 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:42.151341 16970 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:42.113536 16975 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.174844 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.174656 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.177932 11180 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:42.180846 11179 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.182003 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.163872 16876 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:42.186215 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.189486 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.191357 11331 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
W20260709 12:14:42.192728 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 81556us
I20260709 12:14:42.119309 16957 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:42.203852 11331 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:42.201866 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:42.206823 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 122298us
W20260709 12:14:42.206403 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 132925us
I20260709 12:14:42.210989 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.193449 16875 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:42.199985 16870 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:42.196626 16973 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:14:42.216064 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.219112 16571 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.218890 16646 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:42.201100 16895 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:42.216863 16875 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.228705 16875 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:42.218111 16870 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.234254 16870 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:42.238708 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:42.239514 16617 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.241019 16617 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:42.222386 16908 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.020s
W20260709 12:14:42.245118 11075 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 170) took 18239 ms (client timeout 15000 ms). Trace:
I20260709 12:14:42.255175 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.206962 16772 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:14:42.256937 11176 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:42.257664 11177 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.220707 16973 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.260015 11328 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:42.210150 16952 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:14:42.271100 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.274065 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.274886 11325 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:42.264712 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.272900 17007 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:42.286726 17007 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.287166 17007 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:42.290060 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69233.9us
I20260709 12:14:42.290962 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:42.260972 16895 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:42.223994 16974 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.264132 16876 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.254925 16957 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:14:42.296551 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69544.5us
I20260709 12:14:42.301776 16285 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:42.318166 16285 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:42.318629 16285 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:42.319226 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.320441 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.322363 16880 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:14:42.322783 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.324699 16334 raft_consensus.cc:2764] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:14:42.329973 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.308295 11296 leader_election.cc:400] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15.
I20260709 12:14:42.305613 11326 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:42.331542 11296 leader_election.cc:403] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:14:42.315446 16876 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:42.332391 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54208.8us
I20260709 12:14:42.333148 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:42.333397 16876 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:42.337368 11177 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:14:42.334956 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.267684 16973 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:42.332733 11079 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 14, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:42.352313 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.357537 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.346738 16948 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:42.359469 16853 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:42.359683 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.333838 16658 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15.
I20260709 12:14:42.360188 16853 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:42.353884 16924 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:42.334595 16984 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:42.377305 11325 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:42.411110 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:42.411780 11928 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:42.413177 11255 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:42.414101 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:42.414287 11248 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:42.415233 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:42.415665 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.416484 11254 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:42.415966 11079 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 5 to 14, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:42.419281 11079 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 14, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:42.420203 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.421042 11249 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:42.421857 16334 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:42.405503 16646 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.424355 16646 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.427400 16646 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:42.428136 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 68598.6us
I20260709 12:14:42.429224 16934 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:42.429744 16853 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.433722 16895 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.383489 16876 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:42.435853 16876 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.436416 16876 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:42.356782 16879 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.440714 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.440169 16879 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.442013 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.442148 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.442900 11326 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:42.443251 11331 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:42.443843 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.444641 11328 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:42.445499 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.446583 11330 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:42.447269 16772 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:42.447991 16772 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:42.448321 16772 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.448809 16772 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:42.448943 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.449179 16875 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.450063 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.449595 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:42.451946 16875 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.452744 16875 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:42.429226 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:42.454450 11251 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:42.459347 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.460239 11328 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:42.383747 16948 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:42.462018 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.391603 16894 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.478340 11927 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:14:42.480495 11927 pending_rounds.cc:85] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 12
I20260709 12:14:42.400522 16908 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:42.482990 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53906.3us
I20260709 12:14:42.483271 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.486042 16658 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:42.486518 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.421463 11079 catalog_manager.cc:5697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 5 to 14, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 14 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:42.406630 16924 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:42.494187 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.410283 16975 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:42.505890 11329 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:42.507174 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.508054 11325 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.480826 11927 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:14:42.516968 11927 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:42.536669 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:42.454391 16984 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.466869 16948 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:14:42.568804 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54400us
I20260709 12:14:42.448351 16870 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:42.535550 16929 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.005s
W20260709 12:14:42.577186 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 120344us
I20260709 12:14:42.575199 11079 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 16, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 16 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:42.496807 16658 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:42.579558 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.584124 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.567786 16948 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:42.595315 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.514055 16879 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:42.499665 16894 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:42.602241 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.619058 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.583597 16658 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:42.599187 16841 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:42.571257 16984 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:42.613703 16880 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:42.631747 11324 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:42.573499 16908 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.650669 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:42.635337 16920 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.630s	user 0.013s	sys 0.000s
I20260709 12:14:42.624754 16334 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:42.616782 16929 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:14:42.665763 11181 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:42.651477 16984 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:42.679224 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:42.682093 17013 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:14:42.681928 16285 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.684207 16285 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.673609 16646 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:42.686326 16870 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:42.685520 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 101484us
I20260709 12:14:42.676270 16895 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:14:42.689164 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:42.702668 16894 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:42.245656 11075 rpcz_store.cc:269] 0709 12:14:24.005978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.006322 (+   344us) service_pool.cc:224] Handling call
0709 12:14:42.244839 (+18238517us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:36.219668 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:36.220440 (+   772us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:36.220496 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:14:36.220517 (+    21us) tablet.cc:662] Decoding operations
0709 12:14:36.221788 (+  1271us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:36.221909 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:36.221939 (+    30us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:14:36.223238 (+  1299us) tablet.cc:701] Row locks acquired
0709 12:14:36.223300 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:14:36.223497 (+   197us) write_op.cc:270] Start()
0709 12:14:36.223660 (+   163us) write_op.cc:276] Timestamp: P: 1783599276223422 usec, L: 0
0709 12:14:36.223699 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:14:36.224365 (+   666us) log.cc:844] Serialized 4750 byte log entry
0709 12:14:37.213744 (+989379us) op_driver.cc:464] REPLICATION: finished
0709 12:14:37.410502 (+196758us) write_op.cc:301] APPLY: starting
0709 12:14:37.420333 (+  9831us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:37.463569 (+ 43236us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:37.463720 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:38.875213 (+1411493us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b180035ee40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec642b0ec 00007f3ec6477c0a 00007f3ec64774f7 00007f3ec6477490 00007f3ec642ef48 00007f3eca1431bb 00007f3eca18a5d4 00007f3eca0368da 00007f3eca0393cb
0709 12:14:38.919422 (+ 44209us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:38.920495 (+  1073us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 12:14:38.920568 (+    73us) write_op.cc:312] APPLY: finished
0709 12:14:38.921026 (+   458us) log.cc:844] Serialized 113 byte log entry
0709 12:14:39.678007 (+756981us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:39.710496 (+ 32489us) write_op.cc:454] Released schema lock
0709 12:14:39.771529 (+ 61033us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:39.914725 (+143196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:39.950388 (+ 35663us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":312,"mutex_wait_us":2154058,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":196308,"apply.run_cpu_time_us":51511,"apply.run_wall_time_us":2461529,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54383,"num_ops":12,"prepare.queue_time_us":302874,"prepare.run_cpu_time_us":37713,"prepare.run_wall_time_us":383961,"raft.queue_time_us":267539,"raft.run_cpu_time_us":27248,"raft.run_wall_time_us":239656,"replication_time_us":938754,"spinlock_wait_cycles":8007040,"thread_start_us":1562736,"threads_started":5,"wal-append.queue_time_us":1363853,"wal-append.run_cpu_time_us":18272,"wal-append.run_wall_time_us":2785617}]]}
I20260709 12:14:42.704972 16727 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:42.705492 16727 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:42.706511 16727 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.707810 16727 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:42.712896 16876 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.718865 16646 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.719630 16646 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.722427 16646 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.724725 16908 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.732852 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:42.649271 16879 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.733999 16895 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:42.734375 16880 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:42.734521 16895 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.734916 16880 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:42.735142 16895 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:42.735432 16879 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.741370 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.742122 11329 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.747545 11331 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:42.770197 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:42.773640 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.779708 11328 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:42.780879 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.781538 11325 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.781899 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.785662 11326 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.789948 11177 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.664315 16924 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.791533 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:42.770192 11079 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 12 to 14, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:42.799444 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.013s
W20260709 12:14:42.802477 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.000s	sys 0.013s
W20260709 12:14:42.802984 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.008s
W20260709 12:14:42.803543 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.005s	sys 0.008s
I20260709 12:14:42.803678 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:42.740520 16908 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:14:42.804445 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 886) took 1066 ms. Trace:
I20260709 12:14:42.814679 11079 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 11 to 12, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 12 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:42.806193 11182 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:42.813903 11927 rpcz_store.cc:276] 0709 12:14:41.730374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.743821 (+ 13447us) service_pool.cc:224] Handling call
0709 12:14:41.744236 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:42.571576 (+827340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:42.791492 (+219916us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400651b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:42.797035 (+  5543us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1479555840}
I20260709 12:14:42.792299 16929 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:42.742539 17007 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.827165 11322 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:14:42.802554 16646 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:14:42.824895 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 553875us
W20260709 12:14:42.839802 16617 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.598s	user 0.013s	sys 0.000s
I20260709 12:14:42.807058 16957 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:42.841825 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 152766us
I20260709 12:14:42.844885 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:42.813179 16924 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:42.841725 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.842296 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.849462 11256 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:42.851096 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.854224 11330 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:42.825337 16929 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.804241 16894 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:14:42.797855 16658 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.841379 16646 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:42.866240 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.892799 11331 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:42.901219 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.901964 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:42.903318 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 58545.3us
I20260709 12:14:42.892133 11079 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 8 to 13, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 13 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:42.860311 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.839835 17007 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:42.908366 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 617473us
I20260709 12:14:42.908267 11079 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 5 to 12, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 12 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:42.849532 16989 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:42.857339 16908 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:42.911228 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 292236us
I20260709 12:14:42.913800 16908 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:42.914822 17020 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:42.916453 16908 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:42.917026 17017 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:14:42.917889 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.936112 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:42.936542 11331 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:42.937999 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:42.921797 16658 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:42.921907 11323 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:14:42.922547 11079 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 5 to 13, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 13 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:42.939706 11079 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:42.911275 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.942003 11324 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:42.860358 16929 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:42.914896 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.910614 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:42.892144 16617 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.962136 16617 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:42.958652 16973 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.000s
I20260709 12:14:42.964444 16617 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:42.817122 16918 consensus_queue.cc:1175] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 14 status { last_received { term: 14 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 14 index: 11 } }
I20260709 12:14:42.965411 17034 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:42.965998 17034 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:42.966296 16876 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:42.966485 16853 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.966920 17034 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.901575 16646 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:42.970021 16925 consensus_queue.cc:1175] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 15 status { last_received { term: 12 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: [14.11-14.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:42.971205 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:42.971840 11331 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
W20260709 12:14:42.916471 17018 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:14:42.956800 17028 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:14:42.973510 17028 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:42.974066 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:42.974484 17028 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.957099 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:42.975215 17035 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.958045 11249 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:42.975783 17035 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.958580 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:42.977016 11252 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:42.959698 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:42.910765 16974 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:42.977802 16974 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.935281 16772 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.978199 16974 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:42.979880 16772 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:42.980361 16334 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:42.979933 16646 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.980701 11256 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:42.961623 11253 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:14:42.961346 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:42.967103 16853 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.966339 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:42.977346 16970 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:42.978040 17035 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:42.977499 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.964175 16973 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:42.933025 11929 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:42.941516 11079 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:42.954797 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48877.6us
W20260709 12:14:42.987998 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.990092 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:42.990926 11327 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:42.991339 11253 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:42.991707 11253 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:42.926919 16808 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:42.981598 16780 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:42.995539 11253 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.994519 16772 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.005901 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.006492 11325 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:42.854214 16957 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:42.999922 11079 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 14, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.008862 11079 catalog_manager.cc:5697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 10 to 11, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 11 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:14:43.001478 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:42.998741 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:43.011538 11253 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:43.013763 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47508.3us
I20260709 12:14:43.002846 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.011849 11254 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:42.991145 16752 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:14:43.002980 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 88146.3us
I20260709 12:14:43.003490 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:43.004712 11331 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:43.010432 11248 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
W20260709 12:14:42.996052 16934 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.001s
I20260709 12:14:42.992233 16970 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.004357 16780 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:43.015343 16772 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.019906 16970 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:43.022086 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:43.023265 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:43.024253 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:43.024904 11330 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:43.020844 16780 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.019882 17012 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:14:43.026940 17012 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:43.027487 17012 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:43.028374 17012 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.026850 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:43.029947 17012 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.031108 16984 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:43.031719 16984 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:43.032655 16984 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:43.011790 17033 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:14:43.033843 16841 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:43.034296 16841 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:43.034654 16841 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.012953 16808 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:43.002633 17007 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:43.036899 16808 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:43.038239 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.039397 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:43.044505 11253 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:14:43.046192 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:43.046105 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44727.1us
I20260709 12:14:43.040936 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:43.037295 17007 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:43.015146 16876 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:43.051275 11079 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 12 to 13, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 13 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:14:43.050503 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 99765.4us
I20260709 12:14:43.057554 11325 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:43.019302 16752 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.040382 16934 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:43.107635 11079 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:14:43.174815 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 86481.2us
W20260709 12:14:43.186012 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 235467us
I20260709 12:14:43.187947 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:43.051399 16658 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:43.189958 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:43.054054 16571 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:43.194654 16658 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:43.198635 11178 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:14:43.202270 16895 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.168790 16752 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:43.217584 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.019812 16989 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:43.040134 16948 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.058703 16876 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.054999 16929 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.240792 11079 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 10 to 11, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 11 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:43.204164 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 177385us
I20260709 12:14:43.219368 11182 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:43.054116 16958 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:43.230499 16973 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.215000 16658 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.195362 16998 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:43.232404 16780 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:43.232995 17007 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.219031 16334 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:14:43.256789 16334 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:43.257040 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:43.191551 16934 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:43.059682 16879 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:43.186087 16880 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.206609 16571 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.247609 11927 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
W20260709 12:14:43.248941 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.250182 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:43.245184 16958 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:43.185786 16894 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:14:43.195854 17028 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:14:43.240165 16876 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.268814 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:43.245520 17029 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:14:43.271698 16973 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:43.272755 16948 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.257205 16334 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:43.264158 16571 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:43.274057 16334 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.275290 16334 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.276059 16929 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.277307 16853 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.219656 16970 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.274482 16646 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.280305 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.283370 11329 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.284818 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.281690 11327 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:43.257758 16998 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.290712 16998 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:43.292662 16571 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.295087 16571 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.264422 16958 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:43.296998 16772 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.264415 16617 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.285501 11324 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:43.287405 17031 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:14:43.285738 17030 raft_consensus.cc:2764] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:14:43.316322 16727 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:43.316761 16970 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.319121 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.319780 11325 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:43.303481 16880 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.286233 16973 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:43.322292 16973 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:43.323261 16973 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:43.304684 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:43.304968 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.316910 16727 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:43.309569 16895 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.327939 16727 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.310277 16617 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.259853 16934 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:43.328449 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59738.1us
I20260709 12:14:43.354060 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:43.329052 16285 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:14:43.329360 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72444.3us
I20260709 12:14:43.370642 16285 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.371129 16285 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.372551 16876 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.257721 16989 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:14:43.331830 16957 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:43.309752 16841 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.373806 16646 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:43.374346 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:43.375430 16841 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:43.375571 16772 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:43.332108 16974 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.327957 11324 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:43.376530 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:43.336872 11325 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:14:43.378050 11222 leader_election.cc:400] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16.
I20260709 12:14:43.378343 11222 leader_election.cc:403] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:14:43.379081 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:43.375617 16952 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:43.379580 16870 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:43.379710 16957 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.380015 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:43.380117 16952 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:43.380270 16957 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.380860 16895 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:43.381438 16895 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:43.382364 16895 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.383543 16957 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.356532 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:43.385390 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:43.386258 17028 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:43.307848 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 261701us
I20260709 12:14:43.362044 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.380654 16952 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:43.383992 16918 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:14:43.393846 11327 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.396190 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.396998 11181 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.364112 16958 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.412542 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:43.420588 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:43.422192 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.450749 11182 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.422814 16752 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.424922 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:43.411382 16970 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.425453 16918 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:43.411465 16937 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:14:43.426265 16571 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16.
I20260709 12:14:43.426826 16841 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:43.355860 16934 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:43.471930 11929 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:43.471963 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:43.434156 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.387099 17028 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:43.472802 11928 raft_consensus.cc:2376] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19. Re-sending same reply.
I20260709 12:14:43.388834 16870 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:43.431809 17030 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:14:43.387059 16998 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.492352 16998 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:43.463475 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:43.458735 17029 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:43.433101 16925 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.499660 16925 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.500039 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:43.437721 16908 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.501988 16908 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.504498 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.504817 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.443470 16920 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:43.505808 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.506147 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.506647 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.506974 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.507546 11174 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.431851 16772 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:14:43.507827 11175 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:14:43.478114 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.475080 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.478816 16918 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.433764 17035 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:14:43.470803 16970 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.513940 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.448593 16727 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:43.515563 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.505849 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:43.481120 11926 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:43.478403 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:43.428028 16879 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:14:43.519023 11174 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:43.411561 17012 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:14:43.488106 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.522727 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:43.523502 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.523984 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.524633 11173 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:43.526847 16958 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:43.463837 11329 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:43.460629 16876 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:43.493718 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:43.502158 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.535578 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.519946 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.535620 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.537248 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.538760 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.540083 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:43.513187 11181 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:43.541380 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:43.542953 11178 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:43.543838 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:43.486629 11327 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:43.543040 16752 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:43.559042 11930 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:43.563890 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:43.564277 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:43.567548 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.568272 11324 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:43.557319 11173 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:43.612649 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:43.614040 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:43.621428 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.632822 11326 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:14:43.624454 11182 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:43.548451 16646 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:43.631050 16780 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.630976 16752 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:43.649864 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79050.3us
I20260709 12:14:43.663467 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:43.664422 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 414291us
I20260709 12:14:43.638469 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:43.673559 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.533131 16285 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:43.678828 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 433371us
I20260709 12:14:43.686100 11253 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:43.581039 16937 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:43.687306 11256 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:43.690814 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:43.547142 16658 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.720722 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:43.683307 17048 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:14:43.721464 11249 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:43.727145 11254 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:43.738832 11327 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:43.749778 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:43.750938 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:43.751750 11251 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:43.753386 11927 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:14:43.778612 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:43.717958 17034 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.009s	sys 0.015s
I20260709 12:14:43.690253 17043 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:43.795622 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51778.4us
I20260709 12:14:43.795614 17043 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.793612 11179 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:43.796679 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:43.797314 11929 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:14:43.797672 11929 pending_rounds.cc:85] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 12
I20260709 12:14:43.797889 11929 pending_rounds.cc:107] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:14:43.798123 11929 raft_consensus.cc:2889] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:43.798302 17043 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:43.798804 16920 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:43.799048 17043 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:43.799763 17028 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:43.800647 17028 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:43.801486 16879 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:43.801601 17028 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:14:43.690460 16924 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:14:43.803803 16924 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.802275 16334 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:43.804330 16924 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:43.804774 16334 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:43.805678 16334 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.806766 16334 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.802556 17029 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:43.812049 17029 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:43.802717 16880 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:43.814178 16880 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:43.812950 17029 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.815354 17029 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.698895 17051 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:43.679859 16752 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.799348 16920 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:43.802919 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:43.820847 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:43.835141 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48736us
I20260709 12:14:43.821904 11249 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:43.725611 11324 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:43.831493 11175 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:43.794488 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:43.842402 11248 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:43.725832 16984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.016s	sys 0.010s
I20260709 12:14:43.680150 16780 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:43.844626 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 391759us
W20260709 12:14:43.825673 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.843075 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:43.846814 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:43.849285 16984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.016s	sys 0.010s
I20260709 12:14:43.852265 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:43.749159 16948 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:43.859364 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:43.852415 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:43.837934 16973 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.009s	sys 0.015s
W20260709 12:14:43.843619 17034 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.009s	sys 0.016s
I20260709 12:14:43.871562 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:43.844384 17007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.015s	sys 0.011s
I20260709 12:14:43.879666 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:43.880661 11252 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:43.844566 16975 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:14:43.870846 17027 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.022s
W20260709 12:14:43.874310 16973 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.009s	sys 0.016s
I20260709 12:14:43.841027 16958 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:14:43.861989 16727 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.876556 16841 consensus_queue.cc:1243] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:14:43.900837 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 235186us
I20260709 12:14:43.879710 16870 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:43.908051 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 483161us
I20260709 12:14:43.818276 16880 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:43.876222 16879 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:43.910245 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:43.878750 17007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.015s	sys 0.011s
I20260709 12:14:43.910943 11250 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
W20260709 12:14:43.899022 17027 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.000s	sys 0.023s
I20260709 12:14:43.921228 11326 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:43.865898 16948 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.900022 16853 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:43.954526 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:43.955374 11325 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:43.955338 11324 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:43.926348 16879 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:43.946631 16646 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:43.920193 16934 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.933774 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:43.907752 16918 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.926911 16870 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.879280 16998 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:14:43.902032 16975 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:43.915230 16727 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:43.928709 17028 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 12:14:43.993002 17054 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:14:43.993251 17058 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:14:43.971479 17007 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:43.936105 11329 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
W20260709 12:14:44.003796 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 160829us
I20260709 12:14:44.005081 11327 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:43.932981 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:43.972594 16853 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:44.006371 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 96196.7us
I20260709 12:14:44.004614 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:44.008932 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.985877 16880 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:43.985481 16879 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:43.981170 16952 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.017310 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:43.993709 16285 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.007366 17035 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:44.112480 17035 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.115334 17035 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.064805 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:43.934013 16920 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.088762 11252 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:44.089957 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.067907 11329 pending_rounds.cc:85] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:14:44.087284 16937 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:44.106145 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 174813us
I20260709 12:14:44.106760 16646 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.029584 16752 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:44.029340 16894 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:44.017985 16658 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:44.138827 11329 pending_rounds.cc:107] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:14:44.153187 11329 raft_consensus.cc:2889] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:44.011024 16918 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.118988 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:44.067163 16617 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:44.153220 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.151641 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.116447 16948 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.117172 16780 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:44.212306 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.122277 17030 raft_consensus.cc:2764] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 12:14:44.216503 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:44.214020 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 124145us
I20260709 12:14:44.220263 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.222009 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.222816 11181 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:44.231148 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:44.232483 11252 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:44.154525 16952 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:44.165817 17072 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:44.238219 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.237927 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:44.240262 11327 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:44.208189 17034 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.241588 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.244439 11179 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:44.209407 17035 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.219300 16998 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:44.230621 17012 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:44.218891 16894 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:44.217985 17064 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:44.230456 16780 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.216347 17029 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:14:44.266371 11182 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:44.266873 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:44.267643 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 116098us
I20260709 12:14:44.269130 11251 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:44.270123 11248 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:44.275226 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.246048 16617 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.262987 17062 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:14:44.278842 17068 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:14:44.280853 17076 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:14:44.281986 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.282814 11249 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:44.284739 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.285701 16617 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.248988 16334 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:44.293238 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.303651 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:44.314201 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 380520us
I20260709 12:14:44.342623 16870 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.252681 17034 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.276754 17064 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.346235 16727 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:44.346681 17057 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.364s	user 0.000s	sys 0.002s
I20260709 12:14:44.270850 17080 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:44.252058 17072 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.349888 16934 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:44.352016 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.270579 17047 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:44.212502 16975 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.352435 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.438675 11249 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:44.348012 16894 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.369669 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.295914 17077 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:44.425079 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:44.440672 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 168556us
I20260709 12:14:44.450343 11252 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:44.425604 16918 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:44.426002 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:44.414031 17059 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:14:44.353776 16924 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.002s
I20260709 12:14:44.428972 17078 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.429178 17084 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:44.364390 16973 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:44.264603 16937 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:44.274578 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:44.422263 17034 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.420849 17064 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.257302 16853 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.299896 17012 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:44.308616 16334 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.422355 11253 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:44.276057 16780 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.371718 17083 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:44.400137 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.433102 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.433109 11325 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:14:44.261900 17035 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.421885 16617 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.399817 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:44.434675 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 131116us
I20260709 12:14:44.371168 11327 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:44.451167 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:44.428259 17072 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.382337 17073 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.465675 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.496347 11252 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:44.481416 11176 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:44.491887 11933 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:44.499899 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:44.500648 11327 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:14:44.499195 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 485195us
I20260709 12:14:44.503279 11249 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:44.507728 11934 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:44.499100 17082 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:14:44.514578 11249 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:44.399245 17080 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.497844 17064 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:44.531810 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 80775.9us
I20260709 12:14:44.537539 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.579720 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.463326 16924 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.565989 11325 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:44.580897 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.474725 16879 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.508000 17047 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:44.582026 11327 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:44.582650 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.592309 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.592519 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.569060 16975 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.588130 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.538923 17064 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:44.620065 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.599570 11323 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.584308 16934 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:44.626412 11325 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 12.
I20260709 12:14:44.626721 11932 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
W20260709 12:14:44.629988 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50353.8us
I20260709 12:14:44.589474 17094 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:44.593103 16870 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:44.594761 16841 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:44.639765 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.640749 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:44.641562 11325 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:44.641341 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:44.646252 11249 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:44.595268 17028 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 12:14:44.636439 16880 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.001s
I20260709 12:14:44.649883 17047 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:44.653028 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:44.657436 17030 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:44.659248 17030 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.644202 16974 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.659653 17030 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:44.632258 16934 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.628703 17034 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:14:44.661882 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.662686 11325 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:44.663690 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.671599 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:44.467702 17084 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.673919 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.502635 17083 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.674767 11928 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:44.501983 17073 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.675706 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.676358 11928 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:44.677244 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:44.678242 11175 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:44.673913 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.679332 11173 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:44.680658 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.681205 16780 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.682088 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.682546 11181 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:44.686467 11249 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
W20260709 12:14:44.688951 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95453.6us
I20260709 12:14:44.690124 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.690343 17028 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:44.690749 17064 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.692528 16934 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:44.693260 16925 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.459506 17077 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.694286 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.688294 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.696548 11325 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.709472 11175 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:44.710865 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.711133 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.628551 16975 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:44.715376 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.716919 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.759865 11325 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:14:44.714684 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 130682us
I20260709 12:14:44.467423 17078 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.719399 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.719753 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.762473 11179 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:44.762511 17078 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.713526 16952 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:14:44.762832 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.763600 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.763924 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.713828 16334 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.719759 17047 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:44.715863 16970 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.716150 16908 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.713672 16924 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.765506 17047 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.765044 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.766501 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.767185 11330 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:44.767760 17047 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:44.721170 16841 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:44.768703 16841 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.728698 17084 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.730551 17073 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.731213 17083 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.731633 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.770552 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.726140 17026 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:44.710493 17035 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.770948 11928 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:44.761662 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.692608 16957 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:44.726504 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:44.771289 11249 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:44.733371 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.745409 16920 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:14:44.740936 16975 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:44.731313 17067 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.663s	user 0.001s	sys 0.001s
I20260709 12:14:44.775835 16948 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:14:44.791570 16948 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:44.791932 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.792397 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 932) took 1363 ms. Trace:
I20260709 12:14:44.792790 11251 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:44.770969 16841 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.772233 16984 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:44.754204 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.793959 16984 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:44.792618 16948 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.794934 16984 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.795558 16948 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:44.790688 16975 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:44.765516 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.798198 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.798821 11325 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.798944 11248 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:44.756537 17090 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:14:44.790071 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:44.733304 17093 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:14:44.801657 17090 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:44.802194 17090 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:44.802471 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.803232 11252 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:44.803138 17090 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:44.728837 17092 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:14:44.793504 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.739724 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:44.720784 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3258330630001183s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:14:44.808079 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3096330630000921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 12:14:44.808130 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:44.808562 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2889330630000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
W20260709 12:14:44.809064 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2700330630000281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
W20260709 12:14:44.809403 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2646330630000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
I20260709 12:14:44.810942 11331 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:14:44.815346 17070 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.599s	user 0.000s	sys 0.002s
I20260709 12:14:44.814126 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1035) took 1123 ms. Trace:
I20260709 12:14:44.792805 11177 rpcz_store.cc:276] 0709 12:14:43.429213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.433507 (+  4294us) service_pool.cc:224] Handling call
0709 12:14:43.435207 (+  1700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:43.435644 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.435684 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:43.449340 (+ 13656us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.884264 (+434924us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:43.884314 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.884396 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.730740 (+846344us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:44.792047 (+ 61307us) raft_consensus.cc:1685] finished
0709 12:14:44.792075 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.792329 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":627,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11300,"spinlock_wait_cycles":45383040,"thread_start_us":403673,"threads_started":1,"wal-append.queue_time_us":529163}
W20260709 12:14:44.738619 17056 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.797s	user 0.001s	sys 0.000s
I20260709 12:14:44.790793 16920 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:44.827733 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.838068 11248 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:14:44.729461 17077 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.810349 11323 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:44.806360 11180 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:44.814586 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.822214 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:44.822301 11252 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:44.831359 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:44.811045 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:44.832734 11296 connection.cc:256] RPC call timeout handler was delayed by -1.054933063000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
I20260709 12:14:44.841609 11935 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:44.833763 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:44.843360 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.843483 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.751508 11173 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:44.844396 11325 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.836437 17030 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:44.838099 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44670.3us
I20260709 12:14:44.742664 11934 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:44.848915 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.838843 11251 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:44.848223 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.850703 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:44.851549 16934 raft_consensus.cc:2764] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:14:44.835945 11929 rpcz_store.cc:276] 0709 12:14:43.690349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.690517 (+   168us) service_pool.cc:224] Handling call
0709 12:14:43.749070 (+ 58553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.798541 (+ 49471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.813789 (+1015248us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b540053a7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:44.814044 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110124800}
I20260709 12:14:44.751824 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.852959 11253 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.851588 17047 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:44.837211 17080 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.853402 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 891) took 1381 ms. Trace:
I20260709 12:14:44.857016 11930 rpcz_store.cc:276] 0709 12:14:43.472523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.472660 (+   137us) service_pool.cc:224] Handling call
0709 12:14:44.853216 (+1380556us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5400617954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:44.853346 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56196864}
I20260709 12:14:44.859652 17030 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:44.856554 17064 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:44.840653 16958 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:44.861205 16958 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:44.840584 16975 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:44.840797 16920 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:44.863973 16920 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.854470 17107 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:44.866372 17107 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.866814 17107 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:44.870426 17049 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.073s	user 0.000s	sys 0.002s
I20260709 12:14:44.865368 16920 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:44.872336 17097 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:14:44.874588 11330 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.854621 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:44.879874 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:44.881084 17035 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:44.853113 16841 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:44.881850 17035 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:44.882891 17035 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.883085 16958 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.888139 17062 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:44.888698 17062 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:44.890415 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.855835 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:44.855659 16957 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:44.891338 11177 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:44.892628 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.893303 11251 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 11.
I20260709 12:14:44.894428 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.895458 11251 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.890484 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:44.889588 17062 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.897962 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.898746 11181 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:44.899401 17090 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:44.900556 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.892105 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.905224 11252 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:44.906283 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.906840 11252 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:44.907217 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.907773 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.908433 16957 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:44.908998 11252 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:14:44.909363 11252 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:44.906828 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:44.896042 17077 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.910279 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.910995 11329 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:44.839525 11248 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:44.854791 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.921696 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.924742 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:44.925384 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 70550.7us
I20260709 12:14:44.925559 11177 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:44.927544 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:44.927945 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.928088 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.928567 11177 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:44.928907 11329 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:44.929126 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:44.930747 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.930792 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.926734 17084 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:44.933327 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.933048 11181 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:14:44.939361 11929 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:14:44.939589 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:44.939749 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:44.937285 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.934536 11252 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.941412 11252 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:44.939891 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:44.941298 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.942464 11251 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:44.937971 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.935357 11323 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:44.940793 11178 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:44.943380 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:44.945457 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:44.923745 16841 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.950987 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.947573 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.951145 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.951792 11931 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:44.926941 17030 raft_consensus.cc:2764] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:14:44.951586 11928 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:44.953365 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:44.905298 17107 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.957751 11323 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 12: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:44.956018 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:44.960382 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.940498 16934 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:44.938848 17084 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:44.945089 16908 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:44.967942 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:44.930265 17026 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:14:44.972113 11331 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:44.946552 17078 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:44.950899 16646 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:44.926887 17077 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.959371 16870 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:44.978557 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:44.998195 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.971751 16918 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:44.984586 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:44.975808 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:44.987243 17071 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.007s	sys 0.005s
W20260709 12:14:44.988620 17104 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:14:44.983317 16908 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:44.990206 17110 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:44.958753 16841 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:44.962908 16772 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:14:44.986092 16646 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.994319 17114 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:44.998479 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:45.055049 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 56661.4us
I20260709 12:14:45.009735 16918 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.975369 17084 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:44.963362 17103 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:44.975158 17030 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:44.980383 17078 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.013245 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:45.027918 11929 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:45.019083 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:44.994490 17107 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:44.910097 16957 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:45.009549 17071 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.007s	sys 0.006s
I20260709 12:14:45.055814 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:45.000066 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:45.060278 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:45.080479 11927 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:45.011283 16970 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:45.060040 16918 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:45.055629 16646 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:45.078143 11175 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:14:45.081877 11252 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:14:45.058199 16925 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:45.065902 11331 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:14:45.077538 17117 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:14:45.082659 16970 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.087936 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.070448 17078 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:45.120239 16970 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:45.123420 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.125141 16937 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:45.125732 17071 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:45.130102 16929 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:45.131670 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.101086 11252 pending_rounds.cc:85] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 12
I20260709 12:14:45.133705 11252 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:14:45.134199 11252 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:45.146497 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 146531us
I20260709 12:14:45.153903 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:45.144177 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 200639us
I20260709 12:14:45.155486 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:45.157121 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.161038 17103 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:45.119092 17107 raft_consensus.cc:2764] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:14:45.164455 11181 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:14:45.164851 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:45.101992 16780 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:45.166006 16780 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.166159 17107 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:45.166479 16780 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.166720 17107 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.167248 17107 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.178750 16908 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:45.179797 16908 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:45.196422 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:45.145951 17112 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:14:45.104241 16925 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.200004 17103 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:45.207309 16925 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.208027 16841 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:45.209085 11328 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:45.210700 11328 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:14:45.212344 16780 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.215075 16780 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:45.215782 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:45.232040 11326 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:45.232710 11326 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:45.239689 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2731) took 1734 ms. Trace:
I20260709 12:14:45.242790 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.247114 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1037) took 1498 ms. Trace:
I20260709 12:14:45.247728 11926 rpcz_store.cc:276] 0709 12:14:43.748948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.834467 (+ 85519us) service_pool.cc:224] Handling call
0709 12:14:45.246818 (+1412351us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b5400650254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:45.247008 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068765056}
I20260709 12:14:45.250399 11329 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:14:45.252146 11329 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
W20260709 12:14:45.254343 11176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2733) took 1748 ms (client timeout 1978 ms). Trace:
W20260709 12:14:45.254945 11176 rpcz_store.cc:269] 0709 12:14:43.513934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.515235 (+  1301us) service_pool.cc:224] Handling call
0709 12:14:45.254074 (+1738839us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54000d7854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:45.254211 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219330816}
W20260709 12:14:45.256150 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:45.259074 17123 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.052s	user 0.000s	sys 0.002s
W20260709 12:14:45.260478 17115 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:14:45.260716 16853 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:45.261245 16853 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:45.266240 17107 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.266682 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:45.268759 17012 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:45.269376 17012 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.269872 17012 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.270576 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.270345 16918 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.271860 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:45.272769 16918 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:45.273425 17090 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.262049 16970 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.274123 17090 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.275246 16973 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:45.275882 16970 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:45.276356 17090 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:45.277534 17055 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:45.277408 17026 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:45.281241 11174 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:45.261909 17077 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:45.245091 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.240345 11180 rpcz_store.cc:276] 0709 12:14:43.505135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.507532 (+  2397us) service_pool.cc:224] Handling call
0709 12:14:44.806229 (+1298697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.841474 (+ 35245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.239341 (+397867us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002b7d50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:45.239593 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2986099456}
W20260709 12:14:45.344355 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.286813 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:45.285475 11326 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:45.350387 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 154124us
W20260709 12:14:45.347888 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61208.7us
I20260709 12:14:45.352102 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.289466 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:45.353427 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64047.1us
I20260709 12:14:45.354360 11326 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.278160 17127 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:45.363237 11927 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:14:45.202358 17122 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:45.304883 17077 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:45.386081 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:45.389109 11323 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:45.352819 17090 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:45.399024 17064 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.399748 17064 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.402896 11323 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:45.403900 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:45.404964 16957 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:45.405385 16957 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.405740 16957 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.405805 17094 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:45.407724 11323 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:45.408545 11323 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:45.409579 11323 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.372306 11331 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:45.437548 11323 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:45.412286 17052 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.423s	user 0.002s	sys 0.000s
W20260709 12:14:45.420066 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.978s (SENT) [suppressed 15 similar messages]
I20260709 12:14:45.425750 16646 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.426504 17064 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:45.428741 17047 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.568s	user 0.001s	sys 0.008s
I20260709 12:14:45.301255 17084 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.367870 16970 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:45.375675 17127 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.373823 17114 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:45.431385 11326 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:14:45.431624 17053 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:45.432224 17078 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.390084 17077 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:45.429533 17012 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:45.433359 17050 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.630s	user 0.000s	sys 0.003s
W20260709 12:14:45.434468 17060 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.366s	user 0.002s	sys 0.000s
I20260709 12:14:45.435318 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1040) took 1593 ms. Trace:
I20260709 12:14:45.348420 17055 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:14:45.416812 16853 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.504873 11323 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:45.439584 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:45.505084 17055 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:45.505743 17055 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:45.505829 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66364.2us
I20260709 12:14:45.417423 16957 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.506908 17055 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.508509 17055 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:45.508738 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.508837 16957 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:45.445187 16870 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:45.509690 16870 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:45.510234 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.445760 16925 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.451225 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:45.496910 17047 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:45.480014 16970 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:45.483850 17102 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.066s	user 0.000s	sys 0.005s
I20260709 12:14:45.494964 11326 pending_rounds.cc:85] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:45.442512 17090 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:45.495214 17053 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:45.512035 11331 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:45.470101 17122 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.501188 11932 rpcz_store.cc:276] 0709 12:14:43.841811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.879531 (+ 37720us) service_pool.cc:224] Handling call
0709 12:14:43.904321 (+ 24790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.628094 (+723773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.435023 (+806929us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400650ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:45.435239 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594987776}
I20260709 12:14:45.528561 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:45.510670 16870 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.524878 11326 pending_rounds.cc:107] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:14:45.496735 17127 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:45.494794 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 825) took 2645 ms. Trace:
I20260709 12:14:45.496215 16646 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:45.526257 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 154002us
I20260709 12:14:45.519241 17030 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:45.539695 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:45.484481 17136 raft_consensus.cc:2764] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:14:45.524603 16970 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:45.514415 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2268) took 1355 ms. Trace:
I20260709 12:14:45.530663 11326 raft_consensus.cc:2889] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:45.505255 16853 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:45.532460 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:45.535599 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:45.524048 17064 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:45.541198 11324 rpcz_store.cc:276] 0709 12:14:44.159416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.159773 (+   357us) service_pool.cc:224] Handling call
0709 12:14:44.213820 (+ 54047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.214057 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:14:44.237214 (+ 23157us) log.cc:844] Serialized 14 byte log entry
0709 12:14:45.223530 (+986316us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b1800302fe0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:14:45.512955 (+289425us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:45.513033 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:45.513095 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.514228 (+  1133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.514356 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22997,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":5464,"raft.queue_time_us":21889,"spinlock_wait_cycles":3421568,"thread_start_us":289128,"threads_started":1}
I20260709 12:14:45.534052 11255 rpcz_store.cc:276] 0709 12:14:42.848946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:42.849735 (+   789us) service_pool.cc:224] Handling call
0709 12:14:42.850050 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:42.901918 (+ 51868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:42.901948 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:42.914107 (+ 12159us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.233805 (+319698us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b1800345820. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:14:43.682794 (+448989us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:43.682835 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.682903 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:45.411648 (+1728745us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:45.494512 (+ 82864us) raft_consensus.cc:1685] finished
0709 12:14:45.494544 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.494702 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42289,"raft.queue_time_us":50030,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":9598,"spinlock_wait_cycles":34030080,"thread_start_us":448678,"threads_started":1,"wal-append.queue_time_us":1002002}
I20260709 12:14:45.543542 11323 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.547119 11331 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:45.534209 17034 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:45.534897 17078 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:45.524832 16957 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:14:45.550770 11326 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
W20260709 12:14:45.551108 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:45.553730 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 133703us
I20260709 12:14:45.555263 16925 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:45.555450 17053 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.530308 17122 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.581588 17122 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:45.547596 17077 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.582634 11933 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:14:45.557749 17012 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:45.559824 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2440) took 2613 ms. Trace:
I20260709 12:14:45.547616 16772 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:14:45.547919 16870 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.551165 11326 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:14:45.586349 11326 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:14:45.586637 11326 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:45.565763 17090 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.566568 17084 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:45.567281 17142 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:45.594551 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:45.595188 16970 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.549145 17034 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.603360 17034 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.602133 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:45.576673 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:45.548714 17064 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:45.611985 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 830) took 1834 ms. Trace:
I20260709 12:14:45.613556 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.601879 16989 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.586719 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:45.593129 17127 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.594643 17142 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:45.598678 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:45.573232 11323 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:45.548996 17047 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:14:45.569303 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:45.611074 17007 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:45.616916 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:45.618789 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:45.617101 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.621562 11248 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:45.621917 11324 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:45.622625 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.624154 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.625005 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:45.623356 17051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.806s	user 0.004s	sys 0.007s
I20260709 12:14:45.630689 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:45.622381 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:45.613073 11254 rpcz_store.cc:276] 0709 12:14:43.777282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.465463 (+688181us) service_pool.cc:224] Handling call
0709 12:14:44.497359 (+ 31896us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:44.501339 (+  3980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:44.501383 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:44.529166 (+ 27783us) log.cc:844] Serialized 52 byte log entry
0709 12:14:44.647275 (+118109us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:44.839533 (+192258us) log.cc:844] Serialized 14 byte log entry
0709 12:14:44.840134 (+   601us) log.cc:844] Serialized 14 byte log entry
0709 12:14:45.610171 (+770037us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b1800346c60. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec62a2d70 00007f3ec6296293 00007f3ec628bcea 00007f3ec628c23a 00007f3ec66f6a2a 00007f3ec67060bf 00007f3ec6706038
0709 12:14:45.611602 (+  1431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.611672 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:45.611786 (+   114us) raft_consensus.cc:1685] finished
0709 12:14:45.611800 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.611931 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":580448,"prepare.queue_time_us":19746,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":78,"raft.queue_time_us":3869,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":18334,"spinlock_wait_cycles":7442432,"thread_start_us":254789,"threads_started":2,"wal-append.queue_time_us":197416}
I20260709 12:14:45.607584 17078 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:45.631835 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:45.652233 11933 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:45.652578 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.584210 11257 rpcz_store.cc:276] 0709 12:14:42.946687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:42.991066 (+ 44379us) service_pool.cc:224] Handling call
0709 12:14:42.998179 (+  7113us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:43.013233 (+ 15054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.013271 (+    38us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:43.048495 (+ 35224us) log.cc:844] Serialized 78 byte log entry
0709 12:14:43.389809 (+341314us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:43.456597 (+ 66788us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.457107 (+   510us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.833152 (+376045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.833237 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:45.432714 (+1599477us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:45.559595 (+126881us) raft_consensus.cc:1685] finished
0709 12:14:45.559618 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.559762 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14586,"prepare.queue_time_us":105702,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":5917,"raft.queue_time_us":57976,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":4848,"spinlock_wait_cycles":70130944,"thread_start_us":554065,"threads_started":2,"wal-append.queue_time_us":597541}
I20260709 12:14:45.585405 11331 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:45.636804 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:45.636332 11322 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:45.637679 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:45.654163 11176 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:45.638398 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:45.614930 16841 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:45.648906 11251 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:14:45.635954 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:45.675078 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.649753 11931 raft_consensus.cc:2368] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16.
I20260709 12:14:45.649704 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.651074 11928 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.614717 17084 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.633906 11328 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:45.680078 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:45.637171 17107 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:45.609433 16772 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:14:45.687305 16772 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:14:45.688441 17119 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.478s	user 0.000s	sys 0.002s
W20260709 12:14:45.650839 17051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.834s	user 0.004s	sys 0.007s
I20260709 12:14:45.678474 16808 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:45.701254 16808 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.683548 11328 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:14:45.701772 17064 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.683323 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:45.703253 11180 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.701749 16808 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:45.651762 17146 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:14:45.655576 17026 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:45.706065 17146 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:45.707549 17142 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.686035 17062 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:45.672358 17088 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.710382 17062 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:45.710091 17088 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.710893 17062 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:45.711892 17062 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.712634 17088 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:45.686596 17035 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:45.683264 16973 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:14:45.655495 17047 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:45.690284 17139 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 12:14:45.615731 16920 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.727159 11180 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:45.697046 11176 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:45.698220 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:45.699605 11323 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:45.648185 17007 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:45.738343 11927 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 12:14:45.737891 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 88273.4us
W20260709 12:14:45.714469 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.008s	sys 0.028s
I20260709 12:14:45.715150 11174 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
W20260709 12:14:45.740334 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.008s	sys 0.028s
I20260709 12:14:45.713343 17062 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.731457 11929 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:45.740629 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:45.741031 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 609395us
I20260709 12:14:45.639905 16989 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.717763 17047 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:45.743420 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:45.674187 17078 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.749010 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:45.749894 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.750015 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:45.751005 11177 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:45.751081 11180 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.753221 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.726274 17026 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:45.754736 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2451) took 2070 ms. Trace:
I20260709 12:14:45.755718 11250 rpcz_store.cc:276] 0709 12:14:43.684405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.685310 (+   905us) service_pool.cc:224] Handling call
0709 12:14:43.720239 (+ 34929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.933877 (+213638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.754405 (+1820528us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b540010a254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:45.754651 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438145280}
I20260709 12:14:45.730541 17088 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.692138 17084 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.759066 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.771947 11176 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:45.698894 17094 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:14:45.743637 11324 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
W20260709 12:14:45.759590 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74727us
I20260709 12:14:45.771001 11179 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:45.771991 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:45.742779 17007 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:45.739243 17080 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:45.772748 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.806564 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:45.807159 11175 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 11.
I20260709 12:14:45.808990 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:45.809310 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:45.810031 11248 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:45.806320 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:45.776643 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:45.778301 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:45.811682 11251 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:45.776599 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:45.812009 11177 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:45.801044 11180 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:45.813117 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:45.741724 17035 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:14:45.732254 16920 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.814570 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:45.820720 11253 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.816251 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2280) took 1100 ms. Trace:
W20260709 12:14:45.815886 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 75329.3us
I20260709 12:14:45.818724 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:45.822806 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.821425 11327 rpcz_store.cc:276] 0709 12:14:44.716273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.717505 (+  1232us) service_pool.cc:224] Handling call
0709 12:14:44.717862 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:44.895324 (+177462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:44.895369 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:44.896187 (+   818us) log.cc:844] Serialized 26 byte log entry
0709 12:14:45.088874 (+192687us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:45.088944 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.089050 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:45.687651 (+598601us) spinlock_profiling.cc:243] Waited 8.01 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:45.811342 (+123691us) raft_consensus.cc:1685] finished
0709 12:14:45.811364 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.816176 (+  4812us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3668,"raft.queue_time_us":20124,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":94,"spinlock_wait_cycles":18432384,"thread_start_us":300630,"threads_started":2,"wal-append.queue_time_us":278560}
I20260709 12:14:45.824893 11248 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:45.825157 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:45.825673 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:45.773867 17051 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:45.768280 16925 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:14:45.826570 11327 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:14:45.827541 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:45.827564 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.826054 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:45.821696 11324 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:45.838373 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:45.838670 11324 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:45.837839 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:45.901108 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87128.3us
I20260709 12:14:45.817466 16929 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:45.904011 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:45.774164 16870 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:14:45.905525 11253 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:45.816529 16646 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:45.913872 11177 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:45.774282 17088 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:45.963838 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 126064us
I20260709 12:14:45.940047 11324 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.965950 11324 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:45.810034 16841 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:45.818470 16970 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:45.815011 16924 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.136s
W20260709 12:14:45.957995 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:45.901463 16876 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:14:45.973711 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:45.839619 17151 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:45.876618 17122 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:45.813963 17080 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:45.977310 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:45.952656 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:46.020222 11177 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:45.953356 16853 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:45.981741 11323 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:45.984433 11180 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:45.988497 16870 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:14:46.038563 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 200236us
I20260709 12:14:45.829713 17124 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:46.015017 11324 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:45.831769 17071 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.134s
I20260709 12:14:46.022486 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.053978 17078 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:45.995009 17138 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.513s	user 0.001s	sys 0.000s
I20260709 12:14:46.024657 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:46.028330 17135 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.592s	user 0.002s	sys 0.000s
I20260709 12:14:46.056190 17077 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:46.056222 11177 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:45.983877 17155 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:14:46.058770 17155 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:46.058876 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:46.059823 11177 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:46.029759 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52546.1us
I20260709 12:14:46.061297 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.062698 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.062796 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.031839 17084 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:45.965029 17131 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:45.971530 17127 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.019044 16970 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.036840 11928 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
W20260709 12:14:46.073711 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.973270 16929 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:46.017925 17153 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.030577 17151 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:14:45.970995 16646 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.037453 11327 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:46.067623 11180 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:46.073526 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:46.039238 17112 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:14:46.074661 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:46.078019 11324 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:45.952653 17026 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.082301 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.120832 11252 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:46.071560 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:46.096552 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:46.114606 11327 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:14:46.065703 17078 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.065162 16870 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:46.077343 16918 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:46.124022 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:46.048213 16808 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.128280 16974 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.147323 16989 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.059847 17155 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.046285 16853 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.079843 17151 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.080921 17131 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:46.154852 17154 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.112911 17030 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:14:46.147967 17047 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.078684 16876 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:46.159413 11180 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:14:46.075555 16780 raft_consensus.cc:2764] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:14:46.130475 11177 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:46.129107 17051 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:46.161854 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:46.146530 16841 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.127823 17077 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:46.129980 17124 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:46.079627 17122 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:46.080197 16970 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.081172 17153 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.191340 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:46.192911 11930 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:14:46.196933 17073 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.184450 16920 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.149777 17078 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.196522 11324 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:46.215407 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:46.207957 16876 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.174649 17071 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:46.171403 17151 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:46.194955 11254 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:46.228564 11927 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:46.206612 17030 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.173074 16974 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.218194 16973 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.219425 17124 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:46.240468 11180 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:46.174957 16929 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.229048 16925 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.227766 17088 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.228705 17077 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:46.235632 16876 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:46.254940 11180 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:46.185137 17154 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.245040 17124 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:46.244526 17080 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.219961 17051 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.264964 11331 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:46.243309 17071 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:46.243817 17160 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:46.217525 16870 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.267995 11331 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:46.269061 11180 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:14:46.265782 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 169342us
I20260709 12:14:46.260553 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.291410 17161 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.293856 17160 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:46.295254 17154 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:46.295462 16970 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:46.296833 16808 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.290232 17077 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.256810 16925 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.298557 11252 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:46.243283 17030 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.269301 11331 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.245597 16973 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.236065 17073 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:46.300719 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44157.7us
W20260709 12:14:46.321779 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 1066) took 1489 ms (client timeout 1528 ms). Trace:
I20260709 12:14:46.299333 17171 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:46.321723 17161 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.328871 11180 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:46.267670 16646 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.335273 11331 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:46.338483 17170 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.247622 17131 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:46.340412 17007 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.000s
W20260709 12:14:46.330513 17062 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.002s
I20260709 12:14:46.322124 16853 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:46.344192 11330 rpcz_store.cc:269] 0709 12:14:44.810451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.814303 (+  3852us) service_pool.cc:224] Handling call
0709 12:14:46.299331 (+1485028us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:46.299478 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37136128}
I20260709 12:14:46.344570 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:46.341948 17168 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:46.359169 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.358052 17007 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.365556 17007 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.366500 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:46.367182 11330 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:46.361977 16970 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:14:46.351586 17160 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.371821 11257 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:46.355619 17170 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.377099 17073 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:46.383563 17166 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:14:46.384927 17172 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:46.391093 17030 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:46.393440 17161 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.394728 17153 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:46.396693 16973 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.397370 16974 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:46.407579 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 934) took 2183 ms. Trace:
I20260709 12:14:46.408318 11257 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 13.
I20260709 12:14:46.413615 11331 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:14:46.414199 11331 pending_rounds.cc:85] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:14:46.414487 11331 pending_rounds.cc:107] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:14:46.414790 11331 raft_consensus.cc:2889] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:46.408011 11182 rpcz_store.cc:276] 0709 12:14:44.218361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.218514 (+   153us) service_pool.cc:224] Handling call
0709 12:14:44.241862 (+ 23348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.271423 (+ 29561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.401279 (+2129856us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b5400364554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:46.401471 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55885184}
I20260709 12:14:46.361838 17168 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.420836 16918 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:46.401124 17171 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:14:46.427379 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 102425us
W20260709 12:14:46.418697 17034 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.812s	user 0.007s	sys 0.006s
I20260709 12:14:46.429643 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:46.435513 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:46.435856 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:46.436342 11324 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:46.437930 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.439388 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.440779 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.442662 11256 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:46.443005 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:46.443610 11931 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:46.446658 11250 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:46.358457 17071 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.449232 11330 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:46.448526 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.451604 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.452936 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:46.454012 11324 raft_consensus.cc:2368] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13.
I20260709 12:14:46.461221 11931 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:46.446231 17062 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.463224 17062 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.445973 17078 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.446277 17171 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:46.441851 17034 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.479715 17051 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:46.482781 17165 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.224s	user 0.000s	sys 0.003s
I20260709 12:14:46.479751 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.493816 11256 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:46.494885 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:46.484689 17149 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.783s	user 0.003s	sys 0.000s
I20260709 12:14:46.485152 11330 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:14:46.496840 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.498494 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.499250 11330 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:46.500223 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.501156 11330 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:46.494892 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.477303 16876 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.508888 17171 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.511600 17124 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.519122 17034 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.519780 16925 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.521310 16876 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.485679 17170 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:46.523314 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:46.523552 17163 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 12:14:46.524438 17124 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.515401 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.526566 11330 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:46.513978 17153 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.527704 17078 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.529548 17080 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:46.529985 17078 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.530109 17080 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.532034 17078 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.532035 17080 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.528875 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:46.534476 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:46.538424 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2747) took 1249 ms. Trace:
I20260709 12:14:46.538997 11328 rpcz_store.cc:276] 0709 12:14:45.289459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.702488 (+413029us) service_pool.cc:224] Handling call
0709 12:14:45.702814 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:45.809671 (+106857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:45.809702 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:45.815995 (+  6293us) log.cc:844] Serialized 26 byte log entry
0709 12:14:46.132263 (+316268us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:46.132323 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.132418 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:46.482206 (+349788us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:46.525915 (+ 43709us) raft_consensus.cc:1685] finished
0709 12:14:46.525951 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:46.538329 (+ 12378us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4036,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":63,"spinlock_wait_cycles":12602112,"thread_start_us":332431,"threads_started":2,"wal-append.queue_time_us":398248}
I20260709 12:14:46.544247 17088 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.543597 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:46.515293 17168 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.529474 17153 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.553082 17062 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.556559 17157 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.558290 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.542517 17080 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.553182 17153 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.545850 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:46.544848 17088 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.546747 11328 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:46.527647 17170 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.563125 11256 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:14:46.569929 11256 pending_rounds.cc:85] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 13
I20260709 12:14:46.570245 11256 pending_rounds.cc:107] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:14:46.570544 11256 raft_consensus.cc:2889] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:46.557155 17157 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.579713 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.591297 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.592222 11330 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:46.577129 16937 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:46.596421 16937 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:46.607626 11251 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:46.609849 17075 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:46.610898 17075 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:46.611275 17075 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:46.612190 17075 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.613325 17075 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:46.613448 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.614279 11330 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:46.567629 17080 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.617173 17151 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.621075 11329 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:14:46.621368 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:46.622174 17151 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.623142 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1069) took 1055 ms. Trace:
I20260709 12:14:46.588155 17088 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.623417 11329 rpcz_store.cc:276] 0709 12:14:45.567478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.035718 (+468240us) service_pool.cc:224] Handling call
0709 12:14:46.052651 (+ 16933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:46.621664 (+569013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.622872 (+  1208us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b5400285fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:46.623083 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1306655488}
I20260709 12:14:46.578714 17077 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:14:46.575062 17170 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.628355 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.629241 11330 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:46.631950 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.632910 11329 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
W20260709 12:14:46.615404 17142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.002s	sys 0.020s
I20260709 12:14:46.636832 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:46.637662 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:46.625090 17148 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.021s	sys 0.036s
I20260709 12:14:46.643074 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.644553 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.645107 11330 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
W20260709 12:14:46.640122 17142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.003s	sys 0.021s
I20260709 12:14:46.649194 17170 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.653046 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.656140 11175 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 13. Current term is 14. Ops: []
W20260709 12:14:46.643878 17148 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.022s	sys 0.036s
I20260709 12:14:46.642447 16879 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.661396 11251 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:46.659686 11175 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:46.660218 17172 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:46.662547 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.664007 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:46.664675 11175 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:46.665722 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:46.666322 11175 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:46.666572 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.668066 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.670439 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:46.671428 11180 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:14:46.671516 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:46.676798 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.685973 11175 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:46.686972 11330 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:46.687723 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.688405 11328 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:46.689844 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:46.690518 11180 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:46.663554 17172 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:46.679927 17147 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.869s	user 0.002s	sys 0.000s
I20260709 12:14:46.672919 17142 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:46.672036 16780 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.673568 17151 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:46.696818 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.694796 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.698289 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:46.703059 11330 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:46.697896 17142 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.694363 17171 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:46.677182 17153 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:14:46.700672 17151 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.696318 17172 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:46.713013 17142 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:46.706907 17094 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.715215 11296 leader_election.cc:400] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16.
I20260709 12:14:46.705951 16876 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:46.719949 17151 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:46.728420 16876 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.728971 16876 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:46.729178 17150 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.910s	user 0.002s	sys 0.000s
I20260709 12:14:46.718933 17153 consensus_queue.cc:1243] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:14:46.732098 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:46.733204 17110 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:46.717824 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2296) took 1104 ms. Trace:
I20260709 12:14:46.718284 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.719744 17080 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.720451 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:46.722954 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:46.737354 11926 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:46.736340 11324 rpcz_store.cc:276] 0709 12:14:45.613867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.462696 (+848829us) service_pool.cc:224] Handling call
0709 12:14:46.463096 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.463472 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.463504 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.463985 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:14:46.687972 (+223987us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:46.688022 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.688106 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:46.717587 (+ 29481us) raft_consensus.cc:1685] finished
0709 12:14:46.717613 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:46.717763 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1073,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":3832,"thread_start_us":223688,"threads_started":1,"wal-append.queue_time_us":238517}
I20260709 12:14:46.725337 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.703761 16780 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.716698 11330 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-16.11]   Dedup: 11.9->[12.10-16.11]
I20260709 12:14:46.715232 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2298) took 1021 ms. Trace:
I20260709 12:14:46.741174 11325 rpcz_store.cc:276] 0709 12:14:45.693692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.714293 (+1020601us) service_pool.cc:224] Handling call
0709 12:14:46.714677 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:46.714882 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.714911 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:46.714960 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:46.714994 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.715056 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:46.715169 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:46.741950 11330 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: [11.9-16.11]
I20260709 12:14:46.742904 11325 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:14:46.733850 11223 leader_election.cc:400] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13.
I20260709 12:14:46.723901 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:46.725016 11296 leader_election.cc:403] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:46.717752 17171 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.744405 11223 leader_election.cc:403] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:14:46.745035 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:46.746057 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:46.746445 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:46.746877 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2754) took 1052 ms. Trace:
I20260709 12:14:46.749333 11330 rpcz_store.cc:276] 0709 12:14:45.694551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.716045 (+1021494us) service_pool.cc:224] Handling call
0709 12:14:46.716477 (+   432us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:46.746605 (+ 30128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.746830 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:46.734547 17170 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:14:46.738680 17131 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:46.735397 17110 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:46.754796 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.755220 11330 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:46.752941 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:46.763325 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.764624 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.745939 16841 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16.
I20260709 12:14:46.768635 11926 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:46.747723 17171 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:46.772280 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:46.735777 17094 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.756675 17155 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:46.774132 17155 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:46.774673 17155 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:46.775668 17155 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.766849 16870 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:46.777964 16870 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:46.778374 16870 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:46.778885 16780 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.766063 17030 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:46.771098 17173 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:14:46.752345 16772 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 12. Current term is 13.
I20260709 12:14:46.770872 17055 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:14:46.783350 16984 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:46.784369 17055 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:46.770843 17152 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:14:46.784785 17055 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:46.785264 17152 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:46.769954 17154 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:46.785967 17012 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:46.786403 17012 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:46.787247 17012 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.788515 17012 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:46.770660 17064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.024s	sys 0.014s
W20260709 12:14:46.844931 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52812.6us
W20260709 12:14:46.848507 17064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.025s	sys 0.014s
I20260709 12:14:46.849822 16780 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.870328 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.860085 17170 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:46.874104 17170 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.876308 17170 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.887944 17151 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.790838 17055 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.892356 17154 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:46.894706 17055 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:46.894569 17154 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.872021 17152 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.896247 17152 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.896879 17154 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:46.899785 11328 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:46.870863 17172 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.860541 17110 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.927371 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2768) took 1044 ms. Trace:
I20260709 12:14:46.928009 11182 rpcz_store.cc:276] 0709 12:14:45.883026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.416792 (+533766us) service_pool.cc:224] Handling call
0709 12:14:46.421409 (+  4617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.421781 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.421808 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.422277 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:14:46.926609 (+504332us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b1800276040. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1
0709 12:14:46.926795 (+   186us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:46.926865 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.926966 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:46.927042 (+    76us) raft_consensus.cc:1685] finished
0709 12:14:46.927063 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:46.927312 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2663,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":9111,"spinlock_wait_cycles":11533568,"thread_start_us":14765,"threads_started":1,"wal-append.queue_time_us":10623}
I20260709 12:14:46.935263 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 851) took 1118 ms. Trace:
I20260709 12:14:46.935726 11257 rpcz_store.cc:276] 0709 12:14:45.807745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.807891 (+   146us) service_pool.cc:224] Handling call
0709 12:14:46.925749 (+1117858us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b540010ac54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:46.925912 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263288064}
I20260709 12:14:46.936332 11325 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:46.937269 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 165074us
I20260709 12:14:46.873345 16780 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.942958 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:46.934942 17201 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:14:46.943849 11254 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:46.947005 11926 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:14:46.893345 16937 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.947768 11252 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:46.950479 11257 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:14:46.951022 11181 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2752) took 2010 ms (client timeout 1708 ms). Trace:
W20260709 12:14:46.952570 11181 rpcz_store.cc:269] 0709 12:14:44.940977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.961383 (+ 20406us) service_pool.cc:224] Handling call
0709 12:14:46.950861 (+1989478us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400364a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:46.950958 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371063168}
I20260709 12:14:46.949543 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:46.953876 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:46.954305 17152 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.954742 11182 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:46.955967 17203 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:46.956399 17203 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:46.956512 17152 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.956755 16772 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.956836 17203 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:46.957047 17152 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:46.957487 16772 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:46.959321 17152 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:46.966037 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2753) took 1350 ms (client timeout 1608 ms). Trace:
W20260709 12:14:46.966502 11179 rpcz_store.cc:269] 0709 12:14:45.615210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.615378 (+   168us) service_pool.cc:224] Handling call
0709 12:14:46.965861 (+1350483us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400347bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:46.965977 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280765824}
I20260709 12:14:46.959782 16772 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.968113 17094 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:46.945017 17146 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:46.976193 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.956526 16780 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:46.978350 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.979159 11330 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:46.982101 11179 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:46.988215 11248 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:46.989305 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:46.989945 11257 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:46.991007 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:46.993160 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:46.994407 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:46.995007 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:46.995626 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:46.996541 11931 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:46.996439 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:46.999995 11255 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:47.001034 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.995607 11322 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:47.001230 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.001682 11256 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:47.002190 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:46.998864 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:47.000080 11249 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:47.002998 11250 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:47.004683 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:47.007140 11251 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:46.968582 17155 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:14:47.008736 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:47.016927 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.008869 11257 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:47.004611 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.018217 11253 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:47.011240 16937 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.011974 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2755) took 1394 ms. Trace:
I20260709 12:14:47.020107 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:47.020757 11931 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:14:47.020180 11174 rpcz_store.cc:276] 0709 12:14:45.617176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.617305 (+   129us) service_pool.cc:224] Handling call
0709 12:14:45.617565 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:45.740473 (+122908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.011737 (+1271264us) spinlock_profiling.cc:243] Waited 97.3 ms on lock 0x7b54000cd854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:47.011914 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223853440}
W20260709 12:14:47.013128 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.021781 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:47.022204 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.023259 11254 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
W20260709 12:14:46.991127 17177 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.626s	user 0.008s	sys 0.000s
I20260709 12:14:47.024091 11182 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:47.015686 11926 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:47.026743 11928 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:47.027786 11926 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: [11.9-16.11]
I20260709 12:14:47.025048 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.028406 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:47.029426 11176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 950) took 1217 ms (client timeout 1614 ms). Trace:
W20260709 12:14:47.029961 11176 rpcz_store.cc:269] 0709 12:14:45.811747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.005356 (+1193609us) service_pool.cc:224] Handling call
0709 12:14:47.029349 (+ 23993us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:46.996116 17151 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.030857 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.031997 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:47.033804 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:47.034562 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:47.039072 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:47.039839 11935 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:46.974773 17146 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.035478 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.042557 11251 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:14:47.042971 11147 connection.cc:256] RPC call timeout handler was delayed by -1.2486422070001026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 12:14:47.042945 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:47.043717 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0398422069999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
I20260709 12:14:47.046087 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.047472 11254 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:47.047832 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:47.048466 11176 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:47.003566 17075 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:14:47.030906 17170 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:47.054736 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:47.055044 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:47.056110 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:47.056479 11934 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:14:47.057715 11329 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:47.058799 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:47.059160 11251 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:14:47.061383 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.061214 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.055639 17064 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:47.062122 17083 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:47.046248 16841 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:47.064621 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.027715 17030 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:14:47.071898 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2751) took 1487 ms. Trace:
I20260709 12:14:47.066615 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.067219 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:47.055794 17170 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:47.078178 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.078038 11176 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 12.
I20260709 12:14:47.079767 17170 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:47.065867 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:47.079520 11330 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:47.044479 17171 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.053431 16876 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.082976 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:47.083106 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:47.081781 11327 rpcz_store.cc:276] 0709 12:14:45.584585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.123585 (+539000us) service_pool.cc:224] Handling call
0709 12:14:46.151334 (+ 27749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.225245 (+ 73911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.225292 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.236997 (+ 11705us) log.cc:844] Serialized 26 byte log entry
0709 12:14:46.302314 (+ 65317us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:46.302366 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.302469 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:46.990608 (+688139us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:47.065521 (+ 74913us) raft_consensus.cc:1685] finished
0709 12:14:47.065542 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.071825 (+  6283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43083,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15825,"spinlock_wait_cycles":11100544,"thread_start_us":111581,"threads_started":2,"wal-append.queue_time_us":102240}
I20260709 12:14:47.086509 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:47.068418 17172 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 12 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.085232 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.090467 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:47.091504 11330 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:47.068684 17083 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:47.092548 16984 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:47.098044 16975 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:14:47.105767 17208 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:14:47.081678 16841 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:47.110575 17107 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:47.070187 17091 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:14:47.093061 17152 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:47.135494 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.112977 11327 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:14:47.113142 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.115278 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.067925 17064 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.088279 17078 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:47.090076 17034 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:47.113952 16984 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:47.107431 17083 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:47.141613 17107 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.159449 16876 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.141464 17152 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.162202 17152 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:47.149837 11254 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
W20260709 12:14:47.169994 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:47.086782 16870 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.171016 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:47.175060 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:47.178947 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:47.179893 11174 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:47.182130 11327 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:47.188184 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:47.188910 11176 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:14:47.161284 17107 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:47.149940 17064 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:47.111910 16808 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:14:47.150620 17078 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:47.193974 11255 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
W20260709 12:14:47.193081 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 126511us
I20260709 12:14:47.199720 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.194914 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:47.140144 17012 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.152093 17171 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.181164 17034 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:47.210753 11250 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:47.207398 16876 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:47.204900 17091 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:47.220343 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:47.205804 11327 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:47.221797 11257 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:47.205399 17078 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.226061 17211 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:47.240258 17170 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.243497 17170 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.220465 17012 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.215463 11182 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:47.245290 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:47.248826 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:47.246971 11253 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:14:47.154323 16841 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.249912 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:47.246910 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:47.253294 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:47.244334 17114 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:14:47.250965 17077 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:14:47.256582 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:47.256714 11249 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:47.251035 17131 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:14:47.268194 16658 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:47.270370 17124 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:47.282452 16929 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:47.296425 17214 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:14:47.300834 17157 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.270291 16975 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:14:47.249030 16984 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.302572 16929 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:47.290325 17078 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:47.309473 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.281559 17107 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:14:47.283763 17055 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:14:47.284718 17203 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.319985 11174 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:47.306507 16658 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:47.312835 17142 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.578s	user 0.015s	sys 0.000s
I20260709 12:14:47.310068 17064 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.309988 17124 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:47.304404 17170 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:14:47.306191 17091 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:47.320235 16870 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.292685 17034 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.353533 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:47.355623 11323 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:14:47.355854 17091 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:47.378394 17216 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:14:47.402122 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2755) took 1498 ms. Trace:
I20260709 12:14:47.379163 17223 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:14:47.402562 17217 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:14:47.403012 17088 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:47.402598 11331 rpcz_store.cc:276] 0709 12:14:45.903682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.738410 (+834728us) service_pool.cc:224] Handling call
0709 12:14:46.738799 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.739183 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.739216 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.739628 (+   412us) log.cc:844] Serialized 26 byte log entry
0709 12:14:47.141153 (+401525us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b18002c6e00. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:14:47.247846 (+106693us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:47.247906 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.248006 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:47.401663 (+153657us) raft_consensus.cc:1685] finished
0709 12:14:47.401692 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.402039 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1529,"raft.queue_time_us":1087,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4300,"spinlock_wait_cycles":5469440,"thread_start_us":106434,"threads_started":1,"wal-append.queue_time_us":552202}
I20260709 12:14:47.380388 17110 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:47.380723 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:47.347602 17187 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.752s	user 0.000s	sys 0.007s
I20260709 12:14:47.381448 17222 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:47.386235 17124 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.336890 17064 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.369241 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2757) took 1105 ms. Trace:
I20260709 12:14:47.426855 11325 rpcz_store.cc:276] 0709 12:14:46.263433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.743558 (+480125us) service_pool.cc:224] Handling call
0709 12:14:46.743873 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:46.936733 (+192860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.368840 (+432107us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54000e6354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:47.369139 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442155776}
W20260709 12:14:47.382230 11326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2750) took 1811 ms (client timeout 1964 ms). Trace:
W20260709 12:14:47.428705 11326 rpcz_store.cc:269] 0709 12:14:45.571100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.594284 (+ 23184us) service_pool.cc:224] Handling call
0709 12:14:47.381934 (+1787650us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000c08d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:47.382137 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2444330368}
I20260709 12:14:47.422562 17083 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.408690 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:47.335058 17142 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.433538 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:47.438022 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:47.431079 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 121666us
I20260709 12:14:47.425827 17051 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.408869 17012 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.379220 17221 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:47.440595 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:47.430539 17223 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:47.462190 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.466322 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.553886 11324 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:14:47.554513 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.554651 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:47.555928 17213 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:14:47.560307 17053 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:47.560556 17142 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.436498 17211 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:47.561375 17211 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:47.561846 17211 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:47.566322 11176 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:14:47.441812 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:47.447625 17078 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:14:47.565958 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:47.604267 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 163725us
I20260709 12:14:47.606251 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.553059 17051 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.601298 17171 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.608353 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.601714 17170 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:47.609588 16285 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.609411 17170 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.610103 16285 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.505167 17146 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.611657 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:47.608182 17171 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.612406 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:47.624055 11177 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:47.612915 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 171142us
I20260709 12:14:47.359628 16929 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.618932 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.553525 17223 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.623169 11175 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:47.623400 11176 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:47.648705 17064 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:47.649209 16285 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:47.552954 16876 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:47.650815 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.442366 17088 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:47.652472 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:47.659999 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.678819 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.601481 17078 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.609436 17051 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.662401 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:47.682281 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:47.621754 17181 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:14:47.663341 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.694446 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.695271 11175 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:47.696153 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.618613 17012 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:47.703043 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 96933us
I20260709 12:14:47.704370 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.705802 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.692173 11177 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:47.425508 17152 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.699164 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.701161 17203 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.693607 17078 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:47.624958 17221 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:47.702586 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.664710 17223 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:47.708861 11179 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:14:47.737735 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.738946 11177 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:47.681727 17088 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.776952 11179 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:47.816687 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.817970 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.819149 17152 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.819357 11180 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:47.565932 16658 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:47.824394 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 129382us
W20260709 12:14:47.826247 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 110733us
I20260709 12:14:47.708293 16285 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:47.676998 17222 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:47.835217 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.835023 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:47.822536 17088 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:47.708586 16853 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:47.821998 17221 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:47.779050 17064 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.848704 11323 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:14:47.850251 11323 pending_rounds.cc:85] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:14:47.850512 11323 pending_rounds.cc:107] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:14:47.850749 11323 raft_consensus.cc:2889] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:47.857435 17170 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.858544 17083 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.861768 17220 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.862326 17220 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.864724 17171 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:47.865969 11176 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:47.866389 17221 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.835572 17222 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.703133 16876 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.709908 17012 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.879601 11180 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:14:47.882395 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:47.884479 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.888278 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.896626 17215 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:47.903151 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2301) took 1378 ms. Trace:
I20260709 12:14:47.903605 11329 rpcz_store.cc:276] 0709 12:14:46.524855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.642723 (+117868us) service_pool.cc:224] Handling call
0709 12:14:47.902955 (+1260232us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54001c43d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:47.903077 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":950340992}
I20260709 12:14:47.876441 17222 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:47.874699 17170 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:47.936762 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48599.8us
I20260709 12:14:47.951598 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.892530 17152 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:47.875738 17064 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.952430 17233 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.942627 17170 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:47.976423 17064 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:47.985922 17233 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:47.968039 17171 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:47.999993 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:47.973565 17179 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:48.008469 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:48.007306 17179 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.041306 17179 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:48.042484 17170 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.015774 17152 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:14:48.045878 17196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.005s
I20260709 12:14:48.004819 17211 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.046247 17152 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:48.046516 17196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.007s	sys 0.005s
I20260709 12:14:48.046794 17152 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:48.047151 11182 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:14:48.049279 17211 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:48.005890 17223 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.054783 11175 raft_consensus.cc:2368] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:48.056963 17223 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.008245 17171 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.063128 17203 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.063746 17203 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.064915 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.066057 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2500) took 1016 ms. Trace:
I20260709 12:14:48.066241 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.066504 11254 rpcz_store.cc:276] 0709 12:14:47.049945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.054436 (+  4491us) service_pool.cc:224] Handling call
0709 12:14:48.065873 (+1011437us) spinlock_profiling.cc:243] Waited 78.8 ms on lock 0x7b5400276854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:48.065982 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181174144}
I20260709 12:14:48.067567 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:48.068670 17179 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.059984 17157 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:48.071576 16876 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:48.072090 17157 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.072525 17157 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:48.072690 17012 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:48.079202 17077 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:48.090945 11933 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:14:48.091552 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.092571 11929 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:14:48.093592 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:48.094230 11252 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:48.095335 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:48.098067 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:48.098416 17026 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:14:48.098906 11250 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:48.100317 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2756) took 2024 ms. Trace:
I20260709 12:14:48.101222 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:48.102937 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:48.104729 17131 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.105197 17179 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.105576 17131 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.108364 17131 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:48.100867 11324 rpcz_store.cc:276] 0709 12:14:46.075630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.739908 (+664278us) service_pool.cc:224] Handling call
0709 12:14:46.750271 (+ 10363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:47.554398 (+804127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.099938 (+545540us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54003241d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:48.100233 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847185280}
I20260709 12:14:48.124274 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:48.183866 11325 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:48.129724 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:48.184796 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:48.186590 11323 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:14:48.187355 11323 pending_rounds.cc:85] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:14:48.152781 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:48.188055 11323 pending_rounds.cc:107] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:14:48.188547 11323 raft_consensus.cc:2889] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:48.119364 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:48.189988 11326 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:48.115321 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:48.192178 11254 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:14:48.192704 11254 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 10
I20260709 12:14:48.192965 11254 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:14:48.193254 11254 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:48.186064 11249 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:48.197065 11248 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:48.127471 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:48.091487 11930 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:14:48.205180 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:48.208822 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:48.205468 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 378290us
W20260709 12:14:48.134217 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 72643.1us
I20260709 12:14:48.205250 11929 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:48.223691 11330 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:48.223994 11929 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:48.099107 17026 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:48.204591 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 77275.3us
I20260709 12:14:48.207279 11322 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:48.226490 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.224944 11929 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.093587 17035 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:48.231524 11929 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:48.228375 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:48.233086 11254 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:14:48.228861 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:48.199517 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 317165us
I20260709 12:14:48.240602 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:48.239041 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:48.182241 16285 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.242197 11248 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:48.206734 17034 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:48.243559 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:48.182519 17077 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:48.215543 16876 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:48.235255 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:48.242254 17035 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:48.250651 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:48.244588 17151 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:48.253520 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:48.257005 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 431821us
I20260709 12:14:48.248931 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:48.195163 17012 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 12:14:48.274345 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.256464 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:48.246166 16658 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:48.248118 17182 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:48.236311 17211 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:14:48.270787 17030 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:48.254765 16876 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.244581 17179 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:14:48.227841 17088 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:48.298712 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49845.1us
I20260709 12:14:48.283294 17151 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.282382 17110 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:48.290128 17026 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.329924 17026 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:48.307119 17221 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:48.331589 17221 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.331995 17221 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:48.334848 17240 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:14:48.336758 17151 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:48.248195 16285 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.314985 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:48.310257 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2756) took 2692 ms. Trace:
I20260709 12:14:48.329146 17110 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:48.339263 11323 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:48.272699 17223 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 12:14:48.344311 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.010s	sys 0.021s
I20260709 12:14:48.289706 17182 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.293831 17170 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:48.345059 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:48.282600 17122 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:14:48.253852 17034 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.467993 16646 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:48.468632 16646 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:48.469830 16646 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.346006 17215 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.287840 17124 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.171s
I20260709 12:14:48.458281 11323 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:48.307482 16876 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:48.300561 17030 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:48.458915 17182 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:48.459901 11325 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:48.319700 17080 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:48.294076 16984 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:14:48.270610 17077 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.341722 17244 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:14:48.349444 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.010s	sys 0.021s
I20260709 12:14:48.339418 11178 rpcz_store.cc:276] 0709 12:14:45.617491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.617615 (+   124us) service_pool.cc:224] Handling call
0709 12:14:45.617938 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:45.618277 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:45.618306 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.404738 (+786432us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b1800395d40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a
0709 12:14:47.878084 (+1473346us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b2c00013008. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:47.878747 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 12:14:48.002559 (+123812us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:48.309446 (+306887us) log.cc:844] Serialized 14 byte log entry
0709 12:14:48.309787 (+   341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.309910 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:48.309971 (+    61us) raft_consensus.cc:1685] finished
0709 12:14:48.309990 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:48.310178 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40536,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":2683,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4346,"spinlock_wait_cycles":28237440,"thread_start_us":415534,"threads_started":2,"wal-append.queue_time_us":178899}
I20260709 12:14:48.540756 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.504822 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.542722 11175 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:48.471161 16646 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:48.412523 11929 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:14:48.548462 11929 pending_rounds.cc:85] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 12
I20260709 12:14:48.548858 11929 pending_rounds.cc:107] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:14:48.549160 11929 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:48.457909 17170 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.465031 17122 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:48.462174 17171 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:48.526868 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:48.528496 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 299700us
I20260709 12:14:48.573187 11330 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:48.512454 17080 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.349365 17157 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.506129 17030 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:48.497730 17225 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:14:48.567272 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:48.562558 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:48.524400 17233 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.580713 11222 leader_election.cc:400] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:48.586475 11174 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:48.554868 17078 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.746s	user 0.009s	sys 0.003s
I20260709 12:14:48.535704 16937 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:14:48.595757 17157 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.596472 11253 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:48.531010 17251 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:48.597353 11175 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:48.599821 11222 leader_election.cc:403] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:48.600855 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:48.609340 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.502610 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 955) took 1046 ms. Trace:
I20260709 12:14:48.564739 17170 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:48.568771 17071 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:48.502555 17215 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:48.586547 17152 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.001s
I20260709 12:14:48.570914 17244 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.584282 17229 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:48.619765 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.623803 11322 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:48.634809 11182 rpcz_store.cc:276] 0709 12:14:47.456746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.459032 (+  2286us) service_pool.cc:224] Handling call
0709 12:14:47.459484 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:48.047587 (+588103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.502232 (+454645us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b54002b7854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:48.502528 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350832384}
I20260709 12:14:48.622951 11323 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:14:48.620164 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 45730.1us
I20260709 12:14:48.575678 16841 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:14:48.637327 16937 consensus_queue.cc:1243] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:14:48.662330 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61559.6us
I20260709 12:14:48.664433 16285 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:48.665278 17255 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:14:48.614866 17078 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:48.658162 17222 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.001s
I20260709 12:14:48.672791 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:48.638127 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 384654us
I20260709 12:14:48.640709 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:48.637759 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 351354us
I20260709 12:14:48.623621 11175 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:14:48.633711 17157 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:14:48.668710 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 412322us
I20260709 12:14:48.639442 11178 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:48.598013 17110 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.651826 11323 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:48.679960 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:48.681059 11330 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:14:48.668223 17243 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 12:14:48.718585 11181 raft_consensus.cc:2368] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:48.641822 17152 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.647591 16780 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:14:48.638320 17251 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.679736 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:48.717867 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77233.1us
W20260709 12:14:48.726225 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:48.728062 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.720635 11248 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:48.733484 11175 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:48.654690 17229 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.740949 11175 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:14:48.650228 17244 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.708206 16984 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.718225 16658 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:48.750922 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 99142.6us
I20260709 12:14:48.672217 16937 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
W20260709 12:14:48.757139 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.760386 11175 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:48.736773 17157 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:14:48.736407 17034 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:48.699043 17222 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:48.776079 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66906.5us
W20260709 12:14:48.779449 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77578.3us
I20260709 12:14:48.780400 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:48.800391 11176 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:14:48.776464 16925 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:48.763182 16984 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:48.808142 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:48.804646 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 126117us
W20260709 12:14:48.813365 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.813567 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.814258 11182 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:48.789974 17215 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:48.821713 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.829324 11174 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:48.817934 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.765309 16658 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:48.831631 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 528139us
I20260709 12:14:48.802779 16925 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.792361 16937 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:14:48.832996 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:48.826131 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45907.2us
I20260709 12:14:48.849876 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:48.834532 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:48.792111 17012 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 12:14:48.854576 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 68579us
W20260709 12:14:48.857501 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.824503 17262 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:48.864252 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:48.846305 16925 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:48.898525 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:48.880807 17278 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:48.881412 17257 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:14:48.881817 17287 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:48.882599 17280 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:48.889900 17262 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.918414 17244 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.919201 17261 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.138s
I20260709 12:14:48.846482 17171 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.835855 17221 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.920370 17171 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:48.922039 17249 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:14:48.922780 17171 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:48.923125 17277 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:48.920475 17262 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:48.923588 17171 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:48.923897 17266 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.171s
I20260709 12:14:48.924114 17262 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.924197 17171 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.924618 17264 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 12:14:48.832180 17258 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:48.925472 17260 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:48.907974 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:48.926069 17260 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.927918 17211 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:48.928274 17260 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:48.928642 17211 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:48.929342 17280 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.929967 17211 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.931676 17280 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:48.932216 17280 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:48.933421 17248 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:14:48.933908 17248 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.934320 17248 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:48.934415 17091 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.935230 17091 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.905864 17179 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:48.938097 17276 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:48.815240 17268 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.207s
I20260709 12:14:48.939287 17179 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.918570 17281 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:48.939281 17276 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:48.945050 17026 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.004s
I20260709 12:14:48.921154 17261 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:48.904556 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:48.932672 17280 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:48.937742 17253 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:14:48.946280 11182 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 12:14:48.951879 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47463.2us
I20260709 12:14:48.953364 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:48.922696 17221 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.907838 17012 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:48.889132 17030 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.925836 17258 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:48.939417 16984 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:48.908435 16285 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:14:48.939630 17259 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:14:48.902989 17078 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.909173 16780 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:48.925019 17262 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:48.909008 17275 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:48.965747 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:48.953825 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 200974us
I20260709 12:14:48.912307 17077 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:48.945453 17276 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:48.910482 17223 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:14:48.958099 17026 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:49.038170 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:49.038959 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:48.953088 17170 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:48.990790 17287 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:49.048131 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.040138 16984 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.995605 17258 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:49.046548 11175 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:48.997910 17152 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.046382 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.025084 16853 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.037004 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:49.040841 17151 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.005s
I20260709 12:14:48.992151 17157 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:49.027907 17251 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:49.051520 11222 connection.cc:256] RPC call timeout handler was delayed by -1.0414460019999296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
I20260709 12:14:49.051699 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:49.119596 11323 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:49.028723 16780 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:49.029971 17091 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.031100 17275 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:48.950203 17278 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.067366 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.093408 11324 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:14:49.109347 16876 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.009s
W20260709 12:14:49.058199 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 208418us
W20260709 12:14:49.091244 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 267461us
I20260709 12:14:49.132071 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:48.998366 17250 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:14:49.108464 11322 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:14:49.138170 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.094138 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:49.063021 16984 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.114274 17293 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.059104 17276 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
W20260709 12:14:49.115947 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.117092 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2404) took 1025 ms. Trace:
W20260709 12:14:49.119874 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68243.4us
I20260709 12:14:49.051175 17078 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:14:49.122368 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 259465us
I20260709 12:14:49.149809 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:49.150272 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.048316 17222 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.154978 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.128216 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.131529 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.156100 11928 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:49.132685 11326 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.135726 16876 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.133895 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:49.063596 17287 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:49.111855 17030 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.094802 17110 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:49.142264 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 176633us
I20260709 12:14:49.144094 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:49.170846 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.146342 11257 rpcz_store.cc:276] 0709 12:14:48.091478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.108946 (+ 17468us) service_pool.cc:224] Handling call
0709 12:14:48.109266 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:48.109598 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:48.109645 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:48.110222 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 12:14:48.528120 (+417898us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:48.528178 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.528276 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:49.116810 (+588534us) raft_consensus.cc:1685] finished
0709 12:14:49.116839 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.117006 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1256,"raft.queue_time_us":9359,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":19946,"thread_start_us":417406,"threads_started":1,"wal-append.queue_time_us":543179}
I20260709 12:14:49.150905 11933 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:49.151247 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:49.151520 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:49.234500 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.151908 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.101975 11174 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:49.237190 11174 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:49.156597 11324 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:49.142974 16285 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:49.236088 11926 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:49.162127 11928 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:49.162931 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.242626 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:49.242810 11934 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:49.160161 17293 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.163568 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.245186 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.245860 11927 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:49.164024 16876 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.164638 17170 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.164661 11330 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:14:49.166075 17275 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.122537 17151 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.167342 16984 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.140520 17231 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:49.219996 16879 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:49.220645 17110 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.276935 17110 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.221314 17276 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:49.222196 17262 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.146627 17171 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.158530 16925 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.280395 17151 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.144788 17179 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.147223 17215 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:49.224555 11931 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:49.281430 16925 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.281512 17215 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.284119 17215 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.155028 17237 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.273s
I20260709 12:14:49.221262 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:49.285513 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65359.5us
I20260709 12:14:49.285950 17237 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:49.286388 17237 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.286764 17237 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:49.287107 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:49.226532 17278 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.122746 16853 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.289090 17298 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:49.290241 16853 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.227607 11326 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:49.290604 17298 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:49.291548 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:49.292230 11181 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2772) took 1682 ms (client timeout 1759 ms). Trace:
I20260709 12:14:49.291968 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:49.292900 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:49.228508 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94739.7us
I20260709 12:14:49.293953 17083 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:14:49.196494 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.295662 16925 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:49.296753 17266 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:14:49.218358 16780 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.298791 16780 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.299412 17266 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:49.299855 17266 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:49.300736 17266 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.301990 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:49.232020 11322 raft_consensus.cc:2410] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:14:49.204873 17287 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:49.240765 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 82801.7us
I20260709 12:14:49.361073 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:49.244342 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.259352 17295 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.248543 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.268832 11926 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:49.373407 11933 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:49.373413 11935 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: [15.15-15.15]
I20260709 12:14:49.274477 17292 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:49.347620 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:49.295670 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:49.379912 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 84320.7us
I20260709 12:14:49.380661 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:49.292610 11181 rpcz_store.cc:269] 0709 12:14:47.609559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.613610 (+  4051us) service_pool.cc:224] Handling call
0709 12:14:49.292036 (+1678426us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001f3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:49.292172 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2403531008}
I20260709 12:14:49.268157 17170 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:14:49.400084 11179 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.320757 11326 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:49.347620 11323 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.342505 11174 raft_consensus.cc:606] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:49.347136 11178 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 14.
I20260709 12:14:49.346024 11931 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:14:49.267827 16646 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:49.264954 17286 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.350744 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.407563 11254 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:49.351862 11927 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:14:49.355191 17220 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.260970 17157 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.350036 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:49.306193 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 158297us
I20260709 12:14:49.359143 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.359141 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.409123 17157 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.349526 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2431) took 2051 ms. Trace:
I20260709 12:14:49.411041 11331 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:14:49.412190 17157 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.366437 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.412415 11331 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:49.405540 17151 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.405938 17289 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:49.406109 17291 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:49.285301 17035 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.376262 11322 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [15.11-17.12]
W20260709 12:14:49.388015 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.377127 17262 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:49.333787 17280 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.406968 16646 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.398505 17266 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.407796 17170 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:49.365903 11928 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:49.350907 16879 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.415364 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.398411 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:49.455601 11935 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:49.413596 11323 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:49.425040 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:49.458221 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 204902us
I20260709 12:14:49.411172 11173 rpcz_store.cc:276] 0709 12:14:47.298652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.298864 (+   212us) service_pool.cc:224] Handling call
0709 12:14:47.299360 (+   496us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:47.299822 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.299870 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:47.300626 (+   756us) log.cc:844] Serialized 52 byte log entry
0709 12:14:47.334158 (+ 33532us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:47.334218 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.334321 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:49.294746 (+1960425us) spinlock_profiling.cc:243] Waited 71 ms on lock 0x7b18003cc5e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:49.301725 (+  6979us) raft_consensus.cc:1685] finished
0709 12:14:49.301748 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.349458 (+ 47710us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52682,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":133,"spinlock_wait_cycles":163215616,"thread_start_us":33088,"threads_started":1,"wal-append.queue_time_us":57046}
I20260709 12:14:49.416219 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.419349 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:49.459548 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:49.454541 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60255.7us
I20260709 12:14:49.449970 11326 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:14:49.483208 11179 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:49.453318 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:49.486771 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.499390 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2320) took 1186 ms. Trace:
I20260709 12:14:49.499894 11325 rpcz_store.cc:276] 0709 12:14:48.312795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.509026 (+196231us) service_pool.cc:224] Handling call
0709 12:14:48.509572 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:48.510195 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:48.510240 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:48.510674 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:14:49.289580 (+778906us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b18004a7d60. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:14:49.409659 (+120079us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:49.409813 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.450089 (+ 40276us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:49.499051 (+ 48962us) raft_consensus.cc:1685] finished
0709 12:14:49.499073 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.499311 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":17829,"spinlock_wait_cycles":61200256,"thread_start_us":119531,"threads_started":1,"wal-append.queue_time_us":986580}
I20260709 12:14:49.501792 17280 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:49.502573 17280 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.426663 17035 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:49.505858 17221 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:49.507042 17221 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:49.507393 17221 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:49.508189 17221 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:49.440249 17248 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.002s
I20260709 12:14:49.510032 17080 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.414299 17151 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.510411 17248 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.512897 17151 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.514030 11175 raft_consensus.cc:2368] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21.
I20260709 12:14:49.482053 17170 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:49.425417 17295 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.520833 17295 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.415894 17289 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.522320 17289 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:49.523566 17282 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.407s	user 0.001s	sys 0.003s
I20260709 12:14:49.446162 17258 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.527957 17258 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.528646 17258 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:49.417755 16925 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:14:49.529003 17170 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.440958 16646 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.513216 17248 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.505278 17280 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.437830 17293 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.546844 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.533608 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:49.560002 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 100570us
I20260709 12:14:49.529572 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:49.553303 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 57259.7us
W20260709 12:14:49.540913 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49580.6us
W20260709 12:14:49.442068 17182 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.910s	user 0.013s	sys 0.000s
I20260709 12:14:49.509406 17221 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:49.565932 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:49.439910 17071 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.000s
I20260709 12:14:49.564467 11325 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:49.569136 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:49.566536 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:49.570518 11250 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:49.560534 11252 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:49.571354 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:49.575589 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:49.576365 11249 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.580574 11325 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:49.582055 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:49.584131 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:49.585309 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51833.9us
I20260709 12:14:49.552325 16925 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.534127 16984 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.569928 17264 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:49.533982 17302 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.594365 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.531585 17114 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:49.533792 17170 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.480149 17171 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 12:14:49.438494 17262 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.438333 17215 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.168s
I20260709 12:14:49.481057 17077 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 12:14:49.579633 17071 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:49.623029 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.590734 17030 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:49.629875 17302 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.531464 17278 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:14:49.634696 17304 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:14:49.635574 17300 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:14:49.636201 17295 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:49.618903 17114 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:49.637374 17280 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:49.638572 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.639259 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:49.638720 17114 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.639863 11178 raft_consensus.cc:606] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:49.640239 11178 raft_consensus.cc:614] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:14:49.641019 17299 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:14:49.559392 17012 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:14:49.642665 11932 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:49.638919 17280 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.645685 17280 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.590049 17258 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:49.444383 17286 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.583621 17301 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:49.651892 11926 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:14:49.640650 17114 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.655118 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:49.666368 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.583616 17248 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:49.661181 11932 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:49.414337 16285 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:49.664094 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 97614.4us
I20260709 12:14:49.675302 11249 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:49.679957 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:49.574750 17182 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.595949 17260 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:49.598467 16925 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.620886 16984 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.667980 17110 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:49.679137 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 80895.3us
I20260709 12:14:49.680917 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:49.684134 17182 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.684866 11252 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:49.686393 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:49.689797 11178 raft_consensus.cc:606] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:49.690143 11178 raft_consensus.cc:614] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:49.687546 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:49.691782 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:49.697800 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.698458 11251 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:49.700336 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:49.701094 11249 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:49.708554 16984 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.685276 17260 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.692981 17170 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:49.712251 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.695019 17231 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:49.694835 17273 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:49.714951 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:49.718122 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.720185 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.716601 11326 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:49.722929 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.723716 11326 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:49.724754 11331 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:49.725859 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.680552 17237 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.730216 17237 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.732553 17203 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.711644 17215 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:49.711609 17260 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:49.716197 17170 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.739127 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:49.697295 17110 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.740491 11249 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:49.737694 11331 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:49.738233 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.677253 17301 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.745044 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.746215 11251 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:49.745694 11324 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:49.749069 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.750561 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.734519 17273 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:49.726320 17231 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:49.774494 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.741575 17170 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.781241 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:49.784451 11324 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:49.744741 17315 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.789796 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:49.789951 11249 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:14:49.791551 11178 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:49.787374 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.792645 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:49.792804 11251 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:49.794180 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:49.797576 16929 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:49.797801 17315 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.801364 17182 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:14:49.802421 17237 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:14:49.802237 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:49.810098 17310 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:14:49.810708 17315 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.865981 16646 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.868511 17290 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.869021 16646 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.869107 17290 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.873224 17290 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:49.810904 17316 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:14:49.876327 17088 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:49.811762 17307 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:14:49.882480 16658 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.885243 17229 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.888494 17258 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:49.890435 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 180071us
I20260709 12:14:49.891536 11935 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:49.894327 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2796) took 2019 ms. Trace:
I20260709 12:14:49.894779 11177 rpcz_store.cc:276] 0709 12:14:47.877078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.637156 (+760078us) service_pool.cc:224] Handling call
0709 12:14:48.637615 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:48.748960 (+111345us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:48.769702 (+ 20742us) log.cc:844] Serialized 14 byte log entry
0709 12:14:49.094622 (+324920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:49.094681 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:49.094724 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.894089 (+799365us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54001112d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:49.894139 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.894266 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6203,"prepare.queue_time_us":23020,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":27057,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5377,"spinlock_wait_cycles":255452416,"thread_start_us":319891,"threads_started":1,"wal-append.queue_time_us":376593}
I20260709 12:14:49.897821 17266 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.898453 17266 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.899219 17258 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.911295 11328 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:14:49.914063 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:49.914919 11323 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:49.815001 17262 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.818078 11178 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:49.918080 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:49.818439 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.819757 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.820829 17289 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.821055 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.821877 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.923309 11174 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:49.821810 17273 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:49.822710 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.823628 17233 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:49.826128 17302 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.829617 17286 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:49.901422 17266 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.941816 17170 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.942687 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.943454 11174 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:49.943334 17289 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.943979 17170 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:49.944499 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.945114 11174 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:49.830518 11255 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:49.948191 11255 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:49.831411 16925 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.831893 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:49.949299 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 251343us
I20260709 12:14:49.832962 17301 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:49.833148 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2791) took 1993 ms. Trace:
I20260709 12:14:49.837196 11933 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:49.825520 17030 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:49.838977 17157 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:49.839102 17260 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:49.839938 17256 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.201s	user 0.003s	sys 0.004s
I20260709 12:14:49.974958 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.975927 11256 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:49.842195 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.849887 11324 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:49.978097 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.861287 17231 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:49.862445 17237 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:49.983119 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:49.971542 11255 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:49.862898 17312 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:14:49.847458 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:49.889360 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.897285 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.924695 11328 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:49.932932 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:49.934269 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.922978 17171 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:49.935273 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.916508 17254 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:49.935132 17114 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:49.924445 16658 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:49.986992 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139619us
I20260709 12:14:49.992605 16925 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:49.932111 17233 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:49.938237 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:49.952877 11176 rpcz_store.cc:276] 0709 12:14:47.839983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.068639 (+228656us) service_pool.cc:224] Handling call
0709 12:14:48.068984 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:48.800934 (+731950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.832858 (+1031924us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54000d8c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:49.833081 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676441856}
I20260709 12:14:49.951344 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:49.991506 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:49.954696 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2783) took 2291 ms. Trace:
I20260709 12:14:49.998029 11180 rpcz_store.cc:276] 0709 12:14:47.663514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.820080 (+156566us) service_pool.cc:224] Handling call
0709 12:14:47.820329 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.880049 (+ 59720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.954372 (+2074323us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b54003d22d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:49.954618 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122083072}
I20260709 12:14:49.955698 11323 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:49.971977 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.972290 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:49.982882 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:50.000407 11256 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:49.984997 11255 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:49.988541 11935 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:49.938371 17264 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:49.987313 11929 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:49.987641 11177 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:50.011726 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:49.987262 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:50.012593 11931 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:49.993768 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:49.973246 17157 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:50.015082 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 63846.2us
I20260709 12:14:50.015615 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:50.016036 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:50.018414 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:50.020334 11327 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:50.022330 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.001312 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:50.004611 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.023355 11256 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:50.010946 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:49.990998 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:50.021391 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.026911 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:50.029187 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.030051 11931 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:50.031559 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:50.001112 17315 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:50.036695 17264 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:50.037124 17315 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.037343 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:50.014528 17286 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:14:49.985709 17269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.203s	user 0.028s	sys 0.049s
I20260709 12:14:50.038462 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:50.038120 11251 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:50.040375 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:50.041285 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:50.042150 11256 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:14:49.991230 17254 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.047274 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:50.053357 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:50.038580 17269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.268s	user 0.028s	sys 0.050s
I20260709 12:14:50.021561 17157 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:50.029670 17170 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:50.037606 17315 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:49.992449 16658 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.062340 11935 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:14:50.030710 17288 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:49.988276 17237 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.089099 17288 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.090682 11250 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:14:50.092110 11250 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:14:50.081866 17233 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.080700 11251 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:50.093261 17233 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:50.094060 11251 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:14:50.096164 11251 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:14:50.097993 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:50.098351 11251 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:50.079164 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:50.100416 17221 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:50.100967 17221 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:50.101049 17295 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:50.046752 17267 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:50.101593 17295 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:50.102181 17295 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:50.101971 17221 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.068178 17035 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.103185 17035 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.045964 17236 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:50.066541 17034 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:50.065337 17239 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:14:50.088383 11176 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:14:49.990119 17171 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.100618 17080 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:50.102288 17267 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:50.117080 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:50.140204 11327 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:50.105468 17236 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.082370 17182 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:14:50.071354 17264 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:50.107239 16658 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.115377 16879 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.038372 17286 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:50.084599 17314 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:14:50.085589 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2807) took 1049 ms. Trace:
I20260709 12:14:50.117544 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:50.015350 17083 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:14:50.105448 17248 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:50.117619 17034 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.066735 17254 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:50.083699 17269 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:14:50.138206 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:50.179862 11323 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:50.102418 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2327) took 1040 ms. Trace:
I20260709 12:14:50.180342 11330 rpcz_store.cc:276] 0709 12:14:49.061884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.062035 (+   151us) service_pool.cc:224] Handling call
0709 12:14:49.062434 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:49.249113 (+186679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.102108 (+852995us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540026c5d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:50.102350 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234439552}
I20260709 12:14:50.142984 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:50.085592 17260 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.159132 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:50.162014 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 44601.4us
I20260709 12:14:50.185163 11329 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:50.161180 11179 rpcz_store.cc:276] 0709 12:14:49.036012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.143786 (+107774us) service_pool.cc:224] Handling call
0709 12:14:50.085408 (+941622us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54002b8254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:50.085520 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525684096}
I20260709 12:14:50.152604 17236 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:50.155576 17264 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:50.185526 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:50.164263 16658 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.167881 17170 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:50.214113 17170 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.214722 17170 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:50.216441 17260 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:50.211612 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:50.175755 17292 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.219702 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:50.177572 17034 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:50.222309 11255 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:50.234318 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2411) took 2110 ms. Trace:
I20260709 12:14:50.235313 11255 rpcz_store.cc:276] 0709 12:14:48.123468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.123978 (+   510us) service_pool.cc:224] Handling call
0709 12:14:48.124363 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.830419 (+1706056us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b54001408a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6663dc0 00007f3ec66ee1c0 00007f3ec66ef231 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c
0709 12:14:50.233657 (+403238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.234185 (+   528us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1295691520}
I20260709 12:14:50.232146 11329 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:14:50.179255 17269 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:50.164402 17295 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.249197 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:50.245986 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:50.279453 11330 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:50.280848 11330 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:50.208464 17264 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:50.282783 11330 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:14:50.208194 17229 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:50.284593 11329 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:14:50.300074 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.257656 17007 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:14:50.306186 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.310148 17110 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.323956 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.324785 11933 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:50.327690 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:50.327896 11250 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:14:50.328342 11248 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:50.330535 11250 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:50.338569 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:50.343276 17322 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:14:50.353556 11248 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:50.262411 17292 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.226343 16658 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.360210 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:50.383080 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.357200 17083 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:50.350286 17258 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.307086 17229 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.397233 11179 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:14:50.403653 11175 raft_consensus.cc:2410] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 16 because replica has last-logged OpId of term: 15 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
I20260709 12:14:50.396340 11248 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:50.407799 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:50.361605 17336 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:50.396123 17083 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:50.400398 16879 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.336879 17286 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.427809 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:50.428164 17327 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:14:50.429476 17325 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:14:50.406011 17258 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.412190 17336 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:50.359032 17269 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.450469 17337 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:50.451172 17157 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.453109 11254 raft_consensus.cc:2368] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16.
I20260709 12:14:50.459915 17221 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:50.446516 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:50.467446 17329 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:14:50.432737 17171 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.468552 17083 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:50.474468 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61120.7us
I20260709 12:14:50.478118 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.436103 17339 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:50.452281 17337 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:50.464483 17221 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:50.498164 11179 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:50.429464 17315 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.501848 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:50.469333 17331 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:50.504482 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:50.505371 11174 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:50.434223 16879 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.509547 11178 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:50.498911 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:50.569144 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.585602 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.506898 17269 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:50.508488 17083 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:50.609004 11249 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:14:50.510841 17295 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:50.626571 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.629890 11222 leader_election.cc:400] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21.
I20260709 12:14:50.633563 11222 leader_election.cc:403] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:14:50.631489 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.629696 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:50.645321 11174 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:50.646847 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.500460 17339 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.651932 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:50.598510 17341 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 12:14:50.680687 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.684298 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:50.688395 11178 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:50.721886 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:50.692441 17335 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:14:50.696857 17315 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:50.699344 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2412) took 2575 ms. Trace:
W20260709 12:14:50.701305 11926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1066) took 1336 ms (client timeout 1708 ms). Trace:
W20260709 12:14:50.729007 11926 rpcz_store.cc:269] 0709 12:14:49.365084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.398063 (+ 32979us) service_pool.cc:224] Handling call
0709 12:14:50.701053 (+1302990us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400540454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:50.701228 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443959168}
I20260709 12:14:50.707202 17231 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:50.713861 11328 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:14:50.714113 17319 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.418s	user 0.003s	sys 0.001s
I20260709 12:14:50.716401 17291 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:50.731506 17231 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:50.718312 16879 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:50.720971 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:50.658674 17295 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21.
I20260709 12:14:50.727885 11253 rpcz_store.cc:276] 0709 12:14:48.123820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.124479 (+   659us) service_pool.cc:224] Handling call
0709 12:14:48.205744 (+ 81265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:48.617022 (+411278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.699053 (+2082031us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b540010a754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:50.699271 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897929600}
I20260709 12:14:50.668480 16658 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.672683 17304 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:50.734932 17291 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.716866 17157 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.720363 17229 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:50.682121 17170 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:50.736901 17236 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 12:14:50.756814 17304 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.735595 17088 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
W20260709 12:14:50.769271 17233 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.000s
W20260709 12:14:50.810482 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 139330us
I20260709 12:14:50.752950 17338 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:14:50.820370 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.759356 17341 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:14:50.816605 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:50.829898 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 921) took 1679 ms. Trace:
I20260709 12:14:50.830446 11932 rpcz_store.cc:276] 0709 12:14:49.150767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.151037 (+   270us) service_pool.cc:224] Handling call
0709 12:14:50.829647 (+1678610us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400617954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:50.829805 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941697408}
I20260709 12:14:50.831267 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:50.841084 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2525) took 1337 ms (client timeout 1533 ms). Trace:
I20260709 12:14:50.851320 11929 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:50.804945 17304 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:50.841729 11252 rpcz_store.cc:269] 0709 12:14:49.504028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.571697 (+ 67669us) service_pool.cc:224] Handling call
0709 12:14:50.840785 (+1269088us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b54002a67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:50.841005 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194479872}
I20260709 12:14:50.854161 11934 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
W20260709 12:14:50.855113 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.855378 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2455) took 1031 ms. Trace:
I20260709 12:14:50.855908 11176 rpcz_store.cc:276] 0709 12:14:49.823679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.997479 (+173800us) service_pool.cc:224] Handling call
0709 12:14:50.088114 (+ 90635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:50.129775 (+ 41661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.829254 (+699479us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b54002b8250. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:50.855296 (+ 26042us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207566592}
I20260709 12:14:50.857336 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.814687 17292 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.799793 17236 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.853942 17221 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:50.868373 17345 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:14:50.817139 17315 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:14:50.829312 17258 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:50.878252 11932 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:14:50.884924 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 879) took 1046 ms. Trace:
I20260709 12:14:50.903268 11254 rpcz_store.cc:276] 0709 12:14:49.838405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.010635 (+172230us) service_pool.cc:224] Handling call
0709 12:14:50.884712 (+874077us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54001ce8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:50.884858 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986092288}
I20260709 12:14:50.905289 17170 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.907719 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 915) took 1403 ms. Trace:
I20260709 12:14:50.890776 17288 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:50.801967 17264 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.910715 17264 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.911347 11255 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:50.913691 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.914352 11932 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:50.819155 17157 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
W20260709 12:14:50.900924 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67432.4us
I20260709 12:14:50.932879 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2512) took 1561 ms. Trace:
I20260709 12:14:50.908192 17275 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:50.870118 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.916672 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:50.908144 11935 rpcz_store.cc:276] 0709 12:14:49.504247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.114542 (+610295us) service_pool.cc:224] Handling call
0709 12:14:50.114967 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:50.116077 (+  1110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:50.116118 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:50.116677 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:14:50.511458 (+394781us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:50.511530 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.511621 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:50.907497 (+395876us) raft_consensus.cc:1685] finished
0709 12:14:50.907519 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:50.907657 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"thread_start_us":394379,"threads_started":1,"wal-append.queue_time_us":636444}
I20260709 12:14:50.923720 17236 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.924505 11176 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:14:50.930074 17339 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:50.931561 17291 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:50.853273 17286 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:50.933732 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:50.933395 11249 rpcz_store.cc:276] 0709 12:14:49.371708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.043816 (+672108us) service_pool.cc:224] Handling call
0709 12:14:50.044123 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.612995 (+568872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.932539 (+319544us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b540007e754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:50.932793 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273429760}
I20260709 12:14:50.859961 17347 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:14:50.939656 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.947631 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.948830 11931 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:14:50.943598 17254 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.001s
I20260709 12:14:50.953246 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.954145 11174 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:50.955433 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.951449 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:50.932880 17258 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:50.959333 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:50.962335 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:50.941530 17260 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.724s	user 0.014s	sys 0.000s
I20260709 12:14:50.959710 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.968628 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.969520 11174 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:50.971340 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.972254 11174 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:50.976624 11935 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:50.950131 17292 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:50.972190 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 247475us
I20260709 12:14:50.980916 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:50.943840 17286 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:50.985105 11931 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:14:50.986897 17304 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.989914 17286 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:50.991369 17035 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:50.991255 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.992376 11176 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:14:50.993378 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.993232 17336 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.994341 17336 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:50.994603 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.995697 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.996717 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:50.958279 17273 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:51.000231 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.000537 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.001227 11249 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:51.001443 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.001582 17239 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:51.002063 17239 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:51.002399 11255 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:51.002861 17258 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:51.003695 11930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 917) took 1498 ms (client timeout 1874 ms). Trace:
I20260709 12:14:51.004621 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.005045 11933 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:51.004696 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:51.005066 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:51.008489 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:50.974649 17034 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.707s	user 0.009s	sys 0.004s
I20260709 12:14:50.965407 17347 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:51.010557 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.011379 11249 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-17.11]   Dedup: 11.9->[13.10-17.11]
I20260709 12:14:50.961097 17254 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.012144 17124 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:14:51.012362 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.006718 16658 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:50.984344 17292 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:51.003479 17258 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:51.004266 11930 rpcz_store.cc:269] 0709 12:14:49.504782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.638312 (+133530us) service_pool.cc:224] Handling call
0709 12:14:51.003490 (+1365178us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b540015e0d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:51.003635 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228310400}
I20260709 12:14:50.974656 17260 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:14:50.988708 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:51.004206 17171 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:51.009341 11931 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:50.965039 17314 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:51.002956 17239 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:50.983079 17291 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:14:51.023547 11249 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: [11.9-17.11]
W20260709 12:14:51.022816 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 299021us
I20260709 12:14:51.028841 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:51.056391 11181 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:51.066606 11174 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:51.067533 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:51.059127 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.068531 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:51.063127 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74484.3us
I20260709 12:14:51.071719 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1063) took 1720 ms. Trace:
I20260709 12:14:51.071929 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1068) took 1694 ms. Trace:
I20260709 12:14:51.024048 11178 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:51.047761 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:51.024251 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.013s	sys 0.032s
I20260709 12:14:51.074873 11255 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
W20260709 12:14:51.075479 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.013s	sys 0.032s
I20260709 12:14:51.075486 11174 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:51.076282 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:51.076893 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.077476 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:51.078215 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:51.078624 11174 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:51.080355 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:51.081010 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:51.082202 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:51.082401 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:51.083454 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:51.315027 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.316267 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2453) took 1493 ms. Trace:
I20260709 12:14:51.317787 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:51.320482 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.117156 11173 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
W20260709 12:14:51.322464 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 244166us
I20260709 12:14:51.323962 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.325131 11251 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:51.087374 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.305274 11176 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:51.091135 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:51.328341 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:51.329926 11178 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:51.330099 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.101266 11179 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:14:51.316886 11181 rpcz_store.cc:276] 0709 12:14:49.823095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.989445 (+166350us) service_pool.cc:224] Handling call
0709 12:14:49.989753 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.066758 (+1077005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.315551 (+248793us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000d7854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:51.316019 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2452397824}
W20260709 12:14:51.306921 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 238191us
I20260709 12:14:51.044744 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:51.340977 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:51.343366 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 298561us
I20260709 12:14:51.088061 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.084272 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:14:51.346433 11174 raft_consensus.cc:606] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:51.347168 11174 raft_consensus.cc:614] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:51.346506 11255 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:51.346102 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:51.351783 11074 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 17, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 17 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:51.355737 11251 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:51.355664 11074 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:51.359957 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 2.827s (SENT) [suppressed 5 similar messages]
I20260709 12:14:51.363304 11255 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:14:51.364326 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:51.364637 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:51.213980 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 209358us
I20260709 12:14:51.365757 11181 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:51.208632 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:51.208276 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:51.216061 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:51.429531 11327 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:51.071056 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:51.216925 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 242141us
I20260709 12:14:51.072296 11928 rpcz_store.cc:276] 0709 12:14:49.377545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.547547 (+170002us) service_pool.cc:224] Handling call
0709 12:14:49.547847 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.715587 (+1167740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:50.715619 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:50.715659 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:50.715687 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.071706 (+356019us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54004c2554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:51.071748 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:51.071878 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2685083392}
I20260709 12:14:51.072079 11934 rpcz_store.cc:276] 0709 12:14:49.351506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.351669 (+   163us) service_pool.cc:224] Handling call
0709 12:14:50.849760 (+1498091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.854767 (+  5007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.071518 (+216751us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400617950. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:51.071663 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3444228480}
I20260709 12:14:51.204222 16285 raft_consensus.cc:993] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:14:51.049664 16658 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.022964 17034 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.102126 17064 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.450516 17064 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.085541 17091 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:51.454712 17091 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:51.456799 11256 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:51.457993 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:51.459226 11253 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:51.459617 16937 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:51.460186 17034 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:51.172370 17124 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:14:51.179216 17151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.013s	sys 0.016s
I20260709 12:14:51.453361 17064 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:51.461731 17151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.014s	sys 0.016s
I20260709 12:14:51.056030 17171 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:51.463289 17171 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:51.241796 17182 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.128734 17239 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:51.098724 17254 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:51.091523 17258 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:51.493104 11253 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:14:51.128317 17267 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.949s	user 0.020s	sys 0.033s
W20260709 12:14:51.494330 17267 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.317s	user 0.020s	sys 0.034s
I20260709 12:14:51.494588 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:51.203464 17279 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:51.495185 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:51.192022 17269 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.003s
I20260709 12:14:51.049432 17292 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:51.200726 17304 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:51.126173 17291 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 12:14:51.239974 17323 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.940s	user 0.005s	sys 0.000s
I20260709 12:14:51.102793 17347 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.504065 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:51.098075 17349 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:14:51.199077 17359 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:51.219408 17361 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:51.510805 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:51.382238 17378 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:51.384222 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:51.385452 11328 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:14:51.378489 17346 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:51.389658 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:51.513016 11926 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:14:51.513185 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 79379.8us
I20260709 12:14:51.223090 17340 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:51.513593 11323 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:51.514055 17340 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:51.357424 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:51.406086 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46190.6us
I20260709 12:14:51.414289 11251 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:51.515745 17340 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.436640 17287 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.461870 17124 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.462409 17080 raft_consensus.cc:1081] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:14:51.462455 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1069) took 2047 ms. Trace:
I20260709 12:14:51.431864 17374 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 12:14:51.490180 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:51.490352 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.196239 17272 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:51.510418 17359 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.423007 11927 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:14:51.518539 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:51.405429 17083 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:51.519456 17267 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:51.519419 17182 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.410704 17365 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:51.525296 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2516) took 2117 ms. Trace:
I20260709 12:14:51.528889 11933 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:51.555416 11256 rpcz_store.cc:276] 0709 12:14:49.408379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.051268 (+642889us) service_pool.cc:224] Handling call
0709 12:14:50.051535 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.457329 (+1405794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.524941 (+ 67612us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000884d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:51.525214 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3230475648}
I20260709 12:14:51.521570 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:51.523813 17340 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:51.526225 11929 rpcz_store.cc:276] 0709 12:14:49.414626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.044429 (+629803us) service_pool.cc:224] Handling call
0709 12:14:50.044794 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.851833 (+807039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.461783 (+609950us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400667bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:51.462212 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1783662336}
I20260709 12:14:51.569216 11935 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:14:51.569998 11935 pending_rounds.cc:85] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 11
I20260709 12:14:51.570379 11935 pending_rounds.cc:107] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:14:51.570815 11935 raft_consensus.cc:2889] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:51.524027 17287 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.500017 17091 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.574436 17091 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:51.575155 17287 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:51.575456 17359 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:51.552237 17083 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:51.576339 17083 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:51.576463 17272 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.577867 11250 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
W20260709 12:14:51.583914 17384 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.055s	user 0.003s	sys 0.001s
I20260709 12:14:51.513021 17346 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:51.588217 11079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 260) took 23143 ms (client timeout 15000 ms). Trace:
I20260709 12:14:51.588738 17264 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:51.594286 17264 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:51.595582 17264 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:51.599138 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 242103us
I20260709 12:14:51.595943 17231 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.526782 16879 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:14:51.607460 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 89192us
W20260709 12:14:51.605381 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 89562.6us
I20260709 12:14:51.609776 11327 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:51.588104 11322 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:51.596979 17264 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:51.579417 17272 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:51.618908 17267 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:51.663072 11255 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:14:51.663515 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:51.667551 17378 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:51.668820 17378 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:51.669466 17378 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:51.669878 17378 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:51.671190 17378 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.673835 17371 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:51.674373 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 603220us
I20260709 12:14:51.674721 17371 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.675854 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:51.677901 17291 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.678850 17291 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.680579 17371 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:51.675175 17182 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:51.673273 17378 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:51.686110 11257 raft_consensus.cc:2410] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:14:51.687891 11257 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2515) took 2315 ms (client timeout 1822 ms). Trace:
I20260709 12:14:51.688110 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:14:51.688400 11257 rpcz_store.cc:269] 0709 12:14:49.372502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.419056 (+ 46554us) service_pool.cc:224] Handling call
0709 12:14:51.687583 (+2268527us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b540002e9d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:51.687777 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4941315328}
I20260709 12:14:51.689854 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:51.690584 11934 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:14:51.691008 11934 pending_rounds.cc:85] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 13
I20260709 12:14:51.691291 11934 pending_rounds.cc:107] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:14:51.691519 11934 raft_consensus.cc:2889] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:51.694017 11926 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:14:51.696199 11328 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:51.689877 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:51.698166 17287 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:51.698681 17287 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:51.699064 17287 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:51.659322 17357 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:14:51.700630 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.703214 16658 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:51.703377 17064 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:51.588927 11079 rpcz_store.cc:269] 0709 12:14:28.444428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.453138 (+  8710us) service_pool.cc:224] Handling call
0709 12:14:51.587900 (+23134762us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:44.152828 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:44.349814 (+196986us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:44.349896 (+    82us) write_op.cc:435] Acquired schema lock
0709 12:14:44.349930 (+    34us) tablet.cc:662] Decoding operations
0709 12:14:44.420212 (+ 70282us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:44.420322 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:44.420359 (+    37us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:14:44.423881 (+  3522us) tablet.cc:701] Row locks acquired
0709 12:14:44.423931 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:14:44.424143 (+   212us) write_op.cc:270] Start()
0709 12:14:44.424300 (+   157us) write_op.cc:276] Timestamp: P: 1783599284424068 usec, L: 0
0709 12:14:44.424341 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:14:44.484121 (+ 59780us) log.cc:844] Serialized 6736 byte log entry
0709 12:14:45.417250 (+933129us) spinlock_profiling.cc:243] Waited 8.56 ms on lock 0x7b18000ff580. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:45.773818 (+356568us) op_driver.cc:464] REPLICATION: finished
0709 12:14:46.547402 (+773584us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec35e20ac 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3eca122ef0
0709 12:14:46.677854 (+130452us) write_op.cc:301] APPLY: starting
0709 12:14:46.677960 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:46.974687 (+296727us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:46.979344 (+  4657us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:47.407272 (+427928us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:47.407654 (+   382us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 12:14:47.407716 (+    62us) write_op.cc:312] APPLY: finished
0709 12:14:47.408189 (+   473us) log.cc:844] Serialized 156 byte log entry
0709 12:14:48.047211 (+639022us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:48.047478 (+   267us) write_op.cc:454] Released schema lock
0709 12:14:48.047582 (+   104us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:48.048086 (+   504us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:48.048113 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1000,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":871000,"apply.run_cpu_time_us":42769,"apply.run_wall_time_us":1370236,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75323,"num_ops":17,"prepare.queue_time_us":540379,"prepare.run_cpu_time_us":24028,"prepare.run_wall_time_us":563567,"raft.queue_time_us":130255,"raft.run_cpu_time_us":59184,"raft.run_wall_time_us":961642,"replication_time_us":1319866,"spinlock_wait_cycles":23914752,"thread_start_us":2050810,"threads_started":5,"wal-append.queue_time_us":803188,"wal-append.run_cpu_time_us":24576,"wal-append.run_wall_time_us":2503967}]]}
I20260709 12:14:51.659551 17365 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:14:51.709582 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 65554.5us
I20260709 12:14:51.710311 17365 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:51.714146 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:51.714969 17365 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.715775 17286 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:51.521778 11074 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:51.766889 17231 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:51.773768 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.775983 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:51.777582 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:51.780807 11931 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:14:51.696774 17291 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:51.722725 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:51.746276 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:51.827500 11331 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:51.747823 17267 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.745421 17373 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:51.750713 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:51.752951 17124 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:51.755511 17374 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:51.829860 17374 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:51.831064 17374 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.833027 17374 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:51.764168 17248 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:51.833843 17248 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:51.835193 17248 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:51.704507 17064 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.761399 17346 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:51.793697 17391 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.126s	user 0.005s	sys 0.000s
I20260709 12:14:51.768891 11074 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 16, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 16 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:14:51.846599 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 170929us
I20260709 12:14:51.837285 17248 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:51.870751 17064 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:51.874687 17083 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:51.881354 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.890339 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:51.891215 17275 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.925120 11931 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:51.936502 11930 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:51.936755 11323 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:14:51.939518 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:51.947203 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.869051 17171 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:51.940505 17373 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:51.961318 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.962378 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:51.951325 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:51.985482 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56113.7us
I20260709 12:14:51.943054 17286 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:51.991185 11331 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:51.913384 17292 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:52.001247 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.001844 11323 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:14:51.996057 17064 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:52.005944 17286 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.008966 17373 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.840993 11074 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 17. New cstate: current_term: 17 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:51.971609 17080 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:14:52.014534 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:52.020211 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.987326 17346 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.021986 17373 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:52.022498 17367 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:14:52.038687 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 97578.9us
W20260709 12:14:52.026000 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.025894 17064 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.027576 11074 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.033099 11182 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:14:52.035167 17083 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.036221 17291 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.036957 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:52.037144 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.037838 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:52.045027 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.045789 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:52.056262 17279 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.019s	sys 0.032s
I20260709 12:14:52.056473 17395 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:52.056787 17279 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.019s	sys 0.032s
W20260709 12:14:52.057276 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 306809us
I20260709 12:14:51.996836 17359 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:52.061465 11931 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:52.039791 17367 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.990475 17275 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.065789 11933 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:14:52.067142 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 373611us
I20260709 12:14:52.084735 11323 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:52.062875 17279 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:14:52.086699 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.087478 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.087610 17279 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.089308 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.089813 17080 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:52.090875 17292 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.091214 17080 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:52.092494 17171 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.093384 11930 raft_consensus.cc:2376] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15. Re-sending same reply.
I20260709 12:14:52.093760 16772 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:52.094202 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.094635 17231 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:14:52.095278 17231 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:52.060632 17346 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.096210 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.096992 11175 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:52.096722 17231 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.086722 17221 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.076115 17359 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.093838 17091 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:52.060041 17395 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:52.079396 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:52.099274 11180 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:14:52.154999 11174 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:14:52.155120 11179 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:14:52.156229 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.156404 17359 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:52.098436 17346 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.064903 16879 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.105141 17231 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:52.094681 16772 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.064453 17291 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.106040 11074 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 13 to 17, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 17 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.099709 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:52.109779 17286 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.109884 17239 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:14:52.109819 17394 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:14:52.112932 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.114941 17171 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.163223 17392 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.115463 17292 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.157102 17359 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:14:52.116120 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:52.116300 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.117167 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:52.118755 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:52.118821 11326 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:14:52.119918 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.121398 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:52.121912 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:52.173978 17171 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.129957 17083 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:14:52.133947 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54828us
I20260709 12:14:52.135588 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:52.136420 17080 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:52.103199 17275 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.139498 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 979) took 1121 ms. Trace:
I20260709 12:14:52.139806 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:52.142345 17374 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.196048 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.143090 17124 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:52.198381 17124 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:52.142673 17340 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.003s
I20260709 12:14:52.201979 17239 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:14:52.145888 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2184506979999696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 12:14:52.146771 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.202800 17292 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:52.202847 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2175506979999682s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 12:14:52.204375 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1995506979999391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
I20260709 12:14:52.204696 16772 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.147574 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:52.206066 17239 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:14:52.206413 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 58941.2us
I20260709 12:14:52.206532 17239 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:52.207028 17239 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:52.149541 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.048599 17287 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.209324 11181 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:52.090260 11173 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:52.075219 17367 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:52.211867 17287 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.099758 11255 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:14:52.164076 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.158838 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.154109 17091 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:52.214159 11178 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:52.214315 11323 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:14:52.167591 11256 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:52.115939 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.127511 17064 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.157928 17395 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:52.178674 11325 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:52.217339 11074 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.186757 11182 rpcz_store.cc:276] 0709 12:14:51.018361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.083204 (+ 64843us) service_pool.cc:224] Handling call
0709 12:14:52.139200 (+1055996us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b5400364f54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:52.139417 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599013248}
I20260709 12:14:52.178726 11326 pending_rounds.cc:85] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:14:52.223901 11326 pending_rounds.cc:107] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:14:52.224283 11326 raft_consensus.cc:2889] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:52.190215 11254 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:14:52.175354 17392 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.190439 16879 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.192732 11253 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:52.193325 11330 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:52.194314 17080 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:14:52.195576 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55958.2us
I20260709 12:14:52.196221 17374 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.197531 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:52.145404 17389 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:52.204943 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0484052899998915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 12:14:52.214604 17091 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.189850 11176 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:14:52.154975 17221 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.202793 11251 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:52.201332 17124 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:52.240327 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 104941us
W20260709 12:14:52.245863 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0231506980001086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
I20260709 12:14:52.205173 17292 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:52.241263 17291 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.242102 17080 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:52.251063 11250 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:52.241258 17275 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:52.234618 17340 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:14:52.298267 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.302660 11252 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:14:52.303215 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2433) took 1302 ms. Trace:
I20260709 12:14:52.273568 17091 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:52.298168 17389 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.315960 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:52.322893 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:52.320768 11254 rpcz_store.cc:276] 0709 12:14:51.000764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.003174 (+  2410us) service_pool.cc:224] Handling call
0709 12:14:51.003520 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:52.247093 (+1243573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:52.302859 (+ 55766us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400087ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:52.303138 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2720120576}
I20260709 12:14:52.317600 17267 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:52.342885 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.345602 17292 raft_consensus.cc:2764] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:14:52.347409 11331 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:52.350139 17267 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.361017 17267 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:52.362545 17396 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:14:52.368696 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:52.369920 17080 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.376284 17392 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:52.294250 17287 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:14:52.380025 17221 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.382650 11251 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
W20260709 12:14:52.386569 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:52.388298 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 108083us
I20260709 12:14:52.222623 11074 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 11 to 15, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 15 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:52.393919 11326 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:14:52.394033 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.394958 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:52.402220 17221 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:52.404635 17221 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.342860 17275 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:14:52.405305 17287 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:52.406193 17287 consensus_queue.cc:1243] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:14:52.406443 17373 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.406868 17267 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:52.407132 17221 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.407429 17267 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:52.407907 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.407539 17239 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.408874 17373 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:52.408555 17267 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.410915 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:52.412608 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.413036 11253 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:52.413970 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.405201 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:52.418179 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.410215 17395 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.424343 11328 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: []
W20260709 12:14:52.422201 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77724.7us
I20260709 12:14:52.433073 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:52.348315 11250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:14:52.441816 11250 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:52.442413 11250 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:52.443699 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:52.445937 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:52.432490 17171 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:14:52.436149 17267 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:52.468227 11253 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:14:52.444098 17365 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:52.435454 11933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:14:52.473884 11933 raft_consensus.cc:606] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:52.474465 11933 raft_consensus.cc:614] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:14:52.467458 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 466123us
I20260709 12:14:52.471477 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:52.444840 17239 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.477206 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:52.462378 17034 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.037s	sys 0.025s
W20260709 12:14:52.485870 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 80802.4us
I20260709 12:14:52.467067 17374 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:52.499542 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.470312 17392 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:52.461679 17373 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:52.439930 17275 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 12:14:52.487195 17034 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.037s	sys 0.026s
I20260709 12:14:52.511790 11330 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:14:52.513015 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:52.517372 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:52.519121 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.521207 11323 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:14:52.503526 17365 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:52.521112 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:52.508747 17392 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:52.510196 16879 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.508601 17373 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:52.535858 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.519688 17239 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.539518 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:52.543699 17357 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:52.545581 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:52.529373 17392 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:52.553172 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.553998 11330 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:52.559484 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:52.541850 17395 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.562482 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:52.563827 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:52.536725 17279 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:14:52.572089 17347 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:52.571990 17091 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:52.546814 17389 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:52.581697 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:52.591835 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:52.593128 11251 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:14:52.579806 17367 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.594144 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:52.596050 17367 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:52.574240 17239 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.597283 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.599156 17239 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.527159 11074 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 14, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 14 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.601944 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.602320 17392 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.603364 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.602466 11074 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 14, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:52.604494 17392 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:52.605140 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 410307us
I20260709 12:14:52.606006 11932 raft_consensus.cc:2368] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:52.606890 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.578186 17347 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:52.554911 16879 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:14:52.612139 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:52.613817 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 148261us
W20260709 12:14:52.616258 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.607686 11928 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:52.613081 11256 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:52.621330 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:52.621264 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 423928us
I20260709 12:14:52.628820 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:52.579689 17221 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:14:52.628163 11249 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:52.615409 17347 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:52.654263 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 145292us
I20260709 12:14:52.619498 16879 consensus_queue.cc:1243] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:14:52.643805 11250 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.661370 11248 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:52.625980 17239 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.639230 17357 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:14:52.678442 17401 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:14:52.678572 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:52.673256 16285 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.013s	sys 0.037s
I20260709 12:14:52.679646 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:52.682952 16285 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.013s	sys 0.037s
W20260709 12:14:52.685885 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 62968.1us
I20260709 12:14:52.685410 11250 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:52.664335 16772 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:14:52.684787 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 73619us
I20260709 12:14:52.604998 11074 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 16, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 16 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:52.705017 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.707260 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:52.614734 17373 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.715277 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:52.723374 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:52.728778 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:52.708933 17275 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:14:52.738198 17404 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:14:52.740108 17367 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:52.741406 17367 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.741709 17313 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:52.741959 17367 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.742547 17286 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:52.743418 17359 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:52.744321 17359 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.746053 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:52.684437 17239 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.747809 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:52.747835 17371 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:52.748176 17064 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:52.748620 17080 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:52.746407 11074 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 6 to 18. New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:14:52.754302 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49106.9us
I20260709 12:14:52.717289 17340 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:14:52.673667 17365 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.716751 17357 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:52.794837 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:52.807423 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 78803.3us
W20260709 12:14:52.806493 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 132305us
I20260709 12:14:52.810643 11323 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:52.754303 17392 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:52.759090 17287 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:52.768302 17374 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:52.821480 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.763202 17083 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:52.764125 16772 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:14:52.824699 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:52.825713 17286 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:52.838954 11323 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:52.844007 11928 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
W20260709 12:14:52.849608 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.818576 17287 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.854866 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:52.857313 16285 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:52.850427 16879 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:52.870443 11323 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:14:52.873936 16285 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.881482 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:52.898321 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:52.857412 17287 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:52.843842 17411 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:14:52.909756 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:52.912988 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 167042us
I20260709 12:14:52.917009 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.917955 11329 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:14:52.918397 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:52.914222 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.880678 17355 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:52.861835 16772 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:52.910940 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 163059us
I20260709 12:14:52.921233 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:52.940578 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.952489 11181 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:52.897115 17279 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.900303 17417 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:52.954243 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:52.959961 11327 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:14:52.857774 17392 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.922632 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.961542 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:52.953280 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.972288 11248 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:14:52.956801 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.969718 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:52.970454 11178 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:52.978729 11175 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:52.917454 17427 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.957903 17374 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:52.987748 11174 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:14:52.959427 17417 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:52.983883 16285 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:52.890098 16879 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:52.970486 17371 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:52.989179 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:52.981268 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 78883.2us
I20260709 12:14:52.916373 17248 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:14:52.985391 17374 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.008224 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:52.970609 17286 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:14:52.981967 17267 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:52.988431 17428 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:52.970142 17279 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.022914 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:52.997861 17371 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:53.026299 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 56306.4us
W20260709 12:14:53.018123 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 105123us
I20260709 12:14:53.025620 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.029928 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.031544 11323 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:14:53.034312 11323 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:53.036890 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.035053 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:53.039099 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.039566 17365 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.042927 11933 pending_rounds.cc:85] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 11
I20260709 12:14:53.043293 11933 pending_rounds.cc:107] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:14:53.042786 17365 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.043535 17392 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:53.044239 17433 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:53.044734 17433 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.045092 17433 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:53.045272 17365 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.046144 17431 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.046784 17431 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.048583 17367 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.018141 17374 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:53.049852 17438 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:14:53.050890 17438 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:53.051275 17438 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:53.051128 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.052081 11256 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:53.049419 17431 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.052390 17438 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.007877 17357 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.043555 11933 raft_consensus.cc:2889] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:53.054121 17438 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:53.023733 17267 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.066366 17267 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:53.025821 17359 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:53.040266 17423 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 12:14:53.057727 17248 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.078536 17248 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.036309 17371 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.080950 17248 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:53.054209 17419 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:14:53.083946 17417 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.085559 17417 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:53.087030 16879 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.036618 17440 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:53.068838 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:14:53.075958 17231 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.911s	user 0.008s	sys 0.007s
I20260709 12:14:53.029704 17428 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.093530 16285 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.093818 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.098846 16285 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.095450 11181 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:53.089783 17361 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:14:53.114420 11256 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:14:53.012857 17427 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:53.113282 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 78357.7us
I20260709 12:14:53.116475 16285 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.116127 17231 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:14:53.118855 11255 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:14:53.114856 16879 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:53.119616 16879 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.120043 16879 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.122388 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.110834 17361 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.123351 11251 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:53.123456 17361 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:53.124416 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:53.119087 17425 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:14:53.125765 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 70387.4us
I20260709 12:14:53.127085 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:53.128167 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:53.112291 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.125257 11251 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:53.128641 17447 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.109256 17346 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.131281 17442 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:14:53.133234 17421 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:14:53.110630 17239 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.133575 17080 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:53.134444 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.135769 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:53.135885 17428 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.137092 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:53.106406 16772 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.160701 17374 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.141237 17432 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:53.147251 17287 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:53.149914 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.105283 17365 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.197185 17365 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.198311 11255 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:14:53.199553 17365 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.151974 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:53.202325 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 85330.3us
I20260709 12:14:53.203122 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.204332 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:53.112932 17431 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:53.158099 11256 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:14:53.105777 17267 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.205384 17431 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:53.206390 17431 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.207115 17267 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.115204 17286 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.208074 17286 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.210373 17286 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:53.134513 17420 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:14:53.210841 17409 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.384s	user 0.004s	sys 0.000s
W20260709 12:14:53.213183 17439 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:14:53.213739 11256 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
W20260709 12:14:53.163604 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58415.7us
I20260709 12:14:53.164464 11175 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:53.168277 17279 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.169898 16879 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.215288 17287 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.216897 16879 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.170964 17427 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.219008 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.220611 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:53.221386 11932 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
W20260709 12:14:53.222901 17426 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.356s	user 0.000s	sys 0.002s
W20260709 12:14:53.187110 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.157195 17080 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:53.194792 17432 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:53.223496 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:53.222653 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:53.224225 17080 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.224825 11930 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:53.225703 17080 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.227253 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:53.227793 11928 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:53.230206 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:53.230918 11255 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:53.232060 17083 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:53.232873 17083 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:53.233512 17083 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:14:53.234205 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:53.234833 11930 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:53.235329 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.237118 17291 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:53.237730 17291 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:53.238082 17083 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:53.238667 17291 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.239704 17275 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:53.240130 17275 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:53.240231 17340 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:53.227077 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2442) took 1128 ms. Trace:
I20260709 12:14:53.242255 17340 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:53.237236 17373 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:53.241811 11257 rpcz_store.cc:276] 0709 12:14:52.098421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.098990 (+   569us) service_pool.cc:224] Handling call
0709 12:14:52.099913 (+   923us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:52.111185 (+ 11272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:52.111221 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:52.111782 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:14:52.520568 (+408786us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:52.520615 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:52.572955 (+ 52340us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b5400109354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:52.573003 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:53.210175 (+637172us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:53.226866 (+ 16691us) raft_consensus.cc:1685] finished
0709 12:14:53.226887 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:53.227023 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1226,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":32406784,"thread_start_us":408393,"threads_started":1,"wal-append.queue_time_us":645143}
I20260709 12:14:53.246239 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:53.247126 16879 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:14:53.246533 11323 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.247555 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.226642 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.252087 11930 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:53.252579 11928 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:53.264860 11255 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:53.266165 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.264605 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:53.266285 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 115476us
I20260709 12:14:53.274533 11323 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:53.275203 17267 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:53.275718 17267 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.275539 17433 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.276173 17267 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.279489 17374 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:53.281538 17347 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.002s
I20260709 12:14:53.282017 17347 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.284264 17347 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.240083 17291 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.285876 17248 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.287278 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:53.289769 17248 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:53.289884 11248 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
W20260709 12:14:53.291958 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 257277us
I20260709 12:14:53.260147 17371 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.294909 11250 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:53.296205 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.297048 17279 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:53.298261 17395 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.298518 17279 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:53.298851 17395 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:53.299893 11900 connection.cc:256] RPC call timeout handler was delayed by -1.4948414840000623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
I20260709 12:14:53.300076 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:53.300612 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.300797 11325 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:14:53.301450 17411 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.303061 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.303771 11325 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:53.284925 17361 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.302027 17411 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.314813 17427 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.305127 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.315536 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:53.318735 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.322146 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:53.323354 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 58834.6us
I20260709 12:14:53.312575 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.326809 11325 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:53.224557 17432 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.329442 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:14:53.336031 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.337473 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.333314 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:53.241223 17275 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.302492 17445 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:53.357141 17450 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:14:53.378940 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.332641 17248 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:14:53.392325 17248 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.392885 17248 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:14:53.357295 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.358999 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.396000 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:53.333568 17292 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
W20260709 12:14:53.398499 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 61166.5us
I20260709 12:14:53.354753 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.298323 17371 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:53.349395 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 221291us
I20260709 12:14:53.396870 11328 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:53.399909 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45295.8us
I20260709 12:14:53.400125 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.396301 11330 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
W20260709 12:14:53.401165 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.339178 17428 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:53.403585 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:53.359516 17442 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:53.360795 17433 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.409143 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:53.410598 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 114469us
I20260709 12:14:53.409814 11330 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:53.413004 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.413655 11327 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:53.396301 17347 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:14:53.416896 16285 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:53.422103 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.401628 17438 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:53.422510 16285 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.401984 17091 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:53.362538 17221 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:14:53.407829 17275 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:53.428895 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:53.409966 17365 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:53.403869 17428 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:53.440248 17428 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:53.446322 17365 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:53.446945 17365 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:53.414521 17395 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.439635 17411 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.438592 17457 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:14:53.455538 11222 leader_election.cc:400] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:53.457583 11900 leader_election.cc:400] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16.
I20260709 12:14:53.460558 11900 leader_election.cc:403] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:53.448732 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.426563 17438 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.398339 17427 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.459693 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.436468 16879 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:53.458755 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.462170 11222 leader_election.cc:403] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:53.428473 17091 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.468327 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.469851 11330 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:14:53.459653 16285 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:53.503156 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.501144 17231 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.501556 17091 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.506326 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.456861 17346 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:14:53.515012 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.460259 17457 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.501556 17395 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:53.482244 17438 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:53.504487 17449 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:14:53.521085 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:53.518380 16285 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:53.525805 11330 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
W20260709 12:14:53.518187 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61704.5us
I20260709 12:14:53.515619 17371 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18.
I20260709 12:14:53.534477 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.538893 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.541391 11325 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.411610 17445 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:53.517611 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 88078.2us
W20260709 12:14:53.551975 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 121538us
I20260709 12:14:53.554378 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:53.519618 17427 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.518535 17361 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.517799 17083 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.559844 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.534155 17373 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16.
I20260709 12:14:53.518546 16879 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:53.560961 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:53.567121 11248 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.559804 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.557668 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.538097 17457 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:53.567809 11328 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:53.567914 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.541729 17346 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:53.570971 11327 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:53.558601 17313 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:53.574838 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:53.577306 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.561483 17286 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:53.537415 17231 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.581604 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.582453 11254 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:53.579836 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.584102 16285 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.584811 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.585518 11328 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:53.590703 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.569197 17287 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.572252 17064 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:14:53.602487 17455 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:53.604161 17445 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:53.610436 17456 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:53.617791 17346 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.578471 17313 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.578022 17080 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.628908 17313 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.629067 17080 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.631415 17080 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.622824 17431 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:14:53.624864 17221 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:53.626003 17448 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:53.638062 17221 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.638561 17448 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:53.639209 17448 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:53.640595 17221 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.640450 17448 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.622490 17287 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.642711 17448 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.644404 17287 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.601172 16285 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:14:53.645284 17460 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:53.645799 17460 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:53.646181 17460 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:53.571409 17083 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.650348 16285 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:53.650842 16285 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:53.652410 17083 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.651882 16285 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.653685 16285 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:53.627812 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 199664us
I20260709 12:14:53.662123 16879 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.663797 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.661608 17287 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:53.627250 17231 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.665822 17445 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.571800 17267 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.656680 17428 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.585388 17286 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:53.673552 11325 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:53.690394 11327 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:53.693859 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.694954 11325 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 16: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:53.694172 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.698529 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.699036 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.699391 11325 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:53.701118 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.689994 17286 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:53.672497 17064 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:53.711539 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.742558 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.742290 17064 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:53.711977 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 142276us
I20260709 12:14:53.693236 17287 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.745223 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:53.746183 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60845.8us
I20260709 12:14:53.716499 11327 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:53.719372 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.721856 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.723367 17221 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:53.723918 17080 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.723538 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:53.767261 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 43822.3us
I20260709 12:14:53.726768 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.728152 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.771626 11325 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:53.772367 11325 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:53.730803 17438 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.731594 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:53.686168 17231 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:53.734549 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:14:53.736193 17415 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.869s	user 0.000s	sys 0.014s
I20260709 12:14:53.735970 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.782689 11323 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:53.720366 17428 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.737018 17365 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.740679 17373 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.714547 17392 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:53.771113 11251 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:53.767453 17389 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.796124 11328 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:14:53.779268 11933 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:53.767031 11257 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:53.784231 17267 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.767536 17448 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.789672 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.730362 17248 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:53.814020 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68927.1us
I20260709 12:14:53.814375 11248 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:53.781965 17427 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.815181 11249 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:53.798337 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.777686 17080 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.777455 17221 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.787281 17432 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.800650 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.777845 17083 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:53.715443 17460 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.812879 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:53.822499 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:53.826784 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.837882 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.789994 17231 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.828596 17432 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.850338 17432 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.850772 17434 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:53.851256 17434 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:53.851547 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.851625 17434 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:53.852582 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:53.853335 11330 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
W20260709 12:14:53.855281 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 73535.3us
I20260709 12:14:53.858002 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.859411 17347 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:53.859925 17347 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:53.860549 17221 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.860793 17434 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.862408 17365 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.863114 17434 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.863847 17438 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.864223 17248 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.865537 17287 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.865540 17080 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.867666 17460 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:53.823470 17428 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:53.817117 17034 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:53.875430 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.822739 17083 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.873706 11328 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:53.881561 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:53.877349 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.800841 17392 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.888150 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:53.895442 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:53.895730 11933 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:53.896430 11930 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:53.888888 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.895466 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:53.885416 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:53.898154 11326 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:53.883661 17231 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.833097 17448 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.902264 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1110) took 1773 ms. Trace:
I20260709 12:14:53.911037 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.902050 17373 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:53.917045 17373 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:53.918228 11325 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.918830 17373 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.921784 17248 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:53.922528 17248 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:53.922116 17091 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.923874 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2354) took 1526 ms. Trace:
I20260709 12:14:53.919437 11325 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.924316 11326 rpcz_store.cc:276] 0709 12:14:52.397994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.418136 (+ 20142us) service_pool.cc:224] Handling call
0709 12:14:53.895101 (+1476965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:53.898605 (+  3504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:53.923617 (+ 25012us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54001c48d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:53.923803 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3374151040}
W20260709 12:14:53.926090 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 101921us
I20260709 12:14:53.927196 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.927994 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.928747 11248 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:53.928754 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.889369 17083 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:53.933535 16285 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:53.934335 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:53.936607 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.833693 17427 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.898669 17438 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:14:53.923590 17248 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:53.894186 17080 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:53.897621 17292 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:53.966305 17427 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:53.949519 11325 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:14:53.894091 17034 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:53.940243 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.956530 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.958811 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:53.936124 17458 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:53.962808 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.962996 17313 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.951359 17231 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:14:53.979364 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 381196us
I20260709 12:14:53.967911 11325 pending_rounds.cc:85] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 12
I20260709 12:14:53.880272 17457 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.966660 17292 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:53.907101 17287 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:53.909139 11324 rpcz_store.cc:276] 0709 12:14:52.128516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.152340 (+ 23824us) service_pool.cc:224] Handling call
0709 12:14:52.160562 (+  8222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:52.182126 (+ 21564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:52.182176 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:52.193960 (+ 11784us) log.cc:844] Serialized 26 byte log entry
0709 12:14:52.574432 (+380472us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:52.574488 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:52.574581 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:53.735534 (+1160953us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:53.894944 (+159410us) raft_consensus.cc:1685] finished
0709 12:14:53.894965 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:53.902060 (+  7095us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6837,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":17681,"spinlock_wait_cycles":55045888,"thread_start_us":379997,"threads_started":1,"wal-append.queue_time_us":672624}
I20260709 12:14:53.974858 11326 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:53.981959 11325 pending_rounds.cc:107] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:14:53.951941 17389 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:53.993767 11325 raft_consensus.cc:2889] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:53.989315 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:54.158039 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:54.054569 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:54.014385 11322 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
W20260709 12:14:54.158011 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 195350us
I20260709 12:14:54.161726 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.162432 11322 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:54.055521 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67563.8us
I20260709 12:14:53.979794 17427 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:14:54.158496 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 165155us
I20260709 12:14:54.167791 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:54.044000 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.979707 17080 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:14:54.171933 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:54.167433 17445 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:54.172927 11324 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:54.169323 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:53.992022 17287 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.995705 17091 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:54.192870 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 138449us
I20260709 12:14:54.193027 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:53.991739 17292 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:54.188366 11255 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:14:54.203294 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:54.190797 11322 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:54.193797 11928 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:54.193831 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:54.196230 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:54.197757 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.165711 17434 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.202335 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.171031 17427 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:54.274115 17427 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:54.246954 17313 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:53.991514 17432 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.197s
I20260709 12:14:54.277400 17434 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:53.989800 17460 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:54.231573 17395 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:54.211536 11932 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:14:54.281337 11932 pending_rounds.cc:85] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 14
I20260709 12:14:54.281644 11932 pending_rounds.cc:107] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:14:54.176236 16879 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:14:54.281838 17474 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 12:14:54.281900 11932 raft_consensus.cc:2889] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:54.284219 11927 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:14:54.285398 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:54.286741 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:54.287555 11254 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:54.288394 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:53.986819 17438 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:54.289620 17438 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.290436 11331 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:14:54.194792 17287 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:54.226651 17034 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:54.280172 17434 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:54.204733 17355 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:54.299983 17355 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:54.253098 11328 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:14:54.255119 11933 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:14:54.249900 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:54.268287 11324 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
W20260709 12:14:54.307935 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.228137 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:54.172504 17080 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.221613 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:54.299057 17481 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:14:53.990149 17275 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.230s
I20260709 12:14:54.301461 11326 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:14:54.277000 17286 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:54.342684 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 157337us
I20260709 12:14:54.343150 11255 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:54.347357 11928 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:54.347365 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:54.348511 11255 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:54.349785 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:54.351444 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:54.353755 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.354635 11325 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:54.350660 11255 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:54.356340 11927 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:54.357649 11328 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:14:54.300544 17355 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:54.358808 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.359673 11328 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:54.364763 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:54.334961 17080 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:54.370879 11928 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:14:54.342597 17457 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:54.388236 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:54.388886 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:54.389497 11933 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:14:54.390702 11935 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:54.402951 17448 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.391202 17286 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:54.351785 17445 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:54.410532 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:54.407343 11931 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:54.411689 11254 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:54.412973 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:54.356484 17340 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.449877 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:54.449965 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 222878us
I20260709 12:14:54.450909 11255 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:54.451081 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:54.408623 17347 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 12:14:54.454756 11255 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:54.457808 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:54.460546 11254 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:54.473187 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.447126 17275 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:14:54.450166 17340 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:54.461820 17433 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:14:54.459470 17347 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:54.487215 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 123620us
I20260709 12:14:54.463881 17365 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:14:54.504393 11324 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:14:54.485153 17428 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.599s	user 0.007s	sys 0.004s
I20260709 12:14:54.515610 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.516498 11175 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 12.
I20260709 12:14:54.520911 11181 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:14:54.521567 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.522115 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:54.522955 11252 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:54.525043 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.525261 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.528652 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:54.529976 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.488549 17292 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:54.538141 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.539075 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.519769 17428 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:54.542985 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.544831 11181 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:54.539958 11174 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
W20260709 12:14:54.564025 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.531454 17438 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:54.572669 11927 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:54.565470 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.568768 11322 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:54.574329 11179 raft_consensus.cc:2410] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:14:54.574379 11182 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:14:54.532832 17091 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:14:54.583061 17492 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:14:54.583511 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.589722 17275 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:54.590907 17347 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.591943 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:54.592859 17340 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.593717 16772 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:54.594440 16772 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:54.594885 17292 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:54.595630 16772 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.540755 17493 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:54.598102 17493 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:54.598579 17493 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:54.602027 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78007.2us
I20260709 12:14:54.572183 17433 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.603017 17091 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.607738 17433 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.567363 17498 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:54.603282 17438 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.612635 17438 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.563575 17427 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.615221 17427 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:54.616587 17083 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.649s	user 0.006s	sys 0.006s
I20260709 12:14:54.618372 11179 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:54.624873 11181 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:54.617057 17083 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:54.630089 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.630522 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:54.631587 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 896) took 1492 ms (client timeout 1594 ms). Trace:
I20260709 12:14:54.609527 17091 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:54.639544 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:54.640820 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:54.681972 11928 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:54.681838 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.643530 11176 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:14:54.645139 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:54.646091 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:54.630446 17470 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:14:54.660108 17355 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:54.627357 17365 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:14:54.663823 17433 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:54.663468 17287 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:54.623245 17347 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:14:54.669416 11173 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:54.675799 11177 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:54.680481 11182 raft_consensus.cc:2410] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:14:54.678092 11253 rpcz_store.cc:269] 0709 12:14:53.135328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.135472 (+   144us) service_pool.cc:224] Handling call
0709 12:14:54.627528 (+1492056us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:54.627710 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862446848}
I20260709 12:14:54.693233 17287 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.690351 17292 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:54.694977 11175 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:54.696511 17355 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.683048 11181 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:54.677590 17427 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:54.699232 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64434.5us
W20260709 12:14:54.676260 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 225248us
I20260709 12:14:54.686061 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:54.686156 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:54.701308 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:54.702416 11932 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:14:54.705657 17504 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:54.708170 17083 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.714571 11324 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:54.683373 11176 pending_rounds.cc:85] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 8
I20260709 12:14:54.716274 11176 pending_rounds.cc:107] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:14:54.716893 11176 raft_consensus.cc:2889] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:54.705268 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:54.724457 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.781903 11182 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
W20260709 12:14:54.786460 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 83825.2us
I20260709 12:14:54.779184 11176 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 12:14:54.794873 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:54.794775 11176 pending_rounds.cc:85] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 8
I20260709 12:14:54.796034 11176 pending_rounds.cc:107] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 12:14:54.796382 11176 raft_consensus.cc:2889] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:54.796429 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.747522 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.744931 17034 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.790218 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:54.766256 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:54.716277 17470 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:14:54.799700 11331 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:14:54.752635 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.810618 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:54.797317 11177 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:54.798986 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:54.812628 17498 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:54.812768 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:54.784385 11325 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:14:54.702950 17427 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:54.779198 17488 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.393s	user 0.000s	sys 0.005s
I20260709 12:14:54.813269 17498 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:54.786269 17371 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:14:54.747696 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:54.817948 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:54.819970 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2859) took 1024 ms. Trace:
I20260709 12:14:54.799881 17034 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.808965 17432 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:54.809355 17503 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.846997 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:54.842223 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:54.847807 11256 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 14.
I20260709 12:14:54.838999 11181 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:54.775358 17493 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.814477 17373 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:54.814951 17460 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.850080 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:54.852159 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.853303 11177 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:54.853595 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:54.853967 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:54.854864 11248 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:54.841109 11329 rpcz_store.cc:276] 0709 12:14:53.795591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:54.250260 (+454669us) service_pool.cc:224] Handling call
0709 12:14:54.250566 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:54.250774 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:54.631265 (+380491us) log.cc:844] Serialized 14 byte log entry
0709 12:14:54.819570 (+188305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:54.819633 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:54.819692 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.819767 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:54.819908 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57236,"prepare.queue_time_us":108786,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":59114,"thread_start_us":524452,"threads_started":2}
I20260709 12:14:54.857451 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.858268 11329 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:54.840528 17083 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:54.808821 17275 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:14:54.877106 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 129532us
I20260709 12:14:54.842805 17428 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:54.843605 17395 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:54.885496 11174 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:54.846699 17432 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.892865 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.897126 11181 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:54.893290 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:54.901052 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:54.903556 11177 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:54.904868 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.905719 11324 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:14:54.859535 17434 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:14:54.911643 11252 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:54.846086 17355 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:14:54.885298 17248 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.882839 17494 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:54.902621 17432 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:54.885284 17428 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.915037 17371 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:54.890276 17395 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:54.901393 17438 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:54.918767 11176 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:54.895792 17083 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:54.872416 17460 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.916570 16772 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:54.922365 17434 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:54.856519 17503 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.955309 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:54.966169 11932 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:54.952282 11324 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:54.958719 16285 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.956806 11175 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:54.961316 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:54.962898 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:54.967553 16285 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:54.941162 17428 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:54.953593 17516 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:14:54.973450 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:54.940250 17371 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:54.949975 17395 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:54.969846 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:54.994905 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:54.995654 11330 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:54.929553 17248 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:54.969586 17508 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:14:54.978399 17371 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.979174 17503 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.972913 17373 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:55.010860 17373 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:54.988718 17501 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:14:54.919912 17498 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:54.990984 17493 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:54.976135 11933 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:14:55.019520 17434 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.024215 17392 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:54.965615 16772 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.026146 17480 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:55.026621 17480 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.027571 17480 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.029366 17371 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:55.030895 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2395) took 1157 ms. Trace:
I20260709 12:14:55.031358 11322 rpcz_store.cc:276] 0709 12:14:53.873845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:54.215187 (+341342us) service_pool.cc:224] Handling call
0709 12:14:55.030653 (+815466us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540042a9d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:55.030829 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543723392}
I20260709 12:14:55.032720 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.033357 11322 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
W20260709 12:14:55.005306 17080 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.595s	user 0.013s	sys 0.001s
W20260709 12:14:55.005204 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48445.1us
I20260709 12:14:54.970129 17427 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.060843 11932 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:14:55.006631 17517 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:14:55.007423 17513 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:14:55.031009 11327 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:54.956019 17438 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.070969 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.003542 17292 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:55.028082 16772 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:54.980604 17506 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 12:14:55.066079 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.000280 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.097465 11324 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:14:55.075879 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.078167 11330 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:55.082104 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.080686 17231 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:55.087538 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 129342us
I20260709 12:14:55.075369 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:55.059019 17287 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:55.091369 17494 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:55.104295 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:55.096558 11934 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:55.105773 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.106271 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:55.108032 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:55.120879 17514 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:14:55.122682 17457 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:55.123768 11926 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:55.127080 17438 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.110534 17313 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:55.127080 17248 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.067883 17080 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.095028 17489 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:14:55.144671 11182 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
W20260709 12:14:55.145780 17512 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:14:55.146010 17080 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:54.970556 17083 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.149708 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.150707 11330 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:55.152062 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:55.110970 17231 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.156080 17231 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.161814 17287 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:55.079684 17292 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:55.119048 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:55.118862 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.108314 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.163894 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.169231 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:55.095191 17454 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:55.165257 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55130.7us
I20260709 12:14:55.160941 17432 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.162678 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.170581 11329 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:14:55.170881 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62666.5us
I20260709 12:14:55.175231 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:55.095779 17392 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
W20260709 12:14:55.250758 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 149373us
I20260709 12:14:55.250777 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:55.178180 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:55.184806 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:55.188510 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.263756 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.170295 17292 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:55.174569 17454 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:55.181289 17458 raft_consensus.cc:2764] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:14:55.172456 17489 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:55.174823 17080 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:55.174139 17248 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:55.249025 17231 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:55.274652 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.162832 17287 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.278129 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:55.175042 17428 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.272320 17275 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:55.266286 16772 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:14:55.273285 17457 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:55.295970 16285 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.276888 17454 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:55.312007 16285 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:55.271636 17313 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:14:55.280369 17080 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.314307 16285 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:55.317096 16285 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:55.319962 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 137476us
I20260709 12:14:55.281226 17434 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:55.326145 17080 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:55.327194 17489 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:55.290251 17248 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.383788 17248 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.251784 17494 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.336642 17458 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:14:55.289070 17502 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:14:55.347644 17313 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:55.363860 17480 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:55.370217 17511 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.393s	user 0.000s	sys 0.003s
I20260709 12:14:55.373157 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2867) took 1019 ms. Trace:
I20260709 12:14:55.378234 17432 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.379948 17395 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.389962 16285 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.412253 17313 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.423691 17392 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:55.351327 17340 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:14:55.428683 17502 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:14:55.429234 17515 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.476s	user 0.000s	sys 0.002s
I20260709 12:14:55.414414 11327 rpcz_store.cc:276] 0709 12:14:54.353899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:54.359346 (+  5447us) service_pool.cc:224] Handling call
0709 12:14:55.030792 (+671446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:55.067497 (+ 36705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:55.372833 (+305336us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b540042a9d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:55.373073 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1543402368}
W20260709 12:14:55.441443 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 163436us
I20260709 12:14:55.442252 16285 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.421011 17428 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.420800 17231 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:14:55.410849 17484 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.929s	user 0.003s	sys 0.000s
W20260709 12:14:55.437973 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 259887us
I20260709 12:14:55.448391 11324 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:14:55.441330 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 103526us
I20260709 12:14:55.450078 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:55.455483 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 180887us
I20260709 12:14:55.461417 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.450950 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.441963 17502 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.447502 17432 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.468001 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:55.469619 16285 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:55.455502 17460 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:55.517251 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67271.3us
I20260709 12:14:55.472926 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:55.442055 17392 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.454644 17457 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.464026 17248 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.532727 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:55.534655 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 58685.5us
I20260709 12:14:55.531329 16285 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:14:55.539809 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 71890.1us
I20260709 12:14:55.466977 17371 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:14:55.513954 17434 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:55.543836 17434 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.544761 17540 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:55.545365 17540 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.546361 17540 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.470721 17431 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:14:55.464335 17458 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:14:55.534713 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 61927.7us
I20260709 12:14:55.557116 11325 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:14:55.564177 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.566439 11174 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:14:55.537884 17515 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:14:55.466343 17489 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
W20260709 12:14:55.582051 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49391.4us
I20260709 12:14:55.581809 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:55.584568 11179 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:55.508002 17432 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.556857 17534 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:55.596218 16285 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.596884 16285 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.562310 17248 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.597759 17287 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:14:55.605800 17431 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.606055 17493 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:55.610523 17457 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.614270 17395 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:55.614711 17392 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:55.615590 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:55.590394 17536 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:14:55.473592 17313 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.589576 17538 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:55.598402 17371 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:55.631420 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2386) took 2104 ms. Trace:
I20260709 12:14:55.631884 17432 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.632189 17292 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.633816 11322 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:55.629812 17371 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.631899 11323 rpcz_store.cc:276] 0709 12:14:53.526784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.802257 (+275473us) service_pool.cc:224] Handling call
0709 12:14:53.816639 (+ 14382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:53.826335 (+  9696us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:53.826370 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:53.845524 (+ 19154us) log.cc:844] Serialized 52 byte log entry
0709 12:14:53.944676 (+ 99152us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:54.209020 (+264344us) log.cc:844] Serialized 14 byte log entry
0709 12:14:54.228604 (+ 19584us) log.cc:844] Serialized 14 byte log entry
0709 12:14:54.438529 (+209925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.438653 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:54.707777 (+269124us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b1800226ea0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:55.021464 (+313687us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b1800226ea0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:55.592732 (+571268us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b1800226ea0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:55.631156 (+ 38424us) raft_consensus.cc:1685] finished
0709 12:14:55.631182 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:55.631348 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38050,"prepare.queue_time_us":829027,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":23863,"raft.run_cpu_time_us":505,"raft.run_wall_time_us":43389,"spinlock_wait_cycles":61985792,"thread_start_us":491866,"threads_started":3,"wal-append.queue_time_us":384822}
I20260709 12:14:55.623262 17431 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.638492 17371 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.638795 17431 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.645224 17542 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:55.646903 17542 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:55.643442 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44874.2us
W20260709 12:14:55.644124 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:55.649538 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 574196us
I20260709 12:14:55.620744 17470 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.650388 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:55.639232 17458 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:55.669201 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:55.651624 17522 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:14:55.652164 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 545999us
I20260709 12:14:55.670027 11927 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:55.658699 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.654891 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:55.670673 11176 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:14:55.670964 16285 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.671181 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:55.656406 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.622432 17493 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:55.660661 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:55.667584 17470 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.668035 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:55.674943 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.650055 17538 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:55.668756 17395 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:55.675707 11181 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:55.676036 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.687784 17538 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.678102 11928 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:14:55.691099 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:55.683982 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.692081 11329 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:55.617525 17454 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.685822 11323 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:55.686401 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.686661 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:55.686589 17493 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.687383 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.593405 17502 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.641203 17080 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.713898 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:55.703733 11931 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:55.704397 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:55.716099 11934 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:14:55.670141 17313 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:55.705238 11926 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:55.685547 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.674499 17522 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:55.712478 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57677.3us
I20260709 12:14:55.719071 11935 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:55.688735 17489 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:55.700678 17458 raft_consensus.cc:2764] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:14:55.728616 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.729674 11252 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:14:55.727048 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.733269 11250 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:55.734107 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.734906 11256 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
W20260709 12:14:55.735641 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.736699 11177 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:55.726830 17522 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:55.748622 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.718066 17493 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:55.778939 11257 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: []
W20260709 12:14:55.761057 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.737578 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.750077 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.780881 11248 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:55.773254 11179 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:55.750144 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.749295 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.782081 11252 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:14:55.768059 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.770118 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:55.783476 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.772169 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:55.748979 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.785168 11249 raft_consensus.cc:2368] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:14:55.750442 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.717666 17521 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:55.785609 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:55.717927 17498 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:55.729748 17438 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:55.793505 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.720779 17489 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:55.753326 17432 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.739874 17533 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 12:14:55.704156 17534 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:14:55.758780 17494 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:55.757531 17248 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.758220 17428 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:14:55.799645 11178 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:14:55.752280 17395 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:55.758810 17431 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.788970 17292 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:55.800243 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.790154 17504 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:55.827139 11252 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:14:55.800776 17545 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:14:55.760111 17546 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:55.802089 17542 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.802412 17275 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:55.839059 17275 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.840207 17275 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.841307 17275 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:55.765242 17502 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:55.808250 17489 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:55.798377 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.789923 17503 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:14:55.807838 11329 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:55.848786 11179 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:55.808368 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.753767 17080 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.812614 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:55.813285 11331 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:14:55.815876 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.817451 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.817737 11250 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:14:55.818109 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.862648 11249 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:55.862814 11327 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:55.819118 11177 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:55.810320 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.817471 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.831674 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.863940 11928 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:14:55.865139 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.866007 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.866254 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.834102 17431 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:55.868386 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:14:55.868643 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.870224 17431 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.834388 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.870816 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.872120 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.835495 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.836735 17458 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:14:55.873790 11252 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:55.873980 17458 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:55.874459 17034 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:55.875440 17458 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.876698 17498 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:55.877039 17458 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:55.877180 17498 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:55.869284 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 65223.5us
I20260709 12:14:55.865239 17292 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:55.878752 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.879045 17292 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:55.879528 17551 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:14:55.879608 11248 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:55.861965 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.881100 11250 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:55.869985 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.822158 17432 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.805398 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.855757 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:55.878176 17504 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:55.843945 17395 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.818928 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:55.851073 17503 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:55.802872 17064 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:55.865432 17521 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.843483 17454 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.873394 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.870069 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.882339 11250 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:14:55.883848 11253 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:55.883421 11176 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:55.887573 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:55.892236 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.911957 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.903319 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.903527 16285 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:14:55.898365 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.893028 11252 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
W20260709 12:14:55.918087 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:55.918771 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.920804 11254 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:55.920797 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:55.890132 17286 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:55.926172 17286 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.926249 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:55.927453 17286 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.928227 11177 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:55.928696 17286 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:55.883625 17519 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:55.884101 17428 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:14:55.887929 17064 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:55.933259 17064 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.889279 17373 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:14:55.931286 17478 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:55.935246 17478 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.935674 17478 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:55.936082 16772 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:55.936504 16772 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:55.889621 17521 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:55.938537 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.939543 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2408) took 1121 ms. Trace:
I20260709 12:14:55.939937 11324 rpcz_store.cc:276] 0709 12:14:54.818461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.168855 (+350394us) service_pool.cc:224] Handling call
0709 12:14:55.939345 (+770490us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54000e6854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:55.939490 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638245248}
I20260709 12:14:55.941408 11330 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:55.941835 17526 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:14:55.941836 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.886471 17395 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:55.943645 17547 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 12:14:55.948897 17519 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:55.929579 11177 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:14:55.949911 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:55.950343 11173 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:55.950953 16285 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:55.951138 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:55.951334 16285 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.951822 16285 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:55.951992 11253 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:55.952569 11173 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:55.952793 11177 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:55.955220 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.955554 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.884239 17494 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:55.956374 17494 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:55.960376 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:55.960641 11296 connection.cc:256] RPC call timeout handler was delayed by -2.2071482179999293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
I20260709 12:14:55.962519 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:55.962448 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:55.963652 11927 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:55.960790 16285 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:55.961472 11296 connection.cc:256] RPC call timeout handler was delayed by -2.0856482179999603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
W20260709 12:14:55.966621 11296 connection.cc:256] RPC call timeout handler was delayed by -1.9407482179999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
W20260709 12:14:55.967201 11296 connection.cc:256] RPC call timeout handler was delayed by -1.9335482179999417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 12:14:55.881675 17552 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.192s	user 0.003s	sys 0.000s
I20260709 12:14:55.970502 11253 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:14:55.887586 17503 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:55.884110 17432 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:55.979931 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:55.881577 17292 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:55.961848 17458 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:55.995316 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:56.002588 11324 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:55.999125 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:56.000070 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:56.001260 11252 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:56.003278 11256 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:55.967846 17428 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:56.007067 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:56.011615 11250 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:14:56.013265 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:56.014021 11253 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:56.014662 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:56.015848 11326 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:56.015991 11322 raft_consensus.cc:2410] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:14:56.020648 11252 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:14:56.023243 11254 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:14:56.023926 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:56.027444 11254 pending_rounds.cc:85] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 11
I20260709 12:14:56.025208 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:56.026809 11256 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:56.027710 11254 pending_rounds.cc:107] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:14:56.028178 11257 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 15.
I20260709 12:14:56.028510 11254 raft_consensus.cc:2889] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:56.030638 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:56.032956 11327 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:55.890492 17498 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:55.937462 16772 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:56.039113 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:56.040171 11252 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:55.957320 17494 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.010272 17458 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:56.052389 17550 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:56.053828 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 971) took 1013 ms. Trace:
I20260709 12:14:56.053951 17458 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:56.048368 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:56.054272 11933 rpcz_store.cc:276] 0709 12:14:55.040358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.040555 (+   197us) service_pool.cc:224] Handling call
0709 12:14:55.041119 (+   564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:55.639366 (+598247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:55.639400 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:55.639950 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 12:14:55.721858 (+ 81908us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:55.721899 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:55.721992 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:56.053563 (+331571us) raft_consensus.cc:1685] finished
0709 12:14:56.053587 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:56.053760 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":597675,"raft.queue_time_us":597887,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":81519,"threads_started":1,"wal-append.queue_time_us":223375}
I20260709 12:14:56.056414 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:56.057961 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:56.058969 11326 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:14:56.024521 17365 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:56.063983 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:56.065177 17292 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:56.065375 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:56.065901 17292 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:56.066730 17292 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:56.067327 17392 raft_consensus.cc:2764] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:14:56.072113 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:56.074087 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:56.066195 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:56.082831 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.087401 11296 leader_election.cc:400] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:14:56.087960 11296 leader_election.cc:403] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:14:56.065833 17546 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:56.102062 11326 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:56.101986 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:56.020248 17428 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:56.089613 17553 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 12:14:56.080605 17392 raft_consensus.cc:2764] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:14:56.146538 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46675.4us
I20260709 12:14:56.095825 16772 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:14:56.141571 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44134.7us
I20260709 12:14:56.146503 11254 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:14:56.151567 11175 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:14:56.111667 17034 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:56.160897 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:56.147776 17498 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:56.030901 17454 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:14:56.210561 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:56.109522 17534 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:14:56.139704 17519 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:14:56.100684 17275 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:14:56.256055 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:56.261415 11175 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:56.262002 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:56.284332 11327 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:56.110473 17373 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:14:56.159595 17458 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:56.153153 11252 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:14:56.260927 17514 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.006s
I20260709 12:14:56.090886 17503 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.286062 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:56.287256 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:14:56.294713 17549 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:14:56.241516 17034 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:56.242622 17431 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:14:56.317281 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:56.275508 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 173577us
I20260709 12:14:56.331941 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:56.320127 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:56.332650 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:56.333364 11256 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
W20260709 12:14:56.334247 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.334630 11253 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:56.291931 17438 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:56.342448 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:56.346073 11175 raft_consensus.cc:2368] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:14:56.342012 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:56.291929 17458 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.274899 17427 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 12:14:56.350625 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 195454us
I20260709 12:14:56.356927 11252 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:56.338236 17371 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:56.363245 17371 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.363695 17371 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:56.364037 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.359795 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:56.318001 17548 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:56.360415 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:56.357975 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:56.372597 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:56.375628 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.382004 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.335860 17533 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:56.382951 11252 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:56.427094 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.436770 11252 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:56.447086 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:56.448141 11178 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:56.335736 17526 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:14:56.357414 17458 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:56.378930 17548 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.379619 17524 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:56.308260 17546 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:56.426544 17427 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:56.370649 17432 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 12:14:56.385447 17534 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
W20260709 12:14:56.518215 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 166390us
W20260709 12:14:56.520910 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 161169us
W20260709 12:14:56.462666 17559 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.417s	user 0.002s	sys 0.000s
W20260709 12:14:56.525285 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 143364us
W20260709 12:14:56.509752 17557 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.350s	user 0.000s	sys 0.001s
I20260709 12:14:56.534919 11324 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:14:56.535952 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.430289 17542 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:14:56.444638 17519 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.463081 17491 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:14:56.516521 17548 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:56.519310 17460 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:56.521909 17373 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:56.528878 17231 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:56.475063 17454 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 12:14:56.541565 17478 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.006s
I20260709 12:14:56.557413 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:56.521953 17524 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:56.561195 17347 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:56.540961 17432 consensus_queue.cc:1243] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:14:56.535254 17428 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:56.547309 17519 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.571631 17575 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:14:56.571956 17083 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:14:56.553040 17523 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:14:56.558282 17546 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.475605 17034 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.554069 17373 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:14:56.634699 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.634459 11935 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:14:56.639895 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:56.640599 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:56.639430 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 94814.9us
I20260709 12:14:56.642691 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:56.629565 17493 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.839s	user 0.012s	sys 0.000s
I20260709 12:14:56.567533 17478 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.610663 17083 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:14:56.642630 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 215625us
I20260709 12:14:56.576709 17431 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 12:14:56.661736 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 102187us
I20260709 12:14:56.638967 17373 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:56.663240 17564 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
I20260709 12:14:56.671805 17581 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:56.610865 17428 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 12:14:56.677799 17562 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
W20260709 12:14:56.679510 17569 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
W20260709 12:14:56.683733 17489 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.836s	user 0.010s	sys 0.001s
I20260709 12:14:56.683797 17083 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.740698 17083 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:56.684163 17524 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:56.689337 17504 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.805s	user 0.007s	sys 0.004s
I20260709 12:14:56.689704 17493 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:14:56.691840 17502 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.846s	user 0.009s	sys 0.002s
W20260709 12:14:56.691996 17521 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.001s
I20260709 12:14:56.682948 17231 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:56.694228 17563 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 12:14:56.694599 17572 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:14:56.656100 17565 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.344s	user 0.001s	sys 0.001s
W20260709 12:14:56.720206 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 381602us
I20260709 12:14:56.695772 17579 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:56.660480 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:56.709851 17292 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.004s
I20260709 12:14:56.778239 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:56.758435 17504 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:14:56.779249 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 119186us
I20260709 12:14:56.744499 17373 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.768139 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:56.753489 17286 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.824s	user 0.014s	sys 0.002s
I20260709 12:14:56.768446 17493 consensus_queue.cc:1243] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:56.781613 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:56.784421 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.771380 17231 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.792977 11251 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
W20260709 12:14:56.779210 17498 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.002s
I20260709 12:14:56.795336 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.796749 11928 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:56.798492 17576 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.798486 17538 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:56.799111 17576 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:56.799815 17231 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:56.787880 16772 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.000s
I20260709 12:14:56.802026 17585 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 12:14:56.818477 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:56.855033 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.758901 17489 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:56.860045 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64798.6us
I20260709 12:14:56.861140 11932 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:56.783036 17594 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:14:56.877995 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 235366us
I20260709 12:14:56.870803 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:56.801465 17576 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:56.767654 17521 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.882087 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.803668 17581 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:56.885576 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:56.780279 17292 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:56.853967 17572 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:56.865764 17340 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.865830 17231 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:56.867159 17587 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:56.888439 11255 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:56.879668 17573 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:14:56.892800 17571 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
W20260709 12:14:56.883817 17565 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:14:56.883952 17491 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:56.899158 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.901364 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.901820 17594 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:56.902696 17493 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:56.903589 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.905275 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.886135 17581 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:56.906507 17568 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:56.906984 17581 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:56.907639 17599 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:56.908476 17581 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:56.898216 17080 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:56.914199 17080 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.914593 17080 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:56.917760 17478 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:56.896414 17340 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:56.920531 17340 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:56.922211 17522 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.176s	user 0.013s	sys 0.000s
I20260709 12:14:56.922717 17522 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.898572 17587 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.925024 17522 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:56.925868 16772 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.926599 16772 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.927667 11931 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:56.927968 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1116) took 1244 ms. Trace:
I20260709 12:14:56.928409 11930 rpcz_store.cc:276] 0709 12:14:55.683729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.683901 (+   172us) service_pool.cc:224] Handling call
0709 12:14:55.706419 (+ 22518us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:55.766831 (+ 60412us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:14:55.924445 (+157614us) log.cc:844] Serialized 14 byte log entry
0709 12:14:56.075322 (+150877us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:56.366866 (+291544us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b18003808c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a
0709 12:14:56.368190 (+  1324us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b2c000a0068. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:56.922697 (+554507us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b2c000a0068. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:14:56.923222 (+   525us) log.cc:844] Serialized 52 byte log entry
0709 12:14:56.923391 (+   169us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:56.923647 (+   256us) log.cc:844] Serialized 14 byte log entry
0709 12:14:56.923853 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:56.923929 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:56.927659 (+  3730us) raft_consensus.cc:1685] finished
0709 12:14:56.927682 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:56.927899 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59792,"prepare.queue_time_us":231271,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":6090,"raft.queue_time_us":62817,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":34154,"spinlock_wait_cycles":27607936,"thread_start_us":201746,"threads_started":2,"wal-append.queue_time_us":179086}
I20260709 12:14:56.931524 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:56.905757 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:56.932449 11181 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:56.940675 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.957154 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:56.958736 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57209.9us
I20260709 12:14:56.782872 17579 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.963413 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:56.896800 17572 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.888077 17458 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:56.969980 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:56.961122 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:56.978107 11181 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:14:56.978895 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:56.925033 17587 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:56.981958 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2480) took 1161 ms. Trace:
I20260709 12:14:56.940811 17493 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:56.984133 11181 rpcz_store.cc:276] 0709 12:14:55.820242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.854583 (+ 34341us) service_pool.cc:224] Handling call
0709 12:14:56.931290 (+1076707us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:14:56.981890 (+ 50600us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14722944}
I20260709 12:14:56.966176 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:56.985812 11327 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:14:56.946352 17581 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:56.947453 17519 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:56.988387 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:56.989996 17231 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:57.012584 17231 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:57.013096 17231 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:56.990582 17470 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:56.987414 17538 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:14:57.019831 17538 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:57.020390 17538 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:57.021107 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1120) took 1125 ms. Trace:
I20260709 12:14:57.021353 17538 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.022074 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2880) took 2223 ms. Trace:
I20260709 12:14:57.022512 11330 rpcz_store.cc:276] 0709 12:14:54.798406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.252298 (+453892us) service_pool.cc:224] Handling call
0709 12:14:55.941218 (+688920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:55.941913 (+   695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.021857 (+1079944us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54000e6850. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:57.021999 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1583677824}
I20260709 12:14:57.023984 17231 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.025197 17489 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.025328 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:56.962016 17340 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:14:57.026369 17340 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:57.027451 17340 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.021574 11932 rpcz_store.cc:276] 0709 12:14:55.895659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.954986 (+ 59327us) service_pool.cc:224] Handling call
0709 12:14:57.020898 (+1065912us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54005d0054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:57.021018 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952279808}
I20260709 12:14:56.962082 17568 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:57.024370 17587 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.946242 17599 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:56.998116 17586 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.039290 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.041687 11248 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:14:57.042340 11252 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:57.037233 11256 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:14:56.935513 17491 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.048956 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.049316 11257 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:57.052248 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.053009 11926 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:57.026400 17355 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:14:57.040359 17571 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:14:55.837962 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:56.986958 17594 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:14:57.064540 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.059134 11173 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:57.060627 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:57.066854 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.067744 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:57.068518 11250 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:57.046308 17493 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:57.070492 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.078521 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.082254 11324 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 15: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:57.065861 17365 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.087599 11926 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:57.091527 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 978) took 1303 ms. Trace:
I20260709 12:14:57.066064 17347 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:57.081086 17554 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.929s	user 0.002s	sys 0.000s
W20260709 12:14:57.081082 17600 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:14:57.097024 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.065920 17470 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.110034 11252 raft_consensus.cc:1217] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-18.12]   Dedup: 11.9->[13.10-18.12]
I20260709 12:14:57.068673 17355 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:57.111357 11250 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
W20260709 12:14:57.106446 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 147701us
I20260709 12:14:57.114007 11252 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 9
I20260709 12:14:57.116554 11296 leader_election.cc:400] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:14:57.116639 11252 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:14:57.117163 11296 leader_election.cc:403] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:14:57.117389 11252 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:57.098208 11934 rpcz_store.cc:276] 0709 12:14:55.788032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.970284 (+182252us) service_pool.cc:224] Handling call
0709 12:14:55.996120 (+ 25836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:55.996453 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:56.242190 (+245737us) log.cc:844] Serialized 14 byte log entry
0709 12:14:57.090908 (+848718us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b18003c0c40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287
0709 12:14:57.091068 (+   160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:57.091144 (+    76us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:57.091197 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.091273 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:57.091448 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53061,"prepare.queue_time_us":99492,"prepare.run_cpu_time_us":440,"prepare.run_wall_time_us":2772,"raft.queue_time_us":67533,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":19741,"spinlock_wait_cycles":8975744,"thread_start_us":262479,"threads_started":2}
I20260709 12:14:57.087440 17603 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.119527 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.049607 17599 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.088996 17489 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.123646 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:57.053822 17587 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:57.128602 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:57.129470 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.135340 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:57.120844 17493 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:57.062091 17586 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.121050 17568 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.153682 11249 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:57.148214 11177 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:57.126145 17083 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:57.164270 17083 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:57.164784 17083 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:57.168056 17533 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:57.168587 17533 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:57.169291 17548 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.001s
I20260709 12:14:57.169756 17548 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.170248 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.171049 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2898) took 1274 ms. Trace:
I20260709 12:14:57.171378 11182 rpcz_store.cc:276] 0709 12:14:55.897047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.901801 (+  4754us) service_pool.cc:224] Handling call
0709 12:14:57.170094 (+1268293us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:14:57.170985 (+   891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11809024}
I20260709 12:14:57.172097 17548 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:57.172410 17371 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.808s	user 0.010s	sys 0.000s
I20260709 12:14:57.123575 17470 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.172855 17371 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.173708 16285 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.136888 17292 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.174942 17292 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:57.176106 17292 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:57.176605 17292 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:57.175308 17470 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.164999 17080 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.182248 17568 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.184875 17568 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.189585 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.190002 17373 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.190919 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.191378 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2884) took 1622 ms. Trace:
I20260709 12:14:57.191766 11174 rpcz_store.cc:276] 0709 12:14:55.569417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.580433 (+ 11016us) service_pool.cc:224] Handling call
0709 12:14:55.580777 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:55.597988 (+ 17211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:55.598019 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:55.625945 (+ 27926us) log.cc:844] Serialized 26 byte log entry
0709 12:14:55.806757 (+180812us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:55.806800 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:55.806879 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:57.080432 (+1273553us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:57.191039 (+110607us) raft_consensus.cc:1685] finished
0709 12:14:57.191062 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:57.191289 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16816,"raft.queue_time_us":17348,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":18185,"spinlock_wait_cycles":17124736,"thread_start_us":136883,"threads_started":1,"wal-append.queue_time_us":386598}
I20260709 12:14:57.193971 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.195266 17599 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.196738 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.180004 17586 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.197803 17587 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.198132 17292 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.199465 17292 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:57.202147 11253 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:14:57.135577 17489 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.203656 17371 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.204769 17489 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.128253 17603 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.212710 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.213092 17603 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.150820 11932 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:14:57.151434 17610 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:14:57.148732 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 88237.4us
I20260709 12:14:57.152284 17617 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:57.214599 17617 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.215039 17617 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:57.156730 17604 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:14:57.156164 17355 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.111109 17365 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.113168 17347 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.159866 17491 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.161723 17231 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.163455 17572 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.163717 17458 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.221844 16285 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.228644 16285 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.220583 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:57.226511 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.239735 11256 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:57.241178 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.239923 11248 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:57.256989 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:57.258916 16285 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.262615 17347 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.264266 17365 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.268527 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:57.269420 11249 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:57.241914 17478 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:14:57.279484 16285 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.280064 16285 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.239905 17083 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.291181 16285 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:57.284950 17275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.005s	sys 0.005s
I20260709 12:14:57.292461 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:57.293901 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.294471 11249 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:14:57.297017 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:57.297883 17395 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:57.298267 17395 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:57.298655 17395 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:57.298719 17493 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:14:57.278925 17478 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:57.306022 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:57.300274 17275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.006s	sys 0.005s
W20260709 12:14:57.311115 11296 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.554s (SENT) [suppressed 1 similar messages]
I20260709 12:14:57.299589 17617 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.333886 17478 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.335819 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:57.336388 17617 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:14:57.336690 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2894) took 1492 ms (client timeout 1948 ms). Trace:
W20260709 12:14:57.337152 11179 rpcz_store.cc:269] 0709 12:14:55.844470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.851913 (+  7443us) service_pool.cc:224] Handling call
0709 12:14:57.335631 (+1483718us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:14:57.336627 (+   996us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12375296}
I20260709 12:14:57.338596 17083 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.343864 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:57.345314 11179 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:57.347369 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:14:57.346318 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 112939us
I20260709 12:14:57.352360 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.333890 17579 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.354966 17083 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:57.362476 11252 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:57.363657 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:57.357787 17618 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:14:57.356710 17434 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:14:57.375638 11324 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:57.365891 17617 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:57.376876 17617 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:57.377853 17617 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.386610 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 936) took 1593 ms. Trace:
I20260709 12:14:57.387107 11928 rpcz_store.cc:276] 0709 12:14:55.804478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.988249 (+183771us) service_pool.cc:224] Handling call
0709 12:14:56.008813 (+ 20564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:56.797264 (+788451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.386330 (+589066us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b5400650254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:57.386542 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811624704}
I20260709 12:14:57.407406 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.408326 11328 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:57.409310 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.410175 11325 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:57.411396 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.424260 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.421911 11324 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:14:57.436338 11329 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:14:57.445724 16285 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.446539 16285 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.417287 17434 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.417816 17431 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.436334 17458 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.446048 17603 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.458500 16285 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.445209 17572 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.445678 17275 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.452008 17340 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.477938 17340 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.480743 17340 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.421168 17627 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:14:57.482311 11328 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:14:57.457155 17434 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.484786 17434 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.485340 17610 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.485913 17610 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.486193 17395 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.487943 16772 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.459065 17431 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.490471 17431 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.467685 17458 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.474671 17572 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.494506 17267 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:14:57.494952 17267 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:57.495918 17572 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.495879 17267 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.496644 17275 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.498934 17275 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.500298 17080 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:57.500955 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.501308 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:57.501384 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.502043 11323 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:57.502708 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.493317 17458 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.503423 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.504222 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.504896 11253 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:57.497025 17267 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:14:57.510435 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 151522us
I20260709 12:14:57.506213 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:57.535116 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.536111 11928 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:57.544348 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.545290 11325 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:57.471194 17603 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.546397 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.547143 11325 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:57.548159 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.549880 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.550660 11173 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:57.554243 17395 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.524695 17458 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.555351 17458 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.538496 17572 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:57.559233 17572 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.559736 17572 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:57.565647 11322 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:14:57.568011 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:57.568219 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.569702 11173 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:57.548214 17603 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.581069 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.580489 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.581261 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2602) took 1771 ms. Trace:
I20260709 12:14:57.586676 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.573067 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.594192 11251 rpcz_store.cc:276] 0709 12:14:55.809930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.873125 (+ 63195us) service_pool.cc:224] Handling call
0709 12:14:57.581062 (+1707937us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54000f8cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:57.581196 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1986881920}
W20260709 12:14:57.597222 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.598085 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:57.571362 17534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.006s	user 0.010s	sys 0.008s
I20260709 12:14:57.603060 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.600598 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:14:57.605840 17534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.010s	sys 0.008s
I20260709 12:14:57.608208 11322 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:14:57.571301 17639 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:57.612241 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.601234 17603 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:57.609229 17431 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:57.571952 17533 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.604902 17395 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:57.605237 17458 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.637981 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.638878 11934 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:57.639776 17433 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:57.640301 17433 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:57.640564 17533 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.634287 17431 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.618695 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.641861 17431 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:57.642153 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 981) took 1544 ms. Trace:
I20260709 12:14:57.642939 17603 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:57.643330 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.643693 17639 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.644691 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:57.645373 17618 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.645965 17603 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:57.646242 17639 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.646092 17618 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.646874 17603 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:57.647336 17603 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:57.647759 17603 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:57.649471 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.611481 17617 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:57.651759 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:14:57.638741 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2453) took 1891 ms (client timeout 2155 ms). Trace:
I20260709 12:14:57.642562 11935 rpcz_store.cc:276] 0709 12:14:56.098367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.384839 (+286472us) service_pool.cc:224] Handling call
0709 12:14:56.454802 (+ 69963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:56.635524 (+180722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.641896 (+1006372us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54006c5054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:57.642092 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412092800}
W20260709 12:14:57.653076 11255 rpcz_store.cc:269] 0709 12:14:55.746847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.750160 (+  3313us) service_pool.cc:224] Handling call
0709 12:14:57.638524 (+1888364us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400277254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:57.638672 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2535530368}
I20260709 12:14:57.655586 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.656055 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.656106 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.656867 11934 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:14:57.653198 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:57.656523 11927 raft_consensus.cc:2410] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 12:14:57.649748 17618 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.658906 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.658516 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.641301 17433 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.660830 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.663911 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:57.664414 11178 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:57.664932 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:57.672756 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.654390 17498 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.657750 17395 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:57.664507 11257 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:14:57.672329 17632 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:57.676288 17627 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:57.669847 17585 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:57.665859 17458 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:57.670821 17534 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:57.685913 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.688167 11928 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:57.693867 11322 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:57.696861 17586 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.691582 17458 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:57.697466 17586 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.698130 17458 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.695015 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:57.701041 17639 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:57.701545 17639 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:57.702502 17639 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.702899 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.703526 11324 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:57.682147 17498 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.704411 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.705071 11324 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:57.706110 17498 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:57.671970 17579 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:14:57.706957 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.707217 17579 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:57.707459 11329 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:14:57.707762 17579 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:57.708387 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.708720 17579 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.709349 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.691912 17521 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.710492 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.711084 11329 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:57.711599 17433 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:57.685392 17585 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.711933 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.712409 17585 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:57.712867 11329 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:57.713701 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.714412 11329 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:14:57.684454 17083 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:14:57.715898 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.721617 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:57.723650 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.724892 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.728227 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.722592 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:57.730369 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:57.730973 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.732306 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.741302 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.727841 17615 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:14:57.744550 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.766362 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:57.767385 11935 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:57.745347 11322 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
W20260709 12:14:57.768168 17560 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.560s	user 0.000s	sys 0.010s
I20260709 12:14:57.768923 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:57.771862 17431 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.772579 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:57.774086 11328 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:14:57.778661 17618 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:57.779152 17618 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:57.779536 17618 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:57.782229 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.789515 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.789943 11323 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:14:57.790366 11257 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:14:57.791213 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:57.773303 11928 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:14:57.793663 11255 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:14:57.794917 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.795743 11179 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:14:57.798605 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.798596 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.799697 11255 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:14:57.730576 17532 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:57.804436 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.802507 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.804957 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:57.804391 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:57.803503 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:57.805876 11178 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:57.804901 17534 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:57.805004 17532 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.809733 17532 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:57.810524 17603 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.812552 17617 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:57.814816 17521 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.815140 11322 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:14:57.815461 17521 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.736371 17607 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:14:57.817024 17607 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:14:57.817109 17603 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.737200 17080 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:57.739193 17579 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:57.719501 17493 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:14:57.829528 17617 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:14:57.824019 17080 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.830582 17080 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:57.831188 17534 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:14:57.831602 17534 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:57.832476 17534 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.830054 17617 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.834015 17617 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:57.836042 17607 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.837301 17607 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:14:57.822000 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.818576 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:57.821054 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2482) took 2000 ms. Trace:
I20260709 12:14:57.851266 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.850973 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.852067 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.859014 11253 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:14:57.825078 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:57.820704 17248 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.818666 17627 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:57.727828 17591 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:14:57.861109 11257 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:57.870754 11249 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:57.887166 11900 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:57.874383 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 52473.8us
I20260709 12:14:57.891206 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:57.882200 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.879868 16285 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:14:57.883244 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:57.852313 11176 rpcz_store.cc:276] 0709 12:14:55.820765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.929701 (+108936us) service_pool.cc:224] Handling call
0709 12:14:55.934163 (+  4462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:55.939918 (+  5755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:55.939949 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:55.940442 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:14:56.001413 (+ 60971us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:56.001457 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:56.001526 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:57.767298 (+1765772us) spinlock_profiling.cc:243] Waited 8.22 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:57.796962 (+ 29664us) raft_consensus.cc:1685] finished
0709 12:14:57.796988 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:57.820951 (+ 23963us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5222,"raft.queue_time_us":5768,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":18895232,"thread_start_us":60578,"threads_started":1,"wal-append.queue_time_us":160137}
W20260709 12:14:57.898718 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46255.1us
I20260709 12:14:57.899646 11250 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:14:57.903932 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:57.909188 16285 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:57.909765 16285 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:57.930886 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.846403 17521 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.933691 16285 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.934175 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:57.905491 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:57.940218 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.851227 17533 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:57.947558 11250 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:14:57.941159 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.898203 17593 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:14:57.946475 17599 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:14:57.941056 11252 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:14:57.879797 17520 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:57.881428 17581 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:57.906978 17523 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:14:57.882441 17603 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:14:57.910341 17522 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:57.952666 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.846199 17572 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.957513 11249 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:14:57.965958 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:57.885774 17248 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.973454 17585 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.974458 11250 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:57.974879 17248 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.841574 17617 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:57.978420 11328 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:14:57.952795 17587 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:57.910828 17355 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:14:57.954713 17520 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:57.953523 17431 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:57.977478 17617 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:57.970968 17599 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.000029 11325 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:14:57.964634 17627 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 12:14:58.045924 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.970562 17581 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:57.998471 17521 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
W20260709 12:14:58.050748 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 126845us
I20260709 12:14:58.029325 17587 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:14:58.054533 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 180923us
I20260709 12:14:58.055008 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:58.030948 17520 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:58.059060 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 111295us
I20260709 12:14:58.064597 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:58.066174 17572 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.042697 17533 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:14:58.054288 17431 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.038220 17599 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.057950 17395 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:58.093758 11257 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:14:58.064061 17632 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:58.081087 17524 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:58.099493 17652 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.101428 17652 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:58.102413 17641 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:14:58.104571 17652 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.099618 17593 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
W20260709 12:14:58.128644 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61899.9us
I20260709 12:14:58.096236 17632 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:58.128793 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:58.084740 17431 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.129611 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:58.097838 17646 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:58.133121 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:58.137722 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:58.131096 17632 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:58.130564 17618 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.147308 17618 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:58.148566 17643 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:14:58.131341 17587 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:58.144340 17642 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:14:58.154983 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.162875 11331 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:58.166296 17599 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.159067 17658 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:58.169334 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2901) took 1409 ms. Trace:
I20260709 12:14:58.169538 17431 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.169396 17581 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.171036 17660 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:58.173316 17660 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:58.173736 17660 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:58.177316 11927 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:14:58.169863 11326 rpcz_store.cc:276] 0709 12:14:56.759606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.762317 (+  2711us) service_pool.cc:224] Handling call
0709 12:14:56.762651 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:56.773932 (+ 11281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:56.773984 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:56.785468 (+ 11484us) log.cc:844] Serialized 26 byte log entry
0709 12:14:57.398865 (+613397us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b18003d3d20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:14:57.444118 (+ 45253us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:57.454884 (+ 10766us) log.cc:844] Serialized 14 byte log entry
0709 12:14:57.455237 (+   353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.455339 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:58.167205 (+711866us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b18003d3d20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:14:58.169116 (+  1911us) raft_consensus.cc:1685] finished
0709 12:14:58.169138 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.169271 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10802,"prepare.queue_time_us":12381,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":31153,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":79653,"spinlock_wait_cycles":31211008,"thread_start_us":45015,"threads_started":1,"wal-append.queue_time_us":669833}
I20260709 12:14:58.180301 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:14:58.274513 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 141457us
I20260709 12:14:58.139034 17646 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.322118 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:14:58.322518 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45131.7us
I20260709 12:14:58.323315 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:58.274122 17661 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:58.348954 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:58.321830 17657 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:58.367928 17663 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:14:58.371495 17646 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:58.376993 17656 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:14:58.382675 17532 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.572s	user 0.012s	sys 0.001s
I20260709 12:14:58.302807 17658 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.384191 17658 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.387063 17658 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.305826 17647 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:58.424039 17080 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 12:14:58.423784 17647 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.424712 17080 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.383294 17532 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:58.481714 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 344096us
I20260709 12:14:58.387668 17520 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.490707 17542 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
W20260709 12:14:58.497342 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 343075us
I20260709 12:14:58.485136 17618 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:14:58.499191 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.429569 17080 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.491621 17658 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:14:58.487651 17646 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:14:58.517150 17647 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.518297 17532 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.519682 11331 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:14:58.524942 17520 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.529295 17660 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.512810 17658 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:58.530505 17607 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.539219 17607 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:58.540760 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 188236us
W20260709 12:14:58.558319 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.538126 17658 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:58.538348 17520 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:58.593657 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:58.597177 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52940.1us
I20260709 12:14:58.598495 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:58.543947 17428 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:14:58.554484 17458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.020s	sys 0.037s
W20260709 12:14:58.596777 17648 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.000s	sys 0.033s
I20260709 12:14:58.549790 17572 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.570099 17647 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:58.550972 17664 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:58.619297 17589 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.596772 17431 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:58.591148 17581 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:58.686756 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 88353.9us
W20260709 12:14:58.675833 17458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.024s	sys 0.037s
I20260709 12:14:58.622747 17520 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.698643 17581 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:58.680131 17648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.000s	sys 0.037s
I20260709 12:14:58.670969 17392 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.695693 17520 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.678356 17587 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:14:58.714828 17080 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:58.684288 17572 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.671716 17428 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.687759 17647 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.690649 17589 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.697908 17431 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.758443 17607 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.760530 17647 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.761894 17677 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.690377 17664 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.762557 17677 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.764636 17664 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.765048 17677 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.767484 17589 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.767863 17572 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.770934 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:58.771541 17431 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.772995 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.710011 17392 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.779332 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:58.773793 11331 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:58.781329 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:58.783077 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.710772 17665 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:14:58.787875 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:58.756951 17646 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:58.782058 17581 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.799016 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:58.796455 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:58.799542 17428 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.802318 11331 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:14:58.804121 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 115138us
I20260709 12:14:58.788971 17521 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.805114 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:58.806211 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:58.807147 11178 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:14:58.808768 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:14:58.805728 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 75547.2us
I20260709 12:14:58.834482 17674 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:58.837533 17520 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:58.809178 17587 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:58.800455 17596 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:58.841727 17520 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.797439 17646 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.846529 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:58.842447 17596 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.847429 16879 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:58.847834 16879 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:58.852633 11180 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
W20260709 12:14:58.854483 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 121430us
W20260709 12:14:58.808697 17632 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.000s
I20260709 12:14:58.853955 16879 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:58.848515 17521 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.847856 17646 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.838868 17658 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.862687 17392 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.863191 17658 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.863967 11182 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:14:58.843313 17660 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.864928 17632 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.846582 17520 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.844213 17428 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:58.867205 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.867623 16879 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.868364 17596 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.868553 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.854645 17665 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.870553 16879 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:58.871032 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2433) took 1398 ms. Trace:
I20260709 12:14:58.870729 17665 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.871516 11325 rpcz_store.cc:276] 0709 12:14:57.472436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.550303 (+ 77867us) service_pool.cc:224] Handling call
0709 12:14:57.550595 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:58.034216 (+483621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.870801 (+836585us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b540026c5d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:58.870974 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032926592}
I20260709 12:14:58.874559 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:58.842199 17587 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:58.875344 11324 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:58.876219 17587 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.867398 17632 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.841259 17674 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:58.865394 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.881472 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.882179 11329 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:58.882762 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:58.883605 11323 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:58.883790 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.873193 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.881196 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:58.915146 17427 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.916967 17427 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.919749 17427 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:58.914825 17644 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.797s	user 0.005s	sys 0.008s
I20260709 12:14:58.914740 17669 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.923358 17460 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:58.923062 17669 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.923998 17460 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:58.886459 17674 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:58.925202 17460 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.900703 11325 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:58.905961 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:58.927712 17460 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:58.931236 17662 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:14:58.946372 17662 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:58.933818 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:58.934903 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:58.884466 11329 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:58.939109 11257 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:14:58.948519 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:14:58.936772 16879 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:58.867456 17428 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:58.940399 17521 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.932703 17575 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:14:58.949435 17679 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:14:58.966593 11931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 980) took 3045 ms (client timeout 2713 ms). Trace:
I20260709 12:14:58.966720 17679 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:58.967108 11931 rpcz_store.cc:269] 0709 12:14:55.920959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.938223 (+ 17264us) service_pool.cc:224] Handling call
0709 12:14:58.966321 (+3028098us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b54000a7154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:58.966482 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2272923392}
W20260709 12:14:58.956379 11296 leader_election.cc:336] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.554s (SENT)
W20260709 12:14:58.969091 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 113594us
I20260709 12:14:58.933573 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:58.942363 17503 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:58.915771 17584 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:14:58.949951 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:14:59.031025 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 81156us
I20260709 12:14:58.948151 17662 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:58.936483 17665 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.971287 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:58.971975 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:58.972785 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.040148 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.041002 11180 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:58.974313 11178 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:14:58.964572 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:59.042526 11257 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:14:58.976425 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:58.976545 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:58.978976 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:58.970412 11325 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:14:58.979540 11935 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:14:59.038702 11255 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:58.959321 11324 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:14:59.047716 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.048516 11324 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:14:59.048941 11255 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:14:58.983500 11323 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:14:59.050541 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:58.991868 17585 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:58.993798 11927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 947) took 1301 ms (client timeout 1554 ms). Trace:
I20260709 12:14:59.053121 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:58.995107 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:58.997566 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:58.998492 11926 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:59.054952 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.055043 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.055716 11926 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:14:58.999915 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:59.003798 11174 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 12:14:59.057188 11323 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:14:59.005820 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.006714 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.007648 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:14:59.016080 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.058813 11177 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:14:59.019161 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 2918) took 1448 ms. Trace:
W20260709 12:14:59.058969 11222 connection.cc:256] RPC call timeout handler was delayed by -2.0330892819999917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:14:59.059161 11322 rpcz_store.cc:276] 0709 12:14:57.570359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.768606 (+198247us) service_pool.cc:224] Handling call
0709 12:14:59.018893 (+1250287us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b54000c67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:59.019054 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104472448}
I20260709 12:14:59.020227 11330 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:14:59.022454 11176 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:14:58.963905 17428 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:58.967285 17679 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:58.953859 16879 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:59.072434 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.073760 11174 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:14:59.024700 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.025800 17493 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:59.022475 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:59.075079 11931 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:59.075335 11327 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:14:59.034499 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.078011 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:59.042657 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:59.079348 11932 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:14:59.044101 11249 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:14:59.080840 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:14:59.054987 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:14:59.062186 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3737535159998515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 12:14:59.053303 11927 rpcz_store.cc:269] 0709 12:14:57.692245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.692547 (+   302us) service_pool.cc:224] Handling call
0709 12:14:58.993547 (+1301000us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400416c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:58.993732 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024277376}
I20260709 12:14:59.082203 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:59.063088 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:14:59.083058 11253 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:14:59.083324 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:59.063706 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.083686 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:59.084209 11178 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:59.034533 17607 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.087045 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:59.090085 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.066768 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.067939 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.056207 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.065112 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:59.072381 11181 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:14:59.072571 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:58.963552 17669 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:58.989464 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.102161 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.102312 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:59.096544 11935 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:14:59.097720 17682 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:59.111388 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:59.098062 17064 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:59.097911 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:59.097774 17659 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:14:59.113158 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.113427 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.113790 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:14:59.099112 17605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.020s	sys 0.027s
I20260709 12:14:59.099468 17267 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:14:59.100127 17661 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:59.102572 11934 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:14:59.126530 11174 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:14:59.127255 17267 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:59.128398 17267 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.110342 17677 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:59.081893 11173 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:14:59.104287 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.072321 11325 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:59.132787 11931 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:59.101285 17575 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:14:59.101095 17431 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:14:59.071575 17645 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:59.098240 17680 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:14:59.084897 17662 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:59.078138 16879 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:59.033078 17674 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.119575 11322 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:14:59.110562 17607 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.115772 17669 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:14:59.122551 17605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.020s	sys 0.028s
I20260709 12:14:59.118530 17533 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:14:59.115772 17659 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:59.105321 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:14:59.219468 11179 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:14:59.145471 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.145231 11253 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:14:59.162325 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.167035 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.100736 17576 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:14:59.170423 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 343157us
I20260709 12:14:59.174166 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.226739 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.114068 11252 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:14:59.181576 11932 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 12:14:59.080771 17584 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:14:59.191154 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.192961 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.232208 11174 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
W20260709 12:14:59.187374 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 91150.2us
W20260709 12:14:59.200698 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79012.1us
I20260709 12:14:59.210891 11322 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:14:59.235232 11257 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:14:59.233526 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:14:59.212512 11327 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:14:59.183310 17575 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:59.092906 17493 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.241542 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:14:59.215086 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:14:59.236480 11325 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:14:59.246750 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2457) took 1169 ms. Trace:
I20260709 12:14:59.247231 11325 rpcz_store.cc:276] 0709 12:14:58.077376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.069558 (+992182us) service_pool.cc:224] Handling call
0709 12:14:59.072166 (+  2608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:59.246493 (+174327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.246688 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.234617 11322 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.222963 17659 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:59.192238 17431 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.316929 11248 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:14:59.320845 11322 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:14:59.322114 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 77819.6us
I20260709 12:14:59.169957 17682 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:59.341807 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 103174us
I20260709 12:14:59.342280 17682 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.342711 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:59.343772 17682 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:14:59.344125 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:59.345264 17682 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:14:59.345901 17596 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:59.346395 17395 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:59.346500 17292 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:59.346884 17596 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:14:59.223389 17645 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:59.346954 17395 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.347196 17678 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:59.348114 17645 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.348920 17678 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.349694 17395 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:59.346166 17682 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:59.351311 17682 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:14:59.345494 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:14:59.351581 17678 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:59.347602 17632 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:59.352677 17632 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.355463 17632 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:59.347201 17292 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.358803 17292 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:59.243482 17575 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:59.248077 17585 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:14:59.248413 17615 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:14:59.366577 11322 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:14:59.308943 17593 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:14:59.378168 11322 pending_rounds.cc:85] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 11
I20260709 12:14:59.370843 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:59.361850 17646 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.235386 17661 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.127s
I20260709 12:14:59.389529 11322 pending_rounds.cc:107] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:14:59.447834 11322 raft_consensus.cc:2889] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:59.454075 17431 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.454196 17679 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.360103 17596 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:14:59.456488 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 85703.3us
W20260709 12:14:59.440258 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 206265us
W20260709 12:14:59.334679 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 236328us
I20260709 12:14:59.360950 17533 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 12:14:59.470021 17699 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:14:59.455423 17685 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:14:59.425467 17585 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:59.425297 17593 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:59.457653 17681 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:59.482110 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:14:59.419169 17347 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:14:59.424525 17615 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.455760 17427 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:14:59.456729 17646 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.473245 17695 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:14:59.497141 17596 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.523897 17679 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.524703 17679 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.526221 17686 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:14:59.512768 17347 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.528831 17615 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.529620 17347 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:14:59.539255 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 179488us
I20260709 12:14:59.542514 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:14:59.534828 17651 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.467s	user 0.008s	sys 0.006s
I20260709 12:14:59.492085 17676 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:14:59.525234 17679 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.601706 17679 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.601987 17585 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.578792 16285 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:14:59.604168 16285 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:59.583884 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2635) took 1704 ms. Trace:
I20260709 12:14:59.587476 17646 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.588388 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2454) took 1524 ms. Trace:
I20260709 12:14:59.588984 11180 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:14:59.606496 11256 rpcz_store.cc:276] 0709 12:14:57.879497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.893924 (+ 14427us) service_pool.cc:224] Handling call
0709 12:14:57.894426 (+   502us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:57.910874 (+ 16448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.910909 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:57.935766 (+ 24857us) log.cc:844] Serialized 52 byte log entry
0709 12:14:58.055797 (+120031us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:58.062499 (+  6702us) log.cc:844] Serialized 14 byte log entry
0709 12:14:58.064259 (+  1760us) log.cc:844] Serialized 14 byte log entry
0709 12:14:58.064498 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.064572 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.014704 (+950132us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18003b70a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240
0709 12:14:59.015484 (+   780us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b18003b70a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240
0709 12:14:59.016764 (+  1280us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b18003b70a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db
0709 12:14:59.479165 (+462401us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:59.583589 (+104424us) raft_consensus.cc:1685] finished
0709 12:14:59.583619 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.583806 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25907,"prepare.queue_time_us":44434,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"raft.queue_time_us":16808,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":20408,"spinlock_wait_cycles":36946816,"thread_start_us":82839,"threads_started":1,"wal-append.queue_time_us":103197}
W20260709 12:14:59.589843 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:59.590643 17687 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:14:59.591266 17701 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:14:59.607055 11329 rpcz_store.cc:276] 0709 12:14:58.064456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.952036 (+887580us) service_pool.cc:224] Handling call
0709 12:14:58.952566 (+   530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:58.953573 (+  1007us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:59.239744 (+286171us) log.cc:844] Serialized 14 byte log entry
0709 12:14:59.479446 (+239702us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:59.487816 (+  8370us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.491144 (+  3328us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:59.491459 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 12:14:59.493375 (+  1916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.493491 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.588124 (+ 94633us) raft_consensus.cc:1685] finished
0709 12:14:59.588148 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.588317 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":406,"prepare.queue_time_us":418167,"prepare.run_cpu_time_us":9438,"prepare.run_wall_time_us":37632,"raft.queue_time_us":2173,"raft.run_cpu_time_us":542,"raft.run_wall_time_us":20309,"spinlock_wait_cycles":50048,"thread_start_us":328541,"threads_started":2,"wal-append.queue_time_us":213732}
I20260709 12:14:59.605224 16285 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.591413 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2437) took 2025 ms. Trace:
I20260709 12:14:59.613143 16285 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:59.612999 11330 rpcz_store.cc:276] 0709 12:14:57.566062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.778620 (+212558us) service_pool.cc:224] Handling call
0709 12:14:59.019975 (+1241355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:59.060370 (+ 40395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.591141 (+530771us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54000c67d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:14:59.591350 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2854085888}
I20260709 12:14:59.564491 17607 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.595959 17502 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:14:59.596871 17608 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:59.529554 17615 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:14:59.599138 17676 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:14:59.626017 17676 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:14:59.627213 17676 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.629732 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2456) took 1555 ms. Trace:
I20260709 12:14:59.614338 17701 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:59.630503 17701 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:59.596978 16879 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:14:59.629967 17654 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.561s	user 0.013s	sys 0.022s
I20260709 12:14:59.592162 17593 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.630295 11331 rpcz_store.cc:276] 0709 12:14:58.074034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.951053 (+877019us) service_pool.cc:224] Handling call
0709 12:14:58.951493 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:58.951890 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:58.951922 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:58.981964 (+ 30042us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.137902 (+155938us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:59.137942 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.138009 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.629421 (+491412us) raft_consensus.cc:1685] finished
0709 12:14:59.629448 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.629659 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":1553,"thread_start_us":155412,"threads_started":1,"wal-append.queue_time_us":278554}
I20260709 12:14:59.633329 17586 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.634987 17674 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:14:59.655001 17493 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.657814 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2639) took 1777 ms. Trace:
I20260709 12:14:59.658273 11254 rpcz_store.cc:276] 0709 12:14:57.880814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.883838 (+  3024us) service_pool.cc:224] Handling call
0709 12:14:57.884310 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:57.906969 (+ 22659us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:57.950917 (+ 43948us) log.cc:844] Serialized 14 byte log entry
0709 12:14:58.066895 (+115978us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:58.067575 (+   680us) log.cc:844] Serialized 52 byte log entry
0709 12:14:58.067724 (+   149us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:58.067945 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 12:14:58.068152 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.068223 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.589271 (+1521048us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:14:59.657555 (+ 68284us) raft_consensus.cc:1685] finished
0709 12:14:59.657576 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.657753 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22443,"prepare.queue_time_us":98335,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":129,"raft.queue_time_us":26033,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":12344,"spinlock_wait_cycles":21727232,"thread_start_us":66866,"threads_started":1,"wal-append.queue_time_us":84230}
I20260709 12:14:59.664469 17659 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.665649 16879 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.666803 11177 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
W20260709 12:14:59.685729 17639 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.980s	user 0.004s	sys 0.031s
I20260709 12:14:59.627241 17682 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:14:59.713798 17639 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.008s	user 0.005s	sys 0.031s
W20260709 12:14:59.729604 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 243791us
W20260709 12:14:59.720152 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 238118us
W20260709 12:14:59.728166 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:59.844215 11323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.056s	sys 0.059s
W20260709 12:14:59.844776 11323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.056s	sys 0.060s
W20260709 12:14:59.860321 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:59.846369 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 113423us
I20260709 12:14:59.864861 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2921) took 2043 ms. Trace:
I20260709 12:14:59.865504 11328 rpcz_store.cc:276] 0709 12:14:57.821215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.845182 (+ 23967us) service_pool.cc:224] Handling call
0709 12:14:57.845477 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.979016 (+133539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.864542 (+1885526us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400285ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:14:59.864788 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305137920}
W20260709 12:14:59.871428 17720 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:14:59.867424 17248 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.878611 17248 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.881132 17659 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.717878 17575 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.841171 17674 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.882769 17674 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.884021 17575 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.884876 17674 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.841029 17522 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:14:59.847105 17710 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:14:59.886026 17522 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:14:59.887137 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:14:59.887814 11324 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:14:59.887416 17522 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.889119 17522 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:14:59.900362 17682 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:14:59.892886 17503 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.858s	user 0.012s	sys 0.002s
I20260709 12:14:59.906788 11933 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:14:59.907406 17498 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.908206 17534 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:14:59.895638 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.909279 11929 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:14:59.907912 17498 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.879135 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:14:59.911343 11931 raft_consensus.cc:2410] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 12:14:59.911929 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48338.9us
I20260709 12:14:59.898710 17248 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.827174 17699 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:14:59.917479 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.916244 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:14:59.916417 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.918588 11175 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:14:59.917929 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:14:59.931679 11328 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:14:59.867466 16879 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:14:59.943710 17716 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:14:59.944414 16879 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:14:59.949365 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:59.950941 16879 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:14:59.937501 11324 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:14:59.966360 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:14:59.938094 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:14:59.910537 17701 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:14:59.978006 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:14:59.976365 17664 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:14:59.975190 17717 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:14:59.978405 16879 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:14:59.995642 11324 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:14:59.996428 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:00.002601 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:00.002889 11328 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:00.003382 11324 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:00.005321 17661 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:00.006345 11330 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:15:00.006765 11330 pending_rounds.cc:85] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:15:00.007042 11330 pending_rounds.cc:107] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:15:00.007282 11330 raft_consensus.cc:2889] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:59.998181 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.008577 17717 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:00.008898 11325 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:14:59.999311 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.010004 11331 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:00.009593 17717 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.008591 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:00.015465 11324 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:15:00.015981 11328 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:14:59.995849 17664 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:00.018582 17664 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.018934 17701 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:14:59.993203 17659 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:00.030081 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1153) took 1228 ms. Trace:
I20260709 12:15:00.030722 17645 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.030560 11323 rpcz_store.cc:276] 0709 12:14:58.801696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.059250 (+257554us) service_pool.cc:224] Handling call
0709 12:14:59.059650 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:59.099669 (+ 40019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:59.099706 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:59.100264 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.845357 (+745093us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:59.845478 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.845586 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:00.029794 (+184208us) raft_consensus.cc:1685] finished
0709 12:15:00.029817 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.030008 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39381,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":744582,"threads_started":1,"wal-append.queue_time_us":919626}
I20260709 12:15:00.031526 17645 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.046216 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.051254 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.051769 11323 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:00.052050 11932 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:00.059351 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.060328 17080 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:00.061162 17080 consensus_queue.cc:1243] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:00.061847 16285 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:00.061287 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.069751 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:00.060348 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.074664 16285 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:00.083693 16285 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.084851 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:00.070887 17672 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:00.085484 17646 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:00.109169 17645 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:00.111225 11931 raft_consensus.cc:2410] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:15:00.081740 17632 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:00.111795 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.115093 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.116008 11323 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:15:00.097707 17659 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:00.118829 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.130553 16285 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:00.105701 17639 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:00.105156 17672 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:00.140004 17083 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:00.147729 11329 pending_rounds.cc:85] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 8
I20260709 12:15:00.148301 11329 pending_rounds.cc:107] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:15:00.148617 11329 raft_consensus.cc:2889] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:00.126844 17646 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:00.127641 17659 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:00.160697 17618 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.161304 17618 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.166335 17618 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:00.161366 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1017) took 1569 ms. Trace:
I20260709 12:15:00.170461 11182 rpcz_store.cc:276] 0709 12:14:58.591999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.592121 (+   122us) service_pool.cc:224] Handling call
0709 12:14:58.592429 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:58.864405 (+271976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.161073 (+1296668us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54000d9b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:00.161300 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624039296}
I20260709 12:15:00.174785 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.143509 17672 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.168318 11932 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:00.151839 17083 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.121825 17632 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.175778 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:00.176263 11934 raft_consensus.cc:2410] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
W20260709 12:15:00.159149 17713 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.257s	user 0.005s	sys 0.000s
I20260709 12:15:00.179006 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.179952 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.187290 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.180109 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.181272 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.181329 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.188395 11182 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:00.188745 11929 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:00.189810 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.158205 17659 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.181883 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.190145 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:00.192183 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.183037 11928 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.192821 11175 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:00.145179 17639 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.180473 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.184048 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.184171 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:00.184911 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:00.191277 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:00.190986 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.197085 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:00.197105 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.197499 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.182703 17083 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:00.195068 11927 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:00.199250 11929 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:00.204304 17632 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:00.201244 11175 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 15: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:00.210009 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:00.213514 17691 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.740s	user 0.014s	sys 0.000s
I20260709 12:15:00.213841 17434 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:00.214264 17434 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:00.215319 17034 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:00.215720 17034 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:00.215190 17434 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.213282 11928 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:00.216545 17034 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.222970 11927 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:00.225705 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.226392 17639 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:00.210273 17646 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.229122 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.231222 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.207283 17610 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:00.187793 17618 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:15:00.232342 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.232841 17618 consensus_queue.cc:1243] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:15:00.232559 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:00.233306 17592 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:00.233493 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.233786 17592 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:00.234320 11930 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:00.234701 17592 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.235832 17592 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:00.237289 11327 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:15:00.238276 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.199397 17714 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:00.245453 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.249600 17610 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:00.250108 17610 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.250455 17610 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:00.251016 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 984) took 3454 ms. Trace:
I20260709 12:15:00.251483 11933 rpcz_store.cc:276] 0709 12:14:56.793238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.793401 (+   163us) service_pool.cc:224] Handling call
0709 12:14:56.793773 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:59.907209 (+3113436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.240008 (+332799us) spinlock_profiling.cc:243] Waited 3.11 s on lock 0x7b54005afd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:00.247663 (+  7655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7158393088}
I20260709 12:15:00.254094 17546 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.254673 17546 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.257123 17546 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:00.260591 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.020s	sys 0.033s
W20260709 12:15:00.261027 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.021s	sys 0.033s
W20260709 12:15:00.266619 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.266845 11323 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:00.267784 17034 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:00.268214 17632 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:00.268729 17632 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.269147 17632 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:00.268831 17083 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.270079 11179 raft_consensus.cc:2368] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15.
I20260709 12:15:00.257818 17672 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.274848 11173 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:15:00.246177 17714 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.276198 17714 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:00.276764 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 936) took 1214 ms. Trace:
I20260709 12:15:00.277171 11250 rpcz_store.cc:276] 0709 12:14:59.062425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.466707 (+404282us) service_pool.cc:224] Handling call
0709 12:14:59.480931 (+ 14224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:59.481146 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:59.488124 (+  6978us) log.cc:844] Serialized 14 byte log entry
0709 12:15:00.276399 (+788275us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.276461 (+    62us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:00.276509 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.276581 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.276700 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":579500,"prepare.queue_time_us":7122,"prepare.run_cpu_time_us":159,"prepare.run_wall_time_us":9255,"thread_start_us":208374,"threads_started":1,"wal-append.queue_time_us":205585}
W20260709 12:15:00.279315 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:00.279865 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.281356 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:00.282927 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 708349us
I20260709 12:15:00.283246 11328 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:00.283253 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.283707 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:00.285358 11253 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:15:00.286170 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:00.287385 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.291445 11182 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:15:00.292001 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.292927 11176 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:00.287781 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.276023 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:00.306746 11328 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 12:15:00.306928 11181 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.294200 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.295253 11252 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:00.296187 11257 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:00.273545 17083 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:00.299451 17672 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:00.302879 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 935) took 1268 ms. Trace:
I20260709 12:15:00.316148 11250 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:00.304179 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.317590 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.318169 11254 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.310596 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:00.319051 11328 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:00.320441 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.290726 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:00.327720 11255 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:00.332214 11255 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:00.326432 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:00.339337 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.345863 11328 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:15:00.309707 11175 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:00.311748 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:00.350591 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:00.351985 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.352473 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.352687 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.352851 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.312692 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.354408 11256 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:00.313081 11182 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:00.357774 11180 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:00.315459 11249 rpcz_store.cc:276] 0709 12:14:59.034164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.414368 (+380204us) service_pool.cc:224] Handling call
0709 12:14:59.420151 (+  5783us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:59.697436 (+277285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:59.697474 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:59.698212 (+   738us) log.cc:844] Serialized 52 byte log entry
0709 12:14:59.848398 (+150186us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:14:59.898467 (+ 50069us) log.cc:844] Serialized 14 byte log entry
0709 12:15:00.302518 (+404051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.302598 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:00.302640 (+    42us) raft_consensus.cc:1685] finished
0709 12:15:00.302654 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.302804 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":403739,"prepare.queue_time_us":25795,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":146903,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":20255,"thread_start_us":426518,"threads_started":2,"wal-append.queue_time_us":281426}
I20260709 12:15:00.364180 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.365281 11254 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:00.365281 11181 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:00.353377 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.368481 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.368976 17749 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:00.313704 17546 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:00.382616 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.357846 17674 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:00.357542 17593 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:00.393882 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.394753 11254 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:00.393096 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.396647 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.397017 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.366704 17678 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:00.401013 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.403574 11328 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:00.408063 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.379007 17576 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:00.409204 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 125582us
I20260709 12:15:00.387794 17534 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:00.414227 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.410056 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.415105 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.411187 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.411192 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.411936 11326 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:00.416085 11249 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 13.
I20260709 12:15:00.416210 11248 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:00.381924 17722 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.413017 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.415879 11325 raft_consensus.cc:2368] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20.
I20260709 12:15:00.416255 11326 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:00.411870 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.370641 17522 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:15:00.422372 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.422598 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.399698 17593 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.423583 11256 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:00.422997 11329 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 13.
I20260709 12:15:00.425072 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.428242 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.428951 11250 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:00.420414 11326 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.430284 11326 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:00.431912 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.434194 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:00.437002 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:00.496948 11325 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:00.428988 17534 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:00.439952 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:00.415143 17607 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:00.417601 17576 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.453864 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:00.454049 17744 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 12:15:00.436741 17502 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:00.438882 17706 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:15:00.456292 17682 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:00.456310 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.462071 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:00.508158 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76395.6us
I20260709 12:15:00.509083 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.462884 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.463600 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.464311 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.510236 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.471463 17659 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.474099 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.485980 17676 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.428773 17593 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:00.434832 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.514477 11328 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:00.499426 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 65320.3us
I20260709 12:15:00.500649 11329 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:00.499950 17749 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:00.502861 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.512698 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.518133 11254 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:00.494870 17674 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:00.526196 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.531481 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.532390 11249 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:00.503363 17607 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.506973 17431 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:00.427907 17722 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.546753 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:00.511962 17646 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:00.559724 17646 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.560282 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2925) took 1397 ms. Trace:
I20260709 12:15:00.560719 11177 rpcz_store.cc:276] 0709 12:14:59.163137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.165873 (+  2736us) service_pool.cc:224] Handling call
0709 12:14:59.166373 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:59.679129 (+512756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.559983 (+880854us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:00.560193 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150232832}
I20260709 12:15:00.562160 17646 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:00.562585 17733 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.430s	user 0.001s	sys 0.002s
I20260709 12:15:00.563134 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.564342 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.564829 17659 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:00.521416 17522 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:00.529778 17523 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:15:00.566931 11932 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 20. Ops: []
I20260709 12:15:00.557859 17434 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:00.567788 17695 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:00.570155 17608 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:00.570569 17608 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:00.570940 17608 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:00.574043 17534 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.549391 17726 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:00.575179 17596 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:00.558147 17747 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:15:00.558461 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.560094 11326 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:15:00.579449 11326 pending_rounds.cc:85] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 11
I20260709 12:15:00.579712 11326 pending_rounds.cc:107] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:15:00.579777 11257 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:00.579962 11326 raft_consensus.cc:2889] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:00.554800 17576 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:00.539006 17701 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:15:00.582598 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.518041 17749 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.565009 11250 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 13: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:00.549216 11329 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:00.552774 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:00.553707 11248 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:00.597921 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.522205 17674 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:00.582870 11252 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 14.
I20260709 12:15:00.605072 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:00.550742 17607 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:00.597790 11325 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:00.497193 17610 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.538240 17725 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:00.606341 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53661.2us
I20260709 12:15:00.624670 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:00.613809 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.615370 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.550125 17431 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:00.616583 17722 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:00.583076 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 147691us
I20260709 12:15:00.580803 17546 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:00.629694 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:00.591773 17652 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:00.627645 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:00.652339 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:00.641919 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.645108 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.657366 11329 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
W20260709 12:15:00.645931 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 136920us
I20260709 12:15:00.646888 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:00.651273 11257 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:15:00.581094 17575 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:00.659536 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:00.605482 17538 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:00.648109 17431 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:00.663573 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:00.668695 11926 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.665083 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:00.604116 17498 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:00.667600 11257 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:15:00.676306 11257 pending_rounds.cc:85] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 10
I20260709 12:15:00.676800 11257 pending_rounds.cc:107] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:15:00.677136 11257 raft_consensus.cc:2889] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:00.622236 17576 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:00.587802 17714 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.691890 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:00.690683 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63161.3us
I20260709 12:15:00.636250 17083 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:00.635811 17542 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:00.618630 17674 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:00.703013 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:00.704080 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 79476.3us
I20260709 12:15:00.711302 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.649641 17522 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:00.703409 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51189.6us
I20260709 12:15:00.721398 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.707041 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:00.667659 17652 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.729991 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.725325 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:00.718924 11926 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:00.664089 17546 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.694398 17584 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:15:00.741694 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.743518 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:00.744405 17692 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:00.745095 17756 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:00.745882 17692 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:00.747498 11223 leader_election.cc:400] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20.
I20260709 12:15:00.747874 11223 leader_election.cc:403] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:00.748464 17701 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20.
I20260709 12:15:00.750759 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:00.751860 17701 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:00.754030 17756 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:00.755141 17701 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:00.755780 17701 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:00.756780 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.759594 17701 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.758543 11182 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:00.766073 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.767412 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:00.769532 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77780.7us
I20260709 12:15:00.711766 17726 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:00.712018 17725 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:00.728710 17538 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:00.764523 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.771675 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:00.708166 17498 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 12:15:00.739148 17750 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:15:00.728511 17576 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:00.782560 11173 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:00.773113 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:00.684077 17575 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.769856 11177 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:00.790529 11181 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:00.793773 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 134377us
W20260709 12:15:00.785859 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78984us
I20260709 12:15:00.719871 17542 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.796459 11179 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:00.797168 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:00.803510 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:00.806912 11181 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:00.728706 17632 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:00.780216 17706 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:00.811288 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.785216 17248 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:00.787348 17591 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:00.728204 17522 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.828640 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:00.833125 11180 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:00.712730 17083 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:00.790131 17757 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:00.831642 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:00.847023 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.811363 17749 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:00.848591 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:00.816220 17706 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.872040 17706 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:00.898749 17575 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:00.899833 17610 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:00.860949 17757 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:00.914918 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 283478us
W20260709 12:15:00.934104 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 85623.4us
I20260709 12:15:00.937278 17498 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:00.937711 17752 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:00.908203 11329 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:00.934128 17664 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:15:00.940131 17719 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:01.001685 17034 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:00.888698 17680 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:01.044181 17431 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:00.933367 17758 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:15:00.864799 17749 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.009963 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:01.108733 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 308886us
I20260709 12:15:01.103539 17680 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.115330 17749 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:01.116595 17680 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:01.117946 17596 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.004s
I20260709 12:15:01.118789 11253 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:01.121031 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.122260 17522 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.092401 17714 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:01.123999 17762 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:15:01.113308 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 103437us
I20260709 12:15:01.126128 17083 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:01.026436 17725 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:00.899899 17765 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:15:00.996784 11248 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:01.087306 11173 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:01.089237 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.090305 17617 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:01.094833 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.096288 17726 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:15:01.079903 17729 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:15:01.130036 17617 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.131449 17726 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:01.132413 17692 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:01.132834 17692 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:01.132907 17617 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:01.091627 17538 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:01.133952 17692 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.134757 17538 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.136631 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:01.137979 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:01.138844 17542 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.141409 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:01.143329 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 346244us
I20260709 12:15:01.041329 17607 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:00.906937 17751 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 12:15:01.127533 11926 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:01.148415 11325 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:15:01.130311 17596 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:01.146972 11934 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:01.092074 17652 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.150337 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:01.100104 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 163387us
I20260709 12:15:01.151692 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.151815 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:01.152249 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:01.153087 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:01.149907 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:01.150209 17676 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
I20260709 12:15:01.154050 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.156016 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.161178 11934 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:15:01.097453 17064 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:01.154130 11928 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:01.164776 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.111683 17546 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:15:01.165688 11327 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:01.168556 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:01.167032 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.140472 17632 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:01.101182 17034 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.141069 17576 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:01.147646 17680 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:01.147418 17248 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:01.159391 17608 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.001s
I20260709 12:15:01.112260 17763 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:15:01.170042 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:01.180387 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:01.181092 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:01.182403 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:01.150396 17596 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:01.185195 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2496) took 1019 ms. Trace:
I20260709 12:15:01.197006 11323 rpcz_store.cc:276] 0709 12:15:00.166123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.167475 (+  1352us) service_pool.cc:224] Handling call
0709 12:15:01.184965 (+1017490us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b54004295d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:01.185114 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180134784}
I20260709 12:15:01.197211 17596 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.204314 17596 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:01.207223 11928 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:01.173951 17776 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:01.219597 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51127.8us
I20260709 12:15:01.223690 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:01.225430 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 61308.7us
I20260709 12:15:01.189334 17478 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:01.196095 17434 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.001s
W20260709 12:15:01.253033 17593 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.003s
W20260709 12:15:01.256459 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 75346.7us
I20260709 12:15:01.172394 17064 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.220593 17768 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:01.263283 11174 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:01.214574 17664 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.269658 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:01.275051 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:01.252112 17661 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:01.283942 17431 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:01.284734 17431 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:01.286152 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:01.288882 17498 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:01.289811 17498 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:01.289661 17680 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:15:01.290238 17498 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:01.290657 17498 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:01.298986 17034 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.299175 17248 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.300938 17695 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:15:01.301734 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.302088 17064 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.258648 17591 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:01.195705 17608 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.299521 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:01.224691 17519 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:15:01.301342 17695 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:01.305811 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.266199 17605 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:01.263357 17725 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:01.323750 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.219002 17774 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:15:01.258426 17542 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:15:01.257768 17478 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.331800 11250 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:01.303190 17661 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.260567 17593 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.355615 17593 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:01.355930 17718 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.453s	user 0.006s	sys 0.000s
I20260709 12:15:01.356443 11929 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:15:01.305469 17706 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.321247 17768 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:01.358469 17768 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.362923 17608 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:01.367175 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2476) took 1905 ms. Trace:
W20260709 12:15:01.368559 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.371579 11326 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:15:01.375689 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2522) took 1023 ms. Trace:
I20260709 12:15:01.376241 11174 rpcz_store.cc:276] 0709 12:15:00.351880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.352451 (+   571us) service_pool.cc:224] Handling call
0709 12:15:01.375414 (+1022963us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54000d9654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:01.375606 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092010368}
I20260709 12:15:01.379446 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:01.310225 17664 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.388046 11327 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:01.367703 11322 rpcz_store.cc:276] 0709 12:14:59.461543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.461995 (+   452us) service_pool.cc:224] Handling call
0709 12:14:59.462356 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:59.462775 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:59.462813 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:59.463308 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.730636 (+267328us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:59.730709 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.730843 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:01.355174 (+1624331us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:01.366576 (+ 11402us) raft_consensus.cc:1685] finished
0709 12:15:01.366611 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.367090 (+   479us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3100,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":2116,"spinlock_wait_cycles":6116096,"thread_start_us":266897,"threads_started":1,"wal-append.queue_time_us":439466}
I20260709 12:15:01.392168 17064 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:01.392616 17064 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:01.343089 17661 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:01.334728 17725 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.461879 11257 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:15:01.405217 17522 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:01.464319 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 189365us
W20260709 12:15:01.492172 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.492767 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:01.495641 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 272025us
I20260709 12:15:01.508324 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:01.416947 17695 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.514709 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:01.507340 17756 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:15:01.544793 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.511591 17034 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.495803 17593 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:15:01.494534 17522 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.509531 17585 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:15:01.546795 17695 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:01.567948 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:01.583003 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.519271 17680 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:15:01.539260 17776 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
W20260709 12:15:01.626298 17783 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:15:01.624684 17478 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.633483 17589 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:01.634009 17589 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:01.555785 17034 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:01.652230 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:01.636561 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.637863 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:01.558758 17756 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:01.723340 17617 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.742147 17725 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:01.742465 17790 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:15:01.758940 17791 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:15:01.792383 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 955) took 1610 ms. Trace:
I20260709 12:15:01.795859 11932 rpcz_store.cc:276] 0709 12:15:00.182352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.182540 (+   188us) service_pool.cc:224] Handling call
0709 12:15:00.566702 (+384162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.567432 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.787933 (+1220501us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54001be4d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:01.792305 (+  4372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882655488}
I20260709 12:15:01.732041 17752 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:01.789258 17781 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:15:01.740254 17585 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:01.790026 17664 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:01.828569 17792 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:15:01.817098 17725 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.781004 17617 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.833711 17725 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.829123 17756 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.826253 17752 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.852880 17753 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:01.853526 17753 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:01.854867 17794 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.232s	user 0.002s	sys 0.000s
I20260709 12:15:01.858771 17694 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:01.859217 17694 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:01.860096 17694 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:01.861146 17793 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.303s	user 0.002s	sys 0.000s
I20260709 12:15:01.864771 17725 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:01.866012 17788 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.227s	user 0.000s	sys 0.004s
I20260709 12:15:01.861202 17694 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:01.866032 17752 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:01.845374 17760 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.009s	user 0.009s	sys 0.008s
I20260709 12:15:01.871510 17753 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:01.911074 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:01.911339 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:01.915252 11182 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:01.929764 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2680) took 1458 ms. Trace:
W20260709 12:15:01.930883 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.930516 11252 rpcz_store.cc:276] 0709 12:15:00.470760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.606447 (+135687us) service_pool.cc:224] Handling call
0709 12:15:00.606877 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:00.609164 (+  2287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.609211 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:00.619983 (+ 10772us) log.cc:844] Serialized 26 byte log entry
0709 12:15:01.006229 (+386246us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:01.006291 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.006386 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:01.857018 (+850632us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b18004c61e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:01.928856 (+ 71838us) raft_consensus.cc:1685] finished
0709 12:15:01.928902 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.929424 (+   522us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38397,"raft.queue_time_us":630,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":18674,"spinlock_wait_cycles":22611712,"thread_start_us":248033,"threads_started":1,"wal-append.queue_time_us":272387}
I20260709 12:15:01.941186 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:01.942694 11325 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:15:01.945721 11322 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:15:01.942494 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 304562us
W20260709 12:15:01.954356 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 462905us
I20260709 12:15:01.956194 17034 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:01.958319 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 453091us
I20260709 12:15:01.962013 16879 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.963059 16879 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.965130 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:01.966079 11257 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:15:01.966228 11322 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:15:01.966830 16879 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.971318 16879 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.972301 16879 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:01.978344 17248 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.979365 17248 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.981974 17248 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:01.982823 17248 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.983644 17248 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:01.989059 17340 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:01.992992 11325 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:01.999030 17355 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:02.000276 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.000231 17355 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.003489 17355 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:02.006776 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.016198 17692 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:02.017838 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65368.6us
I20260709 12:15:02.018980 17519 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.020012 17519 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.024609 17752 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:15:02.047820 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.060647 17617 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.063519 16879 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.067605 17519 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:02.074936 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 113966us
I20260709 12:15:02.088838 17706 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.092886 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2696) took 1500 ms. Trace:
W20260709 12:15:02.095992 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.106210 17803 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.107317 17803 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:02.112162 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.112550 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.093492 11248 rpcz_store.cc:276] 0709 12:15:00.591790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.132138 (+540348us) service_pool.cc:224] Handling call
0709 12:15:01.132456 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.198422 (+ 65966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.198457 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:01.198498 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:01.198527 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.092273 (+893746us) spinlock_profiling.cc:243] Waited 65.8 ms on lock 0x7b54002a6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:02.092362 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.092650 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151251840}
I20260709 12:15:02.125592 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.127383 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2948) took 1219 ms. Trace:
I20260709 12:15:02.127498 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:02.128087 11324 rpcz_store.cc:276] 0709 12:15:00.907727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.908355 (+   628us) service_pool.cc:224] Handling call
0709 12:15:00.908701 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.117994 (+209293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.118043 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:01.118102 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:01.118141 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.126833 (+1008692us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540042aed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:02.126891 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.127158 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480701312}
I20260709 12:15:02.131265 11254 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:02.125749 11178 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:02.134321 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.135280 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:02.141500 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.134075 17248 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.133437 17519 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.153363 17519 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:02.156984 17596 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.952s	user 0.008s	sys 0.016s
I20260709 12:15:02.162396 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:02.163435 17692 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:02.163661 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.172353 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:02.173451 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.174436 11182 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:02.175443 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:02.175309 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:02.176313 11249 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:02.176764 11179 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:02.178124 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:02.179021 11180 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:02.176597 17519 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.182883 17522 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:02.174446 11174 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:02.192207 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:02.193158 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.193821 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:02.194098 11929 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:15:02.193055 17589 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.195277 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.195778 17589 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:02.196005 11927 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:02.204244 11929 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:15:02.204596 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.207131 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:02.207271 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:02.208225 11927 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:02.193147 11176 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:02.211021 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:02.211777 11255 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:02.211995 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.196022 11175 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:02.212668 11178 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:02.213485 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:02.214459 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.193975 11173 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:02.215430 11248 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:02.216473 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.218772 11174 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:02.218969 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.219942 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.226936 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.227487 17617 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.227878 11254 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:02.228514 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.228428 17617 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:02.230574 17661 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.808s	user 0.010s	sys 0.013s
I20260709 12:15:02.231178 17661 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.232290 11175 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:02.233289 17682 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:02.217260 11173 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:02.235004 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:02.250267 11932 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:02.252625 11248 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:02.254987 17753 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:02.256197 17753 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.258038 17632 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.259032 17632 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.263571 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.280184 11330 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
W20260709 12:15:02.281658 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95139us
W20260709 12:15:02.310421 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82300.7us
I20260709 12:15:02.272667 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:02.337937 11182 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:15:02.335070 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.339434 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.234325 17661 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.297008 17523 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.285795 17692 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:15:02.328694 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 103467us
I20260709 12:15:02.341363 11173 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:02.341686 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:02.343729 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.357679 17627 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.358691 17627 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.360054 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.357851 17617 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:02.361536 17627 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.375562 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:02.377527 17782 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:02.378104 17782 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.378508 17782 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:02.381824 17607 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.383065 17763 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.383985 17763 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:02.385771 17498 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.044s	user 0.008s	sys 0.017s
I20260709 12:15:02.386396 17498 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:02.388260 17767 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.372s	user 0.000s	sys 0.007s
I20260709 12:15:02.389194 17498 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.391016 17593 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:02.392055 17593 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:02.392397 17593 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:02.394237 17753 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:02.393728 17593 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.398514 11328 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:02.399120 17593 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:02.400149 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.401063 11326 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:02.402156 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:02.402529 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.402921 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:02.404639 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.408666 11182 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:02.408531 17519 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:02.409946 17522 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:02.410382 17522 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:02.411355 17782 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.411691 17522 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.357779 11251 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:15:02.412544 17763 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.189039 17811 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.156s
I20260709 12:15:02.353734 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.417399 11176 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:02.428265 11180 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:02.413805 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.417918 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:02.452612 11248 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:02.461476 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.413576 17782 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:02.462901 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.465721 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.467922 11254 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:02.418542 17632 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:02.424402 17692 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:02.484381 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.483083 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:02.488736 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.496135 11249 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:02.479344 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.358577 17523 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:02.501703 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49838.9us
W20260709 12:15:02.503923 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.506704 17523 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.504190 11932 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:02.509294 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.531109 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.494136 17753 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.498162 17498 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:02.501060 17782 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:02.536773 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.537511 11248 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:02.497244 17756 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.549211 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:02.544325 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 126511us
I20260709 12:15:02.536218 17699 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.549340 11180 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:02.553036 17608 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.540524 17589 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.574937 17680 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.575866 17680 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.576292 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:02.577204 17756 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.577965 17617 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:15:02.578524 17813 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 12:15:02.578663 17617 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:02.579173 17692 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.578693 17753 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.580120 17694 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:15:02.580971 17823 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:02.581930 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.577766 17699 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.591548 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:02.574826 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:02.577472 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:02.604012 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49990.1us
I20260709 12:15:02.591456 11180 raft_consensus.cc:606] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.567562 17782 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.611692 17782 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.613621 17699 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.614202 17608 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.615478 17617 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.616611 17608 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.617272 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:02.618628 17753 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.589903 17589 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.623980 17589 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.623860 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:02.614535 17680 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.626521 17680 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:02.626971 17680 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.627404 17680 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:02.618574 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.630371 11180 raft_consensus.cc:614] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:02.631498 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.625782 17756 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.632464 11254 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:15:02.633826 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.632491 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.642758 11251 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:02.613508 11179 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:02.645924 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:02.650295 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72945.5us
I20260709 12:15:02.654542 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:02.677699 11255 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
W20260709 12:15:02.680763 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 124203us
I20260709 12:15:02.679891 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:02.660185 17608 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:02.712087 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57633.6us
I20260709 12:15:02.712248 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.717371 11251 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:02.692960 17823 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:02.720906 17766 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:02.721756 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:02.726028 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 166460us
I20260709 12:15:02.680474 17753 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.728371 17522 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:02.655566 17498 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:15:02.730928 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.718521 11255 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:02.717442 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:15:02.740006 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:02.745922 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:02.748728 17796 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.117s	user 0.004s	sys 0.000s
I20260709 12:15:02.750837 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:02.752671 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.753677 17627 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:02.754222 17627 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:02.755311 17627 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.760265 17753 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.760810 11248 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:02.762439 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 2941) took 1953 ms. Trace:
I20260709 12:15:02.762874 11181 rpcz_store.cc:276] 0709 12:15:00.809328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.812117 (+  2789us) service_pool.cc:224] Handling call
0709 12:15:00.812426 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.832000 (+ 19574us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:01.005810 (+173810us) log.cc:844] Serialized 14 byte log entry
0709 12:15:01.152863 (+147053us) log.cc:844] Serialized 14 byte log entry
0709 12:15:01.217915 (+ 65052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:01.217985 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:01.218050 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.295999 (+ 77949us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b1800234e20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:01.847768 (+551769us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b1800234e20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3eca5c058f 00007f3ec35ff571 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd
0709 12:15:01.849916 (+  2148us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b1800234e20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:02.761915 (+911999us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b54001f45d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:02.762023 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.762233 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":619321,"prepare.queue_time_us":280065,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":6570,"raft.queue_time_us":73071,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":19122,"spinlock_wait_cycles":76659584,"thread_start_us":304978,"threads_started":2,"wal-append.queue_time_us":255196,"wal-append.run_cpu_time_us":1883,"wal-append.run_wall_time_us":1092585}
I20260709 12:15:02.761473 11329 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:02.761490 17787 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:02.780205 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.708850 17756 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.812742 11254 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:02.815450 11901 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:02.822444 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.825402 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.826303 17803 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.845870 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.855675 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.818547 17756 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:02.860482 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49037.4us
I20260709 12:15:02.847000 17766 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:02.840143 17632 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.840286 17823 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:02.898459 17823 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.892910 17632 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.900884 17823 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:02.901010 17632 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:02.894145 11331 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:02.894631 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:02.895682 17826 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:15:02.895857 17756 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:02.899134 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 100640us
I20260709 12:15:02.869467 17782 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:02.909042 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.908190 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.910898 11901 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.889410 17766 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.915436 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:02.915681 17503 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:02.916178 11323 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:02.920699 17034 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:02.925179 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:02.936393 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:02.937834 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:02.933817 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:02.939527 11252 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:02.940363 11330 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:02.940423 11934 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:02.912540 17498 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:02.921350 17766 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:02.947726 11254 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:02.947650 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:02.960597 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.929477 17782 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:02.965350 17083 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.966033 17782 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.966250 17083 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:02.968874 17083 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:02.960575 11331 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:15:02.971474 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.974133 17753 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:02.976691 17657 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:02.977269 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 90715us
I20260709 12:15:02.977414 17657 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:02.979564 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:02.981148 17657 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:02.963060 11257 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:02.901487 17823 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 12:15:03.005218 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.008363 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:03.013149 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:02.998212 17593 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.596s	user 0.016s	sys 0.000s
I20260709 12:15:03.016994 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:03.027877 17519 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:03.028805 17519 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.029691 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.030479 11328 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:03.032931 11328 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:03.033850 17789 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.034494 17789 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.036764 17674 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:03.036971 17789 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:03.045563 17064 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:03.047812 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2486) took 2761 ms (client timeout 1602 ms). Trace:
I20260709 12:15:03.048202 11256 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 12:15:03.048367 11253 rpcz_store.cc:269] 0709 12:15:00.286556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.287155 (+   599us) service_pool.cc:224] Handling call
0709 12:15:03.047349 (+2760194us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54000f7dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:03.047582 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1910358400}
I20260709 12:15:03.052111 11934 raft_consensus.cc:2410] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:15:03.074363 11254 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:03.011809 17576 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:03.077913 17824 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.542s	user 0.003s	sys 0.000s
I20260709 12:15:03.082417 11252 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
W20260709 12:15:03.080662 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 361378us
I20260709 12:15:03.023739 17632 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:03.085783 17576 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.085899 17632 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.088786 17646 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:03.071558 17519 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.091776 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.025794 17596 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:03.101512 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.115149 11330 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 13.
I20260709 12:15:03.118579 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2692) took 2527 ms. Trace:
I20260709 12:15:03.119221 11256 rpcz_store.cc:276] 0709 12:15:00.590912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.910362 (+319450us) service_pool.cc:224] Handling call
0709 12:15:03.047593 (+2137231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.048620 (+  1027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.118251 (+ 69631us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54000f7dd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:03.118481 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4850535680}
W20260709 12:15:03.119992 17816 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.573s	user 0.000s	sys 0.003s
I20260709 12:15:03.114727 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:03.125722 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.129262 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.130494 11327 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:15:03.139452 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 160031us
I20260709 12:15:03.141374 11328 raft_consensus.cc:2410] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:03.089285 17632 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:03.157281 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:03.088872 17576 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.094936 17753 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:03.170085 17576 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:03.171871 17576 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:03.172389 17831 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:15:03.172533 17576 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:03.173353 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.179616 17347 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:15:03.183122 17680 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.007s
I20260709 12:15:03.183784 17680 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.187772 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:03.190623 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.191461 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2681) took 2720 ms. Trace:
I20260709 12:15:03.171262 11176 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:15:03.196954 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:03.156415 17340 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:03.198211 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.202040 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:03.191910 11250 rpcz_store.cc:276] 0709 12:15:00.471038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.642001 (+170963us) service_pool.cc:224] Handling call
0709 12:15:00.642506 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.334089 (+691583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.163072 (+1828983us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b540010c054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:03.191275 (+ 28203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1584754432}
I20260709 12:15:03.203873 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:03.204136 17608 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:03.204346 17680 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.215372 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.215945 11327 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:03.207465 17608 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.208438 17752 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:03.219033 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:03.230361 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 145541us
I20260709 12:15:03.234875 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:03.231961 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:03.235553 11327 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:03.236609 17340 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:03.237236 17340 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.237762 17340 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:03.240358 17064 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:03.241845 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:03.243685 17787 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:03.248461 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.249199 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:03.250260 17593 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:03.262656 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.276741 17692 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.277580 17692 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.280813 17692 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.232998 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:03.292830 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 43760.4us
I20260709 12:15:03.295737 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:03.299546 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 132313us
I20260709 12:15:03.205348 17753 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.223024 17608 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:03.305207 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72345.5us
I20260709 12:15:03.223800 17752 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:03.315516 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:03.297040 17248 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.016s	sys 0.042s
I20260709 12:15:03.301342 17787 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:03.317193 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.317840 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.319983 11327 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
W20260709 12:15:03.326138 11326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.020s	sys 0.019s
I20260709 12:15:03.301031 17680 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:03.220661 17766 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.319238 17752 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:03.372221 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:03.361474 11326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.021s	sys 0.019s
I20260709 12:15:03.333549 17787 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:03.367266 17807 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.006s	sys 0.042s
W20260709 12:15:03.375129 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 478476us
I20260709 12:15:03.336416 17657 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:15:03.325580 17248 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.016s	sys 0.042s
I20260709 12:15:03.384102 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:03.379526 17657 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:03.377987 17807 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.006s	sys 0.043s
I20260709 12:15:03.335784 17498 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:15:03.386997 17657 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:03.389719 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:03.394699 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 68586.5us
I20260709 12:15:03.376061 17787 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:03.415408 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.404564 17699 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:03.421839 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:03.415180 17632 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:03.428918 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60295.2us
I20260709 12:15:03.414930 17579 raft_consensus.cc:2764] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:15:03.435949 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:03.445597 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:03.429728 17830 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:15:03.499536 11296 leader_election.cc:400] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15.
I20260709 12:15:03.505467 11296 leader_election.cc:403] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:15:03.437193 17632 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:03.498771 17787 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.442636 17782 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:15:03.459507 17832 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:15:03.507606 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74699.8us
I20260709 12:15:03.434576 17800 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:03.496213 17431 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.519348 17800 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:03.519639 17431 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.519793 11076 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 6 to 11, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 11 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:03.467661 17083 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:03.456861 17669 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:15:03.522562 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:03.558665 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56242.2us
I20260709 12:15:03.588797 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:03.467561 11074 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 171) took 17023 ms (client timeout 15000 ms). Trace:
I20260709 12:15:03.594983 17680 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:03.595513 17680 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.595880 17680 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:03.569514 17627 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:15:03.602645 17627 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:03.603132 17627 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:03.589818 11074 rpcz_store.cc:269] 0709 12:14:46.444180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.473330 (+ 29150us) service_pool.cc:224] Handling call
0709 12:14:47.164553 (+691223us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:03.467318 (+16302765us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:53.229749 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:53.230710 (+   961us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:53.230782 (+    72us) write_op.cc:435] Acquired schema lock
0709 12:14:53.230812 (+    30us) tablet.cc:662] Decoding operations
0709 12:14:53.232431 (+  1619us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:53.232576 (+   145us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:53.232627 (+    51us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:14:53.288420 (+ 55793us) tablet.cc:701] Row locks acquired
0709 12:14:53.303918 (+ 15498us) write_op.cc:260] PREPARE: finished
0709 12:14:53.362265 (+ 58347us) write_op.cc:270] Start()
0709 12:14:53.408824 (+ 46559us) write_op.cc:276] Timestamp: P: 1783599293339177 usec, L: 0
0709 12:14:53.422242 (+ 13418us) op_driver.cc:348] REPLICATION: starting
0709 12:14:53.520781 (+ 98539us) log.cc:844] Serialized 5196 byte log entry
0709 12:14:53.966651 (+445870us) op_driver.cc:464] REPLICATION: finished
0709 12:14:54.524844 (+558193us) write_op.cc:301] APPLY: starting
0709 12:14:54.540998 (+ 16154us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:54.711797 (+170799us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:54.711967 (+   170us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:55.157044 (+445077us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:55.157409 (+   365us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:14:55.157480 (+    71us) write_op.cc:312] APPLY: finished
0709 12:14:55.285908 (+128428us) log.cc:844] Serialized 121 byte log entry
0709 12:14:55.442356 (+156448us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:55.456335 (+ 13979us) write_op.cc:454] Released schema lock
0709 12:14:55.472946 (+ 16611us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:55.574481 (+101535us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:55.577733 (+  3252us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":214775,"reported_tablets":32,"spinlock_wait_cycles":396906240,"child_traces":[["op",{"apply.queue_time_us":521423,"apply.run_cpu_time_us":44608,"apply.run_wall_time_us":1026364,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58571,"num_ops":13,"prepare.queue_time_us":152078,"prepare.run_cpu_time_us":28023,"prepare.run_wall_time_us":344546,"raft.queue_time_us":257791,"raft.run_cpu_time_us":23096,"raft.run_wall_time_us":493306,"replication_time_us":544219,"thread_start_us":1040574,"threads_started":5,"wal-append.queue_time_us":227291,"wal-append.run_cpu_time_us":84044,"wal-append.run_wall_time_us":2501569}]]}
I20260709 12:15:03.457053 17768 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:03.561568 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 112967us
I20260709 12:15:03.509407 17632 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:03.608111 17692 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:03.468223 17661 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:15:03.567147 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:03.614305 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47878us
I20260709 12:15:03.614984 17661 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:03.615453 17661 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:03.615854 17661 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:03.515058 17782 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.617854 17782 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:03.618579 17768 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:03.620587 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 98109.9us
I20260709 12:15:03.571898 17431 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:03.624310 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1179) took 1366 ms. Trace:
I20260709 12:15:03.624704 11324 rpcz_store.cc:276] 0709 12:15:02.258240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.260761 (+  2521us) service_pool.cc:224] Handling call
0709 12:15:02.261306 (+   545us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:02.464204 (+202898us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:02.464234 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:02.503368 (+ 39134us) log.cc:844] Serialized 52 byte log entry
0709 12:15:03.008864 (+505496us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:03.008932 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.009037 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:03.624103 (+615066us) raft_consensus.cc:1685] finished
0709 12:15:03.624123 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:03.624247 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33291,"raft.queue_time_us":33440,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":124,"thread_start_us":543156,"threads_started":2,"wal-append.queue_time_us":674643}
I20260709 12:15:03.613430 17680 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.627620 17787 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.572363 11076 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 7 to 10, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 10 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:03.629436 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:03.630292 11176 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:03.574729 17674 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
W20260709 12:15:03.575591 17843 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:15:03.515676 17753 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:03.576210 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2707) took 1216 ms. Trace:
W20260709 12:15:03.576537 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.524870 17682 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.582341 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 2968) took 1140 ms. Trace:
I20260709 12:15:03.604292 17627 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.520851 17800 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.607249 17841 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 14. Current term is 15.
I20260709 12:15:03.617753 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:03.437258 17717 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:03.568890 17579 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.583900 17765 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:03.560837 17805 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.176s
I20260709 12:15:03.578047 17083 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:03.608106 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:03.630766 11076 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 1 to 9. New cstate: current_term: 9 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:03.699419 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:03.646549 17340 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.647148 11927 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 12:15:03.723479 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:03.651093 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:03.652170 11249 rpcz_store.cc:276] 0709 12:15:02.359485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.747553 (+388068us) service_pool.cc:224] Handling call
0709 12:15:02.748044 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:02.748569 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:02.748604 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:02.750990 (+  2386us) log.cc:844] Serialized 26 byte log entry
0709 12:15:03.222125 (+471135us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:03.223551 (+  1426us) log.cc:844] Serialized 14 byte log entry
0709 12:15:03.223780 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.223864 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:03.575976 (+352112us) raft_consensus.cc:1685] finished
0709 12:15:03.576000 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:03.576151 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1143,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":10224,"raft.queue_time_us":1288,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":58294,"thread_start_us":470733,"threads_started":1,"wal-append.queue_time_us":758293}
I20260709 12:15:03.663873 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:03.652637 11326 rpcz_store.cc:276] 0709 12:15:02.441844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.442096 (+   252us) service_pool.cc:224] Handling call
0709 12:15:02.467500 (+ 25404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:02.488233 (+ 20733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:02.488261 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:02.538119 (+ 49858us) log.cc:844] Serialized 26 byte log entry
0709 12:15:02.767411 (+229292us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b180036c5e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:03.374201 (+606790us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:03.374242 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.374322 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:03.576590 (+202268us) raft_consensus.cc:1685] finished
0709 12:15:03.576608 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:03.582259 (+  5651us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4436,"raft.queue_time_us":15986,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":34017,"spinlock_wait_cycles":4412672,"thread_start_us":606588,"threads_started":1,"wal-append.queue_time_us":959237}
I20260709 12:15:03.701905 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:03.734058 11178 raft_consensus.cc:606] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:03.704905 17838 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:15:03.706231 17083 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.707294 17766 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.709076 11253 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:15:03.710917 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:03.717782 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.705444 17579 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.719977 11935 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:03.722229 11248 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:03.665499 17682 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.737780 11176 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:03.722718 11927 pending_rounds.cc:85] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 13
I20260709 12:15:03.691149 17717 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:03.719978 17822 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.211s	user 0.002s	sys 0.002s
W20260709 12:15:03.749694 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 141718us
W20260709 12:15:03.751066 17576 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.578s	user 0.002s	sys 0.010s
W20260709 12:15:03.775292 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 164822us
I20260709 12:15:03.772810 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:03.754702 11927 pending_rounds.cc:107] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 12:15:03.776487 11927 raft_consensus.cc:2889] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:03.777995 17801 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.749547 17800 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:03.783375 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.783703 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.784929 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.786242 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.789968 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.753535 17841 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:03.755149 17674 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:03.759487 17717 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:03.797555 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 146742us
I20260709 12:15:03.798341 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.798906 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:03.799866 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.721951 17657 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.800141 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.802209 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:03.806802 11178 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:03.795158 17841 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.809079 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:03.778787 17627 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:03.815212 11182 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:03.808658 17811 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:03.741077 17431 consensus_queue.cc:1175] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 17 status { last_received { term: 17 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 17 index: 12 } }
I20260709 12:15:03.824299 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:03.827001 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:03.772289 17752 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.789085 17576 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.806936 17680 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.830514 11179 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:15:03.809036 17841 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.824370 16879 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:03.850378 11932 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:03.852156 11182 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:03.847240 17853 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:03.847512 17752 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.848228 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1181) took 1589 ms. Trace:
I20260709 12:15:03.848687 17851 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:03.849058 17608 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.001s
I20260709 12:15:03.807399 17674 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.895028 17579 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.895355 17674 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.897063 17657 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.797190 17801 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.899617 17576 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.872704 11248 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:03.869763 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1173) took 1641 ms. Trace:
I20260709 12:15:03.879554 17585 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:03.880869 17850 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:15:03.885838 17608 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.849620 17753 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:03.900573 11927 rpcz_store.cc:276] 0709 12:15:02.209359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.240961 (+ 31602us) service_pool.cc:224] Handling call
0709 12:15:02.262640 (+ 21679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.777112 (+1514472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.835211 (+ 58099us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400690ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:03.850166 (+ 14955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3183043968}
W20260709 12:15:03.908558 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 102653us
I20260709 12:15:03.870867 11322 rpcz_store.cc:276] 0709 12:15:02.258947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.262125 (+  3178us) service_pool.cc:224] Handling call
0709 12:15:02.262647 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:02.381559 (+118912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:02.381594 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:02.382275 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:15:02.463802 (+ 81527us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:02.463854 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.463935 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:03.719408 (+1255473us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:03.847940 (+128532us) raft_consensus.cc:1685] finished
0709 12:15:03.847961 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:03.848166 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44038,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":142,"spinlock_wait_cycles":57683840,"thread_start_us":80311,"threads_started":1,"wal-append.queue_time_us":125878}
I20260709 12:15:03.925230 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.925542 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:03.929834 17674 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.930390 17674 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.829502 17782 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:03.948256 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.950250 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:03.952637 11254 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:15:03.947780 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:03.954713 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 155890us
I20260709 12:15:03.956216 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.971655 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:03.971953 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.972599 11934 raft_consensus.cc:2410] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:15:03.954842 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:03.974926 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:03.917869 17753 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:03.975752 11928 raft_consensus.cc:2410] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:15:03.982933 11179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:03.985728 11179 raft_consensus.cc:606] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:03.986207 11179 raft_consensus.cc:614] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:03.901861 17617 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:15:03.932816 17853 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:03.982259 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:03.996052 11250 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:03.973874 17680 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:04.015098 17034 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:04.016170 17034 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:04.016573 17034 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:04.017652 17034 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.019106 17034 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:03.989063 17753 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:03.917769 17851 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.022904 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:03.994864 17340 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:03.928696 17585 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:03.994613 17787 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:15:04.027077 17857 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:04.032647 17674 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:04.040395 17596 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:04.092006 17853 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.094237 17851 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.095259 17854 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:04.058609 17632 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.103031 11182 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:04.106530 17617 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:15:04.110018 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.032718 17248 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
W20260709 12:15:04.117125 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 187917us
I20260709 12:15:04.122592 11927 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
W20260709 12:15:04.122417 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 138874us
W20260709 12:15:04.122534 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 84336.6us
I20260709 12:15:04.109205 17661 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.138464 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:04.151970 17585 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.166280 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.171391 11173 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:04.171468 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.173885 17869 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:04.174840 17811 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:04.176141 17811 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:04.176541 17811 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:04.187297 17869 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:04.186126 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51888.4us
I20260709 12:15:04.188973 17723 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:04.188854 17869 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.189699 17723 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.192238 17723 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:04.186029 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.198523 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.200397 17811 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:04.195722 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 172425us
I20260709 12:15:04.208779 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:04.200835 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.202818 17858 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:04.212044 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:04.172552 11181 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:15:04.220741 11174 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:04.144938 17871 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:04.229869 17811 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:04.222756 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:04.231988 11256 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:04.235661 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:04.172181 17608 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.239455 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:04.243523 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:04.235248 17859 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:04.247372 17674 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:04.247824 17674 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.237548 17340 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:04.248240 17674 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:04.253715 17498 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:04.254221 17498 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.254580 17498 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:04.259521 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.260004 17753 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.265115 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:04.274523 17801 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:04.231006 17782 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.286677 11256 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:04.283640 11250 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
W20260709 12:15:04.289584 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 67510.1us
I20260709 12:15:04.291571 17674 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.293398 17498 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.247637 17646 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.221576 17596 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:15:04.283488 17248 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:04.299659 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.245455 17347 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:04.222162 17858 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.296291 17617 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:04.310223 17632 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.310477 17674 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:04.311170 17661 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.311549 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.312224 17874 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:04.312835 17627 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:04.312592 17347 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.288698 17431 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:04.333935 17627 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.334403 17431 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.313519 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.296890 17723 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:04.319257 17841 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.322824 17869 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:04.327482 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 112225us
I20260709 12:15:04.296993 17608 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:04.345831 17632 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:04.331733 17617 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.341647 17787 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:15:04.348600 17787 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:04.348969 17787 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:04.238140 17871 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.339296 11929 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:15:04.341015 17594 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:04.341617 17858 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.341909 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:04.256340 17717 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.342187 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.341038 17627 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.299825 17596 consensus_queue.cc:1243] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:04.344040 17841 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:04.346679 17846 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.771s	user 0.005s	sys 0.000s
I20260709 12:15:04.345952 17608 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.323822 17657 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:04.351330 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.350731 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.359948 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.361505 17787 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.362814 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:04.456233 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 96336.1us
I20260709 12:15:04.363839 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:04.338832 17800 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.543s	user 0.013s	sys 0.000s
I20260709 12:15:04.349921 17593 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.299230 17646 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.453154 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.456537 11929 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
W20260709 12:15:04.460242 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 108962us
I20260709 12:15:04.371655 11175 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:04.345980 17869 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.372401 17585 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.468578 11927 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:15:04.474161 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 110411us
I20260709 12:15:04.474979 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:04.462471 17874 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:04.475622 17585 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.466248 17657 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.477914 17880 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:04.478562 17880 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.480309 17860 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:04.481956 17880 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.471273 17593 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.485648 17646 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.337913 17248 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.353827 17723 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.488137 17248 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.488710 17723 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.473377 17627 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:04.490217 17608 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.465490 17753 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:15:04.491048 17871 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:04.490479 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 144986us
I20260709 12:15:04.492528 17869 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.492977 17498 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.473472 17034 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:04.493276 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.494589 17431 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:04.495213 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.495393 17717 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.497414 17869 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:04.497802 17787 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:04.477903 17585 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:04.498679 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.503111 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.507495 17871 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:04.507666 17431 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:04.513706 11934 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:04.513748 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:04.515365 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:04.525832 17753 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:04.526715 17753 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.529162 17871 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:04.517086 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.508896 17869 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:04.530488 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.541018 17431 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:04.514533 17811 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:15:04.547561 17657 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:04.547968 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.551016 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.554019 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:04.560313 17585 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:04.529479 17753 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.564143 17807 raft_consensus.cc:993] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:04.565210 17807 raft_consensus.cc:1081] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:04.565395 17347 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.566602 17347 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:04.566962 17347 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:04.567342 17347 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:04.569173 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:04.569945 11177 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:04.572304 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.573608 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.574983 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.575541 11330 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:04.578234 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.573664 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.580621 11331 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:15:04.581076 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.582552 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.583377 11326 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:15:04.584487 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.594691 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.595711 11329 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:04.595743 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.594911 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:04.598783 11173 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:15:04.601866 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 266798us
I20260709 12:15:04.602246 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.585517 11326 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 23 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:15:04.606426 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:04.607712 11177 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:04.607059 11331 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:04.608402 11328 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:04.609025 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.609534 11182 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:04.610450 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:04.611315 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 108329us
I20260709 12:15:04.613013 11330 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:04.614199 11327 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:04.611356 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.614900 11331 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:15:04.618346 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:04.618346 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.619645 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.621730 11324 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 14: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:04.622152 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.623449 11326 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:04.622610 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:04.624200 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.626169 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.628510 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.629324 11176 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:15:04.629163 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:04.629690 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.627960 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:04.631994 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:04.633177 11177 pending_rounds.cc:85] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 9
I20260709 12:15:04.633430 11177 pending_rounds.cc:107] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 12:15:04.633594 11177 raft_consensus.cc:2889] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:04.613958 17585 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:15:04.636830 17585 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:04.636721 17723 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:04.630874 17811 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:15:04.637372 17585 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:04.637706 17498 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:04.638259 17498 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:04.638731 17596 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:04.638334 17585 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.640193 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:04.617928 17860 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:04.643652 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.623368 17627 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:04.620936 17034 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.639665 17585 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:04.639207 17498 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.614202 17851 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:15:04.644985 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.647408 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:04.672130 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.634488 17717 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:04.650159 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:04.684952 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48806.4us
I20260709 12:15:04.640988 17593 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:04.640004 17608 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:04.642839 17519 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:04.653079 11322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:15:04.645547 17723 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:04.652917 17646 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:04.649309 17627 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.655371 17800 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:04.691177 11322 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:04.701212 11322 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:04.644224 17248 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:15:04.677490 17853 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.677438 17431 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:04.654348 17682 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.689594 17823 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:04.639240 17596 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.684234 17064 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:04.689870 17717 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:04.691552 17519 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.692116 17811 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:04.678318 17787 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:15:04.701136 17627 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:04.707669 17723 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:04.720400 17853 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.725705 17248 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:04.727602 17591 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:04.725256 17431 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.736568 17723 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.737609 17431 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.734403 17248 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:04.734848 11328 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:15:04.735167 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 105346us
I20260709 12:15:04.710815 17800 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:04.689582 17034 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.724062 17519 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:04.749616 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:04.752640 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58677us
W20260709 12:15:04.751098 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56878.9us
I20260709 12:15:04.757263 11173 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:04.757659 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:04.759754 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 72920.8us
I20260709 12:15:04.760396 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.768603 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:04.769618 11173 raft_consensus.cc:606] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.770164 11173 raft_consensus.cc:614] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:04.775841 17034 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.761864 17805 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:04.778862 17661 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:04.806592 17661 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:04.806861 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.807863 17661 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.809054 11250 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:04.751839 17248 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.733619 17064 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.810572 17248 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.778689 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.761873 17823 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:04.779476 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.779937 17596 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.781132 17853 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.790645 11322 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.773331 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.803761 11249 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:15:04.803894 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54414us
W20260709 12:15:04.818030 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57699.9us
I20260709 12:15:04.804456 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.776871 17800 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.805088 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:04.805795 17805 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.814220 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:04.770138 17891 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:04.821573 11928 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:04.823788 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69423.7us
I20260709 12:15:04.819203 11934 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:04.838711 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.827553 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.833431 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.842931 11322 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:04.843096 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.845278 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.846066 11250 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:04.841070 11253 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:04.847918 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:04.848510 11254 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:04.850478 17893 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.851047 17893 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.851477 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.852751 17064 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.853305 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.847735 17873 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:04.853808 17064 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:04.854198 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.854139 17873 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.855669 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.856433 17873 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.821656 17661 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:04.824800 17717 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.857172 17431 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:04.857849 17596 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:04.861770 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:04.821465 17895 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:04.853340 17893 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.896840 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:04.897519 11250 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:04.900607 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 75111.4us
I20260709 12:15:04.907722 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:04.910068 17895 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:04.911360 17895 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.918915 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:04.918962 11250 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:04.930598 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.930848 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.931640 11250 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
W20260709 12:15:04.815024 17811 consensus_queue.cc:1175] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fecfc03de52a4308bc9b045cad5aac1e" responder_term: 14 status { last_received { term: 12 index: 7 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 13. Current term is 14. Ops: [13.8-13.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:04.961622 17347 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:04.963214 17627 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:04.964243 17347 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:04.948559 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.973089 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:04.973470 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:04.973804 11257 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:04.975380 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:04.976296 17871 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:04.976547 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.977082 17871 raft_consensus.cc:2764] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:15:04.977816 17871 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:04.946358 17805 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:04.981115 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:04.971212 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:04.966440 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.984982 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.985376 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:04.986420 11251 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:15:04.988473 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.989243 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:04.990118 11329 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:15:04.991811 11322 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:15:04.994714 11257 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:05.004215 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.005218 11329 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:05.006225 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.007787 17646 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:05.009723 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.009841 17823 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:05.010409 11330 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:05.010622 17823 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.015470 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.017467 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.019776 11327 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:04.986836 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.040721 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:04.965528 17627 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:05.048561 11250 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:05.049376 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.048595 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:05.050652 11328 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:15:05.054021 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.029229 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.062944 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.063429 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.063923 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:05.065428 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 84483.7us
I20260709 12:15:04.992996 17519 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.076767 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.077831 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:05.056596 17347 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.079805 11927 raft_consensus.cc:2368] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19.
I20260709 12:15:05.083706 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.083619 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:05.063494 17803 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:05.068152 17860 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:05.104097 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:05.112119 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:05.112779 11250 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:05.113024 11254 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:05.068293 17894 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.110716 11328 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:05.117161 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:05.075925 17823 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:05.118165 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:05.118168 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.097165 17682 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:05.123720 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:05.124910 11250 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:05.108359 17347 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:05.153484 17347 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:05.075114 17248 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:15:05.116252 17431 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:05.156165 11933 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:15:05.159246 11933 pending_rounds.cc:85] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 12
I20260709 12:15:05.159585 11933 pending_rounds.cc:107] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:15:05.159914 11933 raft_consensus.cc:2889] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:05.115747 17753 raft_consensus.cc:2764] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:15:05.119982 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:05.119310 17608 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:05.183578 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:05.187173 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69112.3us
I20260709 12:15:05.149765 17498 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:05.125996 17034 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:05.115983 17699 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:15:05.198604 11930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:15:05.112157 17869 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:15:05.202317 11930 raft_consensus.cc:606] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.181051 17823 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:05.175527 17864 raft_consensus.cc:2764] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:15:05.185474 17519 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:05.206564 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:05.206565 17904 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:15:05.214774 17889 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:15:05.227252 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 155413us
I20260709 12:15:05.219162 11930 raft_consensus.cc:614] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:05.158800 17431 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:05.240327 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 120446us
I20260709 12:15:05.194557 17682 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:05.244398 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.245826 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.242660 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.254746 11174 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:05.250150 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.256040 11177 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:05.253897 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61810.9us
I20260709 12:15:05.258857 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.264233 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:05.265175 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 188478us
I20260709 12:15:05.266332 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.272192 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.275156 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.277347 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:05.286036 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.287565 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.289937 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.227288 17823 raft_consensus.cc:2764] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:15:05.287685 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.294294 11254 raft_consensus.cc:2368] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:05.290588 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.291631 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.289700 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:05.307384 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.253250 17431 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:05.314859 11932 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:05.249765 17801 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:05.264178 17869 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:05.356249 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.357117 11173 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:05.364707 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:05.361586 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 178099us
I20260709 12:15:05.366384 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.369858 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:05.371768 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.402678 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.403551 11180 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:05.403625 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.404831 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.318538 17083 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:05.405309 11931 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 12:15:05.365024 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 118419us
W20260709 12:15:05.434708 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 170232us
I20260709 12:15:05.438361 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:05.393667 17888 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:15:05.440151 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.446643 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:05.443835 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.446791 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:05.393854 17498 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:05.455928 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.456962 11180 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
W20260709 12:15:05.477608 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 188136us
I20260709 12:15:05.362033 17801 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.512836 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.430446 17803 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:05.543062 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:05.545130 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:05.438647 17585 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:05.545475 17347 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:05.545933 17585 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:05.546403 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.546820 17585 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.547991 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:05.548276 17912 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:15:05.548588 11257 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:05.538934 11254 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:05.147847 17894 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:05.550915 17906 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:15:05.551187 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.512036 11181 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: [18.10-19.11]
I20260709 12:15:05.552060 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:05.552294 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.552754 11174 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:05.553130 11174 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:05.554592 11176 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:15:05.552891 11175 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:05.553216 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:05.559675 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:05.544128 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 103717us
I20260709 12:15:05.575730 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.579360 11929 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:05.458359 17895 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:05.584422 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:05.371439 17889 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:05.586416 11250 raft_consensus.cc:2368] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:05.595727 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.603276 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.589048 17782 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:15:05.668813 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 109212us
I20260709 12:15:05.632611 11330 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:05.674708 11330 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:05.644039 17801 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:05.646613 17909 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:15:05.541867 17657 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:05.593559 17898 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:15:05.651175 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:05.651593 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.651829 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:05.598841 11174 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:05.653872 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.654788 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:05.689450 11180 raft_consensus.cc:2410] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:15:05.369653 17869 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.545351 17803 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:05.658685 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 97506.7us
I20260709 12:15:05.664312 11250 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:15:05.672694 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:05.673947 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 363160us
I20260709 12:15:05.692461 11251 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 14: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:05.692477 11173 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:15:05.689459 11248 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:05.692593 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.537504 17922 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:05.679560 11257 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:05.630108 11931 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:05.587604 17083 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:05.672848 17811 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.672572 17929 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:05.704535 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:05.705363 11256 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:05.706382 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:05.689334 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 143604us
I20260709 12:15:05.697253 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:05.709728 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.721168 11252 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:15:05.722029 11331 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:05.722445 11331 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:05.724284 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.713879 11330 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:05.701898 17894 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:05.706652 17901 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:05.702572 17895 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:05.739105 17921 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.725778 17801 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.740043 17340 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:05.740458 17903 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:05.709700 17929 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:05.766486 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.743642 17889 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:05.698252 17922 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:05.720605 17902 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:05.735514 11331 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.757738 11178 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:15:05.768492 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 85348.7us
I20260709 12:15:05.769922 11173 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:05.708506 17811 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.766523 17340 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:05.776571 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:05.778640 11331 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:05.782084 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:05.788470 17871 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:05.745119 17617 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:15:05.794492 11178 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:05.794711 11256 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:05.789999 17928 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:05.769675 17929 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:05.745277 17431 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:05.797448 17903 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:05.815997 17903 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:05.816969 17431 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:05.772852 17902 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:05.817981 17902 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:05.818323 17903 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:05.819991 17861 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:05.788872 17340 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.822412 17340 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:05.771680 17922 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:05.809306 11180 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:05.768946 17801 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.815091 17811 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:05.824637 17917 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:15:05.802264 17871 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:05.830365 17801 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:05.831881 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:05.834015 17869 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:05.834509 17083 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.835711 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:05.854131 11178 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:05.855593 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:05.846273 11331 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:15:05.766875 17895 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.766283 17921 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:05.857968 11331 pending_rounds.cc:85] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 13
I20260709 12:15:05.827065 17928 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:05.865357 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:05.862864 11331 pending_rounds.cc:107] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:15:05.867551 11331 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:05.863857 17931 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:05.868839 17931 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:05.869239 17931 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:05.861275 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 153067us
I20260709 12:15:05.847249 17871 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:05.871107 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:05.915251 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.866256 11178 raft_consensus.cc:606] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:05.961563 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 96655.3us
W20260709 12:15:05.964136 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 48976.5us
I20260709 12:15:05.965166 11296 leader_election.cc:400] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:05.965557 11296 leader_election.cc:403] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:05.966741 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.968107 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:05.915663 17871 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:05.973676 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 102667us
W20260709 12:15:06.022420 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.153230 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.154630 11176 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:06.154807 11181 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:06.154642 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:06.156541 11177 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 16: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:06.158432 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 188609us
I20260709 12:15:06.178345 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:05.971912 17935 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:06.179348 11932 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:06.151787 17617 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:06.194999 11175 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:15:06.180202 17933 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:15:06.155946 17869 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:06.203131 17937 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.203763 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:15:06.204087 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.183280 17431 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:06.218109 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.219116 11928 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
W20260709 12:15:06.208745 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 240707us
I20260709 12:15:06.221580 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:06.182251 17853 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.145645 17723 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 12:15:06.245203 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:06.199002 17930 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.235308 17935 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:06.255216 17617 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:06.198618 17932 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.286s
I20260709 12:15:06.278873 11296 leader_election.cc:400] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
W20260709 12:15:06.281370 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 106608us
I20260709 12:15:06.280841 11296 leader_election.cc:403] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:15:06.250730 17864 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:15:06.227761 17937 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.262499 17940 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:06.291080 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:06.292186 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1197) took 1514 ms. Trace:
I20260709 12:15:06.296706 11927 rpcz_store.cc:276] 0709 12:15:04.769948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.770199 (+   251us) service_pool.cc:224] Handling call
0709 12:15:06.284253 (+1514054us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540069fad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:06.284398 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612899200}
I20260709 12:15:06.246236 17853 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:06.303339 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.270952 17930 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.315297 17941 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:06.207237 17928 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:06.329810 17938 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.410s	user 0.001s	sys 0.000s
I20260709 12:15:06.336093 17921 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.285185 17869 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:06.338244 17576 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.283908 17945 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:15:06.339036 17576 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.342592 17853 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.301038 17918 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.343750 17918 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:06.352648 17340 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.530s	user 0.004s	sys 0.008s
I20260709 12:15:06.354390 17617 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:06.357661 17945 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:06.357062 17895 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.366734 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:06.356796 17889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.052s	sys 0.057s
I20260709 12:15:06.374488 11251 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:06.365604 17902 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.002s
I20260709 12:15:06.362358 17617 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.364048 17576 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.365026 17945 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:06.377364 17889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.052s	sys 0.058s
I20260709 12:15:06.386490 17576 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.388057 17576 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.389664 11252 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:06.390403 17576 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.389667 17431 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:06.391149 17431 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:06.398443 11933 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: []
W20260709 12:15:06.399999 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 109005us
I20260709 12:15:06.401175 17889 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.401829 17889 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.403239 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:06.405790 17869 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:06.411427 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.379567 17902 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.380524 17617 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.421423 11249 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:06.420280 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.423947 11252 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:06.420447 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.425781 17951 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.426367 17951 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.427145 17617 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.427736 17617 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:06.428697 17951 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.435727 11330 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:06.436197 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:06.435658 17931 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.566s	user 0.005s	sys 0.005s
I20260709 12:15:06.438823 17871 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.439517 17931 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.441020 17871 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.441565 17931 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.420147 17940 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:06.445912 17940 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:06.447016 17940 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.411273 17918 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.460179 17954 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:06.463342 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 2989) took 1222 ms. Trace:
W20260709 12:15:06.463691 17922 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.001s
I20260709 12:15:06.463824 11182 rpcz_store.cc:276] 0709 12:15:05.241306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.241437 (+   131us) service_pool.cc:224] Handling call
0709 12:15:06.463104 (+1221667us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:06.463245 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2601079552}
I20260709 12:15:06.464200 17922 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.465340 11248 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:06.468324 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.469533 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:06.471110 11330 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:15:06.471630 11330 pending_rounds.cc:85] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:15:06.472016 11330 pending_rounds.cc:107] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:15:06.472340 11330 raft_consensus.cc:2889] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:06.473671 11182 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:15:06.475181 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.476063 11252 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:06.476120 17936 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:06.476691 11179 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:06.476997 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.477793 11173 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:06.481400 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.476773 17936 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.473588 17954 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:06.486011 17936 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:06.494006 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.473161 17871 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:06.499138 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:06.504002 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.497076 17617 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.507561 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:06.500255 17931 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.508106 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.503432 17954 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:06.422150 17945 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.512183 11927 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:06.509796 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:06.513233 11928 raft_consensus.cc:2368] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:06.513381 17936 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:06.515616 17936 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.517949 17936 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.518427 17902 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.519640 17945 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:06.525826 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:15:06.517390 17929 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.669s	user 0.006s	sys 0.006s
I20260709 12:15:06.530105 17929 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.532511 17929 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.506430 17803 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.542754 11330 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:06.508497 17871 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:06.562201 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:06.563145 11322 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:06.553551 17922 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.566255 17929 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:06.512233 17931 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:06.579198 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70050.7us
I20260709 12:15:06.586642 11330 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:06.581297 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.580238 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.552860 17864 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:06.513381 17617 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.584004 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 968) took 1268 ms. Trace:
I20260709 12:15:06.548200 17936 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:15:06.598829 11933 rpcz_store.cc:276] 0709 12:15:05.315552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.315749 (+   197us) service_pool.cc:224] Handling call
0709 12:15:05.358203 (+ 42454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:06.399834 (+1041631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.583721 (+183887us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400415fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:06.583942 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391073536}
I20260709 12:15:06.603473 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:06.585492 17964 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:15:06.634006 17902 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.634912 17936 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.634721 17902 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.635465 17936 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.636989 17902 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.638414 17936 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.572506 17889 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.599032 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.640647 11329 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:06.640913 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:06.596098 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.581462 17871 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:06.643628 17871 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.570362 17954 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:15:06.646137 17954 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.646749 17954 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.608338 17941 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:06.599987 17864 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.624735 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:06.654145 17864 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.623786 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:06.624730 17596 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.659031 17596 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.659706 17753 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.660339 17753 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.627044 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.594139 17431 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.664896 17431 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.665302 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:06.613437 17917 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.668041 17871 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.668829 17917 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.669203 17871 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:06.670186 17854 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:06.670697 17854 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.671077 17854 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:06.671303 17917 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:06.664960 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.630254 11930 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:06.673986 17657 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:06.632756 17248 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.674602 17657 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.675305 17248 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:06.676461 17907 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.170s	user 0.009s	sys 0.005s
I20260709 12:15:06.549300 17803 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.677800 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1204) took 1850 ms. Trace:
I20260709 12:15:06.679013 17519 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:06.679420 17803 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.680119 17248 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.654578 17945 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:06.680894 17945 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.682960 17930 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.686249 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:06.687119 11928 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:06.688038 17657 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.639485 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.688759 11251 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:15:06.678310 11934 rpcz_store.cc:276] 0709 12:15:04.832317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.832448 (+   131us) service_pool.cc:224] Handling call
0709 12:15:04.832803 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.846300 (+ 13497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.846351 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:04.846961 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:15:04.953194 (+106233us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b4800041658. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:15:05.099904 (+146710us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:05.471621 (+371717us) log.cc:844] Serialized 14 byte log entry
0709 12:15:05.539417 (+ 67796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.539502 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:06.531216 (+991714us) spinlock_profiling.cc:243] Waited 5.88 ms on lock 0x7b180031f780. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:06.618569 (+ 87353us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b180031f780. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db
0709 12:15:06.675845 (+ 57276us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:06.677478 (+  1633us) raft_consensus.cc:1685] finished
0709 12:15:06.677510 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:06.677713 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84851,"prepare.queue_time_us":582954,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":97908,"raft.run_cpu_time_us":2480,"raft.run_wall_time_us":102029,"spinlock_wait_cycles":103457792,"thread_start_us":563948,"threads_started":2,"wal-append.queue_time_us":518001}
I20260709 12:15:06.691375 11322 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:06.689738 17931 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.692587 11175 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:06.629719 17617 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.641993 11324 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:06.700412 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.702064 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.703583 17596 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.704012 11174 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:06.706071 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.706892 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.707116 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.707502 11324 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:06.707767 11928 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:06.708489 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.709316 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.711014 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.711613 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.711615 11173 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:06.670369 17871 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.714075 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:06.716995 11174 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:06.717593 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2545) took 1165 ms. Trace:
I20260709 12:15:06.717053 11181 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
W20260709 12:15:06.717595 17787 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:15:06.718216 11174 rpcz_store.cc:276] 0709 12:15:05.551706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.551827 (+   121us) service_pool.cc:224] Handling call
0709 12:15:06.528370 (+976543us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b1800315180. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6727646 00007f3ec666158e 00007f3ec66e2f9e 00007f3ec66e47de 00007f3ec66f027c 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c
0709 12:15:06.703777 (+175407us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b580078e6d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66480aa 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66f027c 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c
0709 12:15:06.717533 (+ 13756us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446299776}
I20260709 12:15:06.727494 11251 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:06.728670 17753 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.729666 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.529392 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:06.737250 17945 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.738241 17937 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:06.679673 17519 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.740087 17954 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:06.742094 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.743158 11322 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:06.770222 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.774273 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.781267 11932 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:06.778489 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.782617 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.778479 11296 leader_election.cc:400] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:06.773002 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.784710 11296 leader_election.cc:403] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:15:06.784971 11254 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:06.786430 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.788178 17854 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.788899 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.789757 11330 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:06.790009 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:06.790482 17083 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:06.791127 17083 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.790650 11331 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:06.788209 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 984) took 1821 ms. Trace:
I20260709 12:15:06.801231 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:06.772217 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:06.802644 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.803318 11322 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:06.806484 11250 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:15:06.814952 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52194.9us
I20260709 12:15:06.808398 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.811205 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.591946 17929 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.813607 11178 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:06.818320 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.819037 11329 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:06.822988 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.825032 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:06.797369 11253 rpcz_store.cc:276] 0709 12:15:04.967296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.967493 (+   197us) service_pool.cc:224] Handling call
0709 12:15:04.967924 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.968193 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:05.270322 (+302129us) log.cc:844] Serialized 14 byte log entry
0709 12:15:06.626219 (+1355897us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b1800438d00. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a
0709 12:15:06.627124 (+   905us) log.cc:844] Serialized 14 byte log entry
0709 12:15:06.627721 (+   597us) log.cc:844] Serialized 14 byte log entry
0709 12:15:06.756760 (+129039us) log.cc:844] Serialized 14 byte log entry
0709 12:15:06.764830 (+  8070us) log.cc:844] Serialized 14 byte log entry
0709 12:15:06.769984 (+  5154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:06.770060 (+    76us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:06.787941 (+ 17881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.788047 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:06.788158 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128440,"prepare.queue_time_us":614796,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":6787,"raft.queue_time_us":138764,"raft.run_cpu_time_us":442,"raft.run_wall_time_us":25634,"spinlock_wait_cycles":3360512,"thread_start_us":721284,"threads_started":2,"wal-append.queue_time_us":552017}
I20260709 12:15:06.829233 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:06.830003 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51798.1us
I20260709 12:15:06.830072 11322 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:06.835160 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.836889 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.837410 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.838152 11178 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:06.789765 17800 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:06.841745 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.793555 17083 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.842805 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.863703 11331 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:06.844332 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.846168 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.847425 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.800880 17945 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:06.820667 17907 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:15:06.823110 17893 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:06.876600 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:06.852756 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.858839 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.859494 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.801699 17753 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:06.866305 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:06.811074 17431 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.817561 17803 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:06.820529 17929 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:06.877285 11180 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:06.828861 17932 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.842322 17861 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:06.868548 17954 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:06.874856 17945 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:06.892968 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52676.8us
I20260709 12:15:06.911034 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.912226 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.894140 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:06.876137 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:06.894395 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.894717 17805 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:06.896130 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.896202 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.898267 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.920873 11329 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:06.921562 11322 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:15:06.893435 17854 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.880991 17801 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:15:06.904711 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.928645 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.907271 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:06.929963 11173 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:06.908617 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:06.893442 17851 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:06.914405 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:06.915889 17937 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:06.915196 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:06.868722 17935 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:06.919664 11326 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:15:06.870780 17939 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.022s	sys 0.032s
I20260709 12:15:06.931720 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:06.939018 17811 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:06.940194 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:06.940367 11329 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:06.941969 17782 raft_consensus.cc:993] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:15:06.929437 11179 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
W20260709 12:15:06.980423 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 104352us
I20260709 12:15:06.950098 11182 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:06.893289 17902 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:15:06.984433 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:06.954157 11326 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:15:06.955228 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:06.986773 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:06.906456 17803 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:06.898305 17519 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:06.956619 17939 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.022s	sys 0.033s
I20260709 12:15:06.927762 17966 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:06.997713 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:06.999425 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59305.7us
I20260709 12:15:07.011417 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:07.012082 11331 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 14.
I20260709 12:15:07.010337 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:06.956871 17941 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:06.974454 17869 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:06.975687 17811 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.022922 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.028934 11181 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:06.973312 17646 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:06.918767 17932 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.997388 17851 raft_consensus.cc:1081] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:15:07.032359 17717 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:06.927965 17617 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:15:07.027547 17941 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.918577 17753 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:06.973816 17935 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:06.916426 17968 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:15:06.918434 17861 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.036388 17945 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:06.922704 17431 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.054207 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:06.996795 17803 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:07.010416 17918 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:07.014371 17782 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:07.007894 17967 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:07.014420 17823 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:07.065511 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:07.130654 11331 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:07.052866 17811 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:07.078529 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:07.083165 17932 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.088233 17929 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.092315 11325 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:07.095616 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.058997 17869 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:07.051319 17646 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:07.104614 17753 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.108829 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.074615 17941 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:07.076303 17717 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:07.072275 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.173290 11182 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:07.135434 11328 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:07.137600 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.178084 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:07.126096 17782 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.114048 17918 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.197070 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:07.189230 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80534.4us
I20260709 12:15:07.185192 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:07.199122 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:07.208261 17801 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:07.162304 17967 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:07.208803 17801 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.209216 17801 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:07.209614 17967 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.214874 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:07.170614 17932 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:07.169077 17699 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:07.166167 11173 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:07.192795 17937 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:15:07.193490 17851 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:07.197322 17782 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.224887 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:07.195737 17498 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:07.203246 17918 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:07.234589 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:07.220186 17576 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:07.231027 17699 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.231834 17498 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:07.245071 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:07.247107 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:07.231374 17948 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:07.246749 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 249120us
I20260709 12:15:07.255321 11256 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:07.253741 11180 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:07.256886 11178 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:07.235179 17931 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:07.264210 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.245815 17959 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:07.262251 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.236443 17935 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:07.244903 17864 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:07.260254 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.253657 17948 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:07.246052 17498 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:07.247490 17596 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:07.246320 17064 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:07.268478 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.271121 11182 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:15:07.275434 11178 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:07.254850 17608 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:07.252135 17576 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:07.248656 17937 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:15:07.278939 11250 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:15:07.279361 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.273097 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:07.278736 17935 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:07.281407 17954 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:07.282260 17699 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.269647 17931 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:07.267609 17959 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:07.259583 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:07.279870 17948 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:07.264487 17717 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.228581 17646 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.289336 17431 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.290762 17929 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:07.292511 17981 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:15:07.293109 17801 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:07.349212 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69962.8us
I20260709 12:15:07.301143 17596 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:07.354621 17935 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.355131 17966 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:07.342330 17864 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.332456 17608 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:07.352654 17985 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:07.372948 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:07.374732 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 157763us
W20260709 12:15:07.374984 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 364694us
I20260709 12:15:07.385967 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.387544 11147 leader_election.cc:400] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19.
I20260709 12:15:07.387954 11147 leader_election.cc:403] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:07.382153 17608 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:07.388443 17929 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:07.388823 17869 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:07.394941 17935 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:07.375865 17596 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:07.401908 17931 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.445706 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:07.359743 17954 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.364881 17576 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.452669 17985 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:07.454650 17811 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.456909 17811 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.457659 11181 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:07.450241 17851 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19.
I20260709 12:15:07.458317 17801 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:07.459424 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47867.8us
W20260709 12:15:07.382411 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 182460us
W20260709 12:15:07.442770 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 56063.6us
I20260709 12:15:07.444034 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.384179 17864 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.472117 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:07.473922 17929 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:07.474726 17929 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:15:07.469270 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56724.3us
I20260709 12:15:07.475184 17929 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:07.475628 17929 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:07.477438 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:07.448348 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.473294 17864 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.491389 17871 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.382959 17803 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.496034 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:07.420346 17966 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:15:07.441828 17782 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.515625 11935 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:07.533039 17987 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:07.539409 17803 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:07.543946 17954 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.544723 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.520043 17823 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:15:07.551641 17941 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.564018 17941 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.556491 17596 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:07.558533 17994 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:07.576156 17596 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.560009 17782 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.493562 17959 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.580103 17959 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:07.554170 17608 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.588268 17608 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.594116 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:07.609659 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 109023us
I20260709 12:15:07.557137 17948 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.620393 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.604386 17966 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.620450 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:07.625959 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:07.626775 11929 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:07.628545 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:07.604928 17935 consensus_queue.cc:1175] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 19 status { last_received { term: 19 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 19 index: 9 } }
I20260709 12:15:07.608673 17608 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.628635 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.609061 17596 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.616726 17498 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.639981 11255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:15:07.644691 11255 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.645223 11255 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:07.642771 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:07.643898 11935 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:07.631176 17966 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.654505 17871 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:07.655035 17871 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:07.656069 17871 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.641376 17608 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.662782 17608 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.664428 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:07.666344 17996 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:15:07.666770 11250 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:07.680497 17861 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.682181 11331 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:15:07.682631 11331 pending_rounds.cc:85] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 14
I20260709 12:15:07.682878 11331 pending_rounds.cc:107] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:15:07.683111 11331 raft_consensus.cc:2889] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:07.684854 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:07.685765 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:07.687091 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:07.687428 17646 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:07.688186 17717 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.688731 17801 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:07.647370 17596 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.691177 17596 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.688747 17717 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.694581 17717 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.696944 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:07.697759 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.699882 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.700683 17959 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:07.701668 17993 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:07.702967 17929 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.710168 11256 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:07.712531 17498 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.713581 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:07.714524 17966 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:07.717001 11330 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:07.654825 17948 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.712163 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:07.728842 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:07.734948 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.736181 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.739594 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:07.743774 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:07.744917 11257 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:07.745960 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.754288 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.774611 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:07.774916 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.779528 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:07.782879 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.784297 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:07.786361 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 133756us
I20260709 12:15:07.786785 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:07.787530 11251 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.788867 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.791706 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:07.792083 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.793691 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:07.794472 11180 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:07.797700 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.802692 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.806258 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.808158 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:07.808961 11177 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:07.810729 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:07.812180 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:07.818471 11932 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:15:07.821940 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:07.822966 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 110940us
I20260709 12:15:07.826229 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.818370 17954 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:07.823994 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:07.788204 17918 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.003s
I20260709 12:15:07.864511 11180 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:07.865459 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:07.856093 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.866799 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:07.788074 17498 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:07.870224 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 368563us
I20260709 12:15:07.819741 17657 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:07.925890 17657 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:07.926406 17657 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:07.827024 17869 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:07.831497 17871 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:07.827849 17969 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:07.840359 17936 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:07.932560 17936 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:07.933171 17936 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:07.852504 17699 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:07.852721 17954 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.938714 17954 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.939685 17657 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.820233 17932 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:15:07.944190 17954 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:15:07.950628 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:07.951198 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:07.952745 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:07.953526 11324 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:07.955829 17988 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:07.956984 17988 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:07.916998 11254 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:07.918893 17596 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:07.918959 11177 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:07.955885 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:07.957962 11325 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:07.956804 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:07.957737 17596 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:07.964340 17801 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.964795 17596 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.964972 17801 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.967487 17801 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:07.967744 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3022) took 1052 ms. Trace:
I20260709 12:15:07.968124 11181 rpcz_store.cc:276] 0709 12:15:06.915042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.057785 (+142743us) service_pool.cc:224] Handling call
0709 12:15:07.967553 (+909768us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54000c0b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:07.967688 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743941888}
W20260709 12:15:07.971312 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 115275us
I20260709 12:15:07.973134 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:07.973806 17902 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:07.974218 17657 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:07.874372 17248 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:15:07.975351 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:07.976176 11324 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:07.977448 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:07.985064 11325 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:07.851186 11255 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:07.988732 11176 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:07.877722 17929 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:07.915247 11929 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:07.899994 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:07.957363 17988 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:08.019783 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:07.868600 17918 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.026141 11330 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:08.027760 11255 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:08.040350 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 109400us
W20260709 12:15:08.044772 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71740.9us
W20260709 12:15:08.059636 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 159692us
I20260709 12:15:08.070564 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:08.042914 17596 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:08.076763 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:08.054595 17947 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.087448 11257 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:08.051044 17924 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:08.050166 17431 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:08.091708 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:08.102769 11176 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:08.074234 17608 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:08.079331 17800 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.086555 11250 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:08.055104 17999 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:08.089332 11935 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:08.058353 17987 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:15:08.100517 17864 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:08.116750 17699 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:08.087437 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 295387us
I20260709 12:15:08.122189 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:08.090865 17787 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:15:08.123536 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:08.129755 11928 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:08.130394 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:08.122942 17340 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:15:08.134641 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:08.139012 17717 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:08.139559 17340 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:08.139709 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:08.091373 17596 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.143014 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:08.151684 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:08.105536 17431 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:08.100714 17924 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.152668 17959 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:08.119740 17864 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:08.155376 17991 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:08.097103 17947 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.195051 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:08.196925 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:08.198019 11325 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:15:08.160490 11330 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:08.117429 17800 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.162775 17935 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:08.178162 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:08.179303 17924 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:08.173532 17608 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:08.183020 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:08.192957 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:08.184295 17864 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:08.107685 17988 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.243736 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:08.245015 11928 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:15:08.253123 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 161490us
I20260709 12:15:08.245290 11329 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:08.254585 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:08.185740 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:08.246379 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 151227us
I20260709 12:15:08.248384 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:08.185170 17991 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:08.263182 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 77549us
I20260709 12:15:08.263410 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:08.252418 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:08.264077 11933 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:08.252722 11249 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:08.255901 11248 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:15:08.256009 11181 raft_consensus.cc:2368] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20.
I20260709 12:15:08.165468 17871 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:15:08.258986 11251 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
W20260709 12:15:08.259631 18007 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:08.230504 17498 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:08.310142 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57824.1us
W20260709 12:15:08.310030 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55534.6us
I20260709 12:15:08.312598 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:08.315277 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:08.230182 17987 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:15:08.203608 17954 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
W20260709 12:15:08.314270 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 77093.6us
I20260709 12:15:08.339486 11329 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 15.
I20260709 12:15:08.354570 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:08.316115 17646 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.004s
I20260709 12:15:08.270282 17991 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:08.311492 17869 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:15:08.392161 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76956.5us
I20260709 12:15:08.256923 17924 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 12:15:08.333212 17889 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:08.340811 17969 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:08.338223 17918 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:08.411432 18003 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:15:08.315122 17498 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.402238 17805 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:08.435905 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60418us
W20260709 12:15:08.478483 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87680.8us
I20260709 12:15:08.488541 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:08.489084 11076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 259) took 69585 ms (client timeout 15000 ms). Trace:
I20260709 12:15:08.491164 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:08.462338 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:08.382602 17999 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 12:15:08.519785 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:08.519685 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 249068us
I20260709 12:15:08.642204 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:08.642899 11182 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:08.643249 11182 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:08.530472 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 114544us
I20260709 12:15:08.530941 17889 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:08.496891 11078 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 10 to 13, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 13 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:08.498643 18011 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:08.489612 11076 rpcz_store.cc:269] 0709 12:13:58.858977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.861838 (+  2861us) service_pool.cc:224] Handling call
0709 12:13:59.281587 (+419749us) spinlock_profiling.cc:243] Waited 70.1 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:08.444165 (+69162578us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:04.017337 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:04.018137 (+   800us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:04.018201 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:15:04.018223 (+    22us) tablet.cc:662] Decoding operations
0709 12:15:04.019062 (+   839us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:04.019182 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:04.019223 (+    41us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:15:04.019806 (+   583us) tablet.cc:701] Row locks acquired
0709 12:15:04.019859 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:15:04.020129 (+   270us) write_op.cc:270] Start()
0709 12:15:04.020323 (+   194us) write_op.cc:276] Timestamp: P: 1783599304020041 usec, L: 0
0709 12:15:04.020364 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:15:04.021157 (+   793us) log.cc:844] Serialized 1312 byte log entry
0709 12:15:04.362743 (+341586us) op_driver.cc:464] REPLICATION: finished
0709 12:15:04.706266 (+343523us) write_op.cc:301] APPLY: starting
0709 12:15:04.730557 (+ 24291us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:04.831881 (+101324us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:04.832015 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:04.948002 (+115987us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:04.949831 (+  1829us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 12:15:04.949906 (+    75us) write_op.cc:312] APPLY: finished
0709 12:15:04.950214 (+   308us) log.cc:844] Serialized 41 byte log entry
0709 12:15:04.950417 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:04.950511 (+    94us) write_op.cc:454] Released schema lock
0709 12:15:04.950570 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:04.950985 (+   415us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:04.951014 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":150371,"reported_tablets":50,"spinlock_wait_cycles":161173376,"child_traces":[["op",{"apply.queue_time_us":316315,"apply.run_cpu_time_us":11557,"apply.run_wall_time_us":258367,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16691,"num_ops":3,"prepare.queue_time_us":120059,"prepare.run_cpu_time_us":21793,"prepare.run_wall_time_us":77199,"raft.queue_time_us":50948,"raft.run_cpu_time_us":11179,"raft.run_wall_time_us":253511,"replication_time_us":342214,"thread_start_us":397002,"threads_started":4,"wal-append.queue_time_us":205454,"wal-append.run_cpu_time_us":11808,"wal-append.run_wall_time_us":1730769}]]}
I20260709 12:15:08.446151 17969 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:08.663065 17969 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:08.646648 11182 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.664467 11182 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:08.668905 17935 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:08.669801 18011 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:08.579234 18022 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:15:08.590142 18028 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:08.591675 17947 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:08.593870 17596 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:08.595431 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:08.599135 17498 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:08.520845 17805 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:08.600188 18026 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:15:08.601713 18021 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:15:08.602494 17431 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:08.603513 18027 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:08.439551 17608 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:08.606040 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1006) took 1466 ms. Trace:
I20260709 12:15:08.483418 17966 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:08.609440 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:08.744127 11176 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:08.468871 17985 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:08.473097 17646 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:15:08.410053 17929 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:08.416364 17893 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:08.638092 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:08.649564 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 129863us
I20260709 12:15:08.652479 11932 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 12:15:08.634991 18012 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.645437 17889 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:08.578392 18010 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.263s	user 0.000s	sys 0.000s
I20260709 12:15:08.684288 11182 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:08.669409 17935 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:08.691488 11078 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 11 to 12, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 12 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:08.693886 11181 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:08.699564 17991 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:08.703037 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:08.703946 17864 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:08.711826 11933 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:08.715793 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:08.722728 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78698.6us
I20260709 12:15:08.722772 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:08.738736 18032 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:08.738857 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:08.739995 11253 rpcz_store.cc:276] 0709 12:15:07.139491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.139662 (+   171us) service_pool.cc:224] Handling call
0709 12:15:07.182688 (+ 43026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.208687 (+ 25999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.208721 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.209261 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.004693 (+795432us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b18003b1820. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:08.250423 (+245730us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:08.250466 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.250813 (+   347us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.605545 (+354732us) raft_consensus.cc:1685] finished
0709 12:15:08.605582 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.605947 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32511,"raft.queue_time_us":21805,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":97,"spinlock_wait_cycles":14467328,"thread_start_us":245542,"threads_started":1,"wal-append.queue_time_us":1394588}
I20260709 12:15:08.748118 18011 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:08.706029 17871 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:08.707484 17969 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:08.757441 11176 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:08.762233 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:08.851585 11180 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:08.784327 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:08.801229 11930 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:08.859305 11249 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:08.766114 17889 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:08.758533 18028 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:08.770502 18027 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:08.785411 17936 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.852s	user 0.012s	sys 0.001s
I20260709 12:15:08.869858 17717 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:08.871622 18032 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:08.878757 17248 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:08.882194 11250 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:08.885515 11254 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:08.761200 17966 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.890247 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1255) took 1186 ms. Trace:
I20260709 12:15:08.850210 17869 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:08.898837 11322 rpcz_store.cc:276] 0709 12:15:07.704239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.729864 (+ 25625us) service_pool.cc:224] Handling call
0709 12:15:07.730351 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.730865 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.730911 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.731578 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.043044 (+311466us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:08.043085 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.043158 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.889976 (+846818us) raft_consensus.cc:1685] finished
0709 12:15:08.890007 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.890170 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3674,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":29451,"thread_start_us":311083,"threads_started":1,"wal-append.queue_time_us":510826}
I20260709 12:15:08.770890 17959 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.119s
W20260709 12:15:08.897014 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 188187us
W20260709 12:15:08.903676 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.019s	sys 0.055s
I20260709 12:15:08.758484 17985 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:08.904094 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:08.904053 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.019s	sys 0.055s
I20260709 12:15:08.907235 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:08.917989 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:08.766356 18012 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.861330 17871 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:08.863992 17969 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:08.941044 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 452213us
I20260709 12:15:08.970471 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:08.945696 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:08.943346 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59863.4us
I20260709 12:15:08.939217 17948 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:08.974594 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:08.978981 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:08.849813 17902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.011s	sys 0.008s
I20260709 12:15:08.917032 17941 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:08.988119 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:08.954819 18013 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:15:08.893260 17936 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:08.990844 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:08.890893 18027 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:08.977348 18030 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:15:09.008016 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:08.954808 18018 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:08.939451 17959 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:09.014295 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:08.957445 17800 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.017366 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:09.017830 17893 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:08.913571 17987 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 12:15:08.983103 17596 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:09.017237 17948 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:08.998871 18017 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:09.048623 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:09.050333 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.056346 11256 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:15:09.048156 11248 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:09.036427 17800 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:09.060123 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:09.037480 17596 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:08.982523 17869 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.046691 18040 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:09.076911 11078 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 9 to 12, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 12 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:09.029577 17959 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.051730 18018 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:09.046593 17893 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:09.089037 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:09.098397 11181 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:09.089959 18012 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.091725 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:09.091980 18009 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:15:09.093341 11078 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:09.096268 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.101389 11930 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:09.058095 18017 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:09.068714 17800 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:09.079501 17596 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:09.119831 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:09.129886 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:09.083151 17864 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.134403 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:09.098345 18018 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.131577 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.141073 11254 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:15:09.014912 18028 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.020666 18032 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.121656 17991 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.118314 17948 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:09.128346 17800 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:09.138975 18012 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:09.142123 18018 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:09.150734 18011 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:09.142035 18036 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:09.152201 17864 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:09.161454 17985 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:09.169741 18012 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.175048 17871 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:09.164224 17948 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.191785 18012 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:09.195199 17936 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.185770 17969 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.201941 17959 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:09.218623 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 206961us
I20260709 12:15:09.189754 17864 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.213815 18048 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.228250 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:09.229068 11928 raft_consensus.cc:2368] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22.
I20260709 12:15:09.229831 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.232177 11222 leader_election.cc:400] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22.
I20260709 12:15:09.232609 11222 leader_election.cc:403] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:15:09.195964 18017 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.234376 17948 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.238076 17787 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:09.238502 17787 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:09.239274 17787 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.240477 17787 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:09.240680 18038 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:15:09.242473 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2619) took 1043 ms. Trace:
I20260709 12:15:09.242879 11330 rpcz_store.cc:276] 0709 12:15:08.199104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.201803 (+  2699us) service_pool.cc:224] Handling call
0709 12:15:08.202161 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:08.232387 (+ 30226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.232434 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:08.242191 (+  9757us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.744176 (+501985us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:15:08.905494 (+161318us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:08.905535 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.905601 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:09.242165 (+336564us) raft_consensus.cc:1685] finished
0709 12:15:09.242186 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.242410 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28912,"raft.queue_time_us":32429,"raft.run_cpu_time_us":4028,"raft.run_wall_time_us":509066,"spinlock_wait_cycles":6214912,"thread_start_us":578826,"threads_started":1,"wal-append.queue_time_us":756139}
I20260709 12:15:09.190225 17893 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.250571 17948 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:09.259717 18049 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:15:09.278291 17864 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.283342 17800 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:09.290285 18068 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:15:09.293200 17861 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:09.293695 17861 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:09.294528 17861 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.295544 17861 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:09.232301 18048 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.300215 17935 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.301303 17935 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:09.343320 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:09.344280 11331 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:09.300239 18048 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.361155 17858 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:09.361693 17858 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:09.362332 17948 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22.
I20260709 12:15:09.362854 17858 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.364833 17858 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:09.366182 17340 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:09.366632 17340 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:09.367640 17340 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.371189 18027 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.377825 18064 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.378403 18064 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.380963 18064 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.382419 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.383154 11930 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:09.384014 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.384513 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.368964 17340 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:09.397111 18011 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:09.400916 17596 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:09.405198 18063 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
I20260709 12:15:09.408177 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.408255 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:09.423513 11331 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:09.392288 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:09.434162 11328 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:09.438205 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:09.447475 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 474185us
I20260709 12:15:09.356822 17800 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.457015 11174 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:09.462643 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:09.463306 11174 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:09.463982 17782 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:09.465175 17782 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:09.431849 17902 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.466279 17902 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.468053 18039 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.468736 18039 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.469837 17991 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:09.470963 17991 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:09.471154 18039 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.471366 17991 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:09.462078 18064 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.472224 17991 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.472555 18064 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.473246 17991 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:09.475148 18064 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.445396 11251 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:09.434108 18048 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:09.478366 11174 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:09.479943 17596 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.480786 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:09.459389 18053 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.482267 18053 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.483723 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:09.484814 18053 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:09.469831 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.429491 17811 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:09.490396 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:09.490581 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.491488 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:09.494127 11177 raft_consensus.cc:2368] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22.
I20260709 12:15:09.493216 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:09.499001 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:09.501264 11253 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:09.502295 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.492069 18039 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.459021 18000 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.514717 18000 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.516775 18039 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.519116 18039 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.477556 18048 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.519897 18048 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:09.520002 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.513157 18045 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:09.513994 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.523725 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.526480 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.465569 17782 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:09.501279 17811 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.534720 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.535593 11323 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:15:09.538070 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.539996 11331 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:09.508623 18011 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:09.546093 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.546803 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:09.551056 11174 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:09.530905 18045 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:09.557257 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:09.557689 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.572314 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:09.572733 11253 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:09.559458 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.561779 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:09.569492 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:09.570750 11251 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:09.576344 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.596933 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.599287 11222 leader_election.cc:400] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22.
I20260709 12:15:09.602660 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:09.603621 11181 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 21 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:15:09.608178 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fd765747f9648049b905c70696b206d"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:09.638705 11173 raft_consensus.cc:606] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:09.639170 11173 raft_consensus.cc:614] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:09.607779 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 154651us
I20260709 12:15:09.574036 17959 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.556201 17717 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:15:09.617901 18038 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:15:09.596961 17941 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:09.638373 18067 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:09.640205 18039 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:09.642470 17889 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.753s	user 0.012s	sys 0.001s
I20260709 12:15:09.648900 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:09.587698 17948 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.062s
W20260709 12:15:09.661056 18029 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.910s	user 0.001s	sys 0.009s
W20260709 12:15:09.661401 18015 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.277s	user 0.003s	sys 0.000s
I20260709 12:15:09.663241 17902 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.678951 18000 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.683424 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:09.684227 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.684989 11330 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:15:09.646528 18070 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:09.674441 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:09.553750 18011 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.672242 17889 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.718791 11180 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:09.706558 17893 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:09.647711 17717 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.714401 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.710218 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.646263 17959 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.753060 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:09.764317 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44708.9us
I20260709 12:15:09.727883 17893 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:09.767995 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:09.759517 18025 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:09.718978 17869 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:09.762406 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51022.3us
I20260709 12:15:09.719502 17969 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.781569 17935 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:09.659528 18018 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.707322 18034 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:09.648506 17782 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.712530 18000 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:15:09.788002 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 116750us
I20260709 12:15:09.790000 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:09.763741 17717 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:09.782804 17988 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:09.675745 17941 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.686846 18039 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.739537 17954 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.782976 17811 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:09.840728 11929 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:09.836974 11254 raft_consensus.cc:2368] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22.
W20260709 12:15:09.829305 17340 consensus_queue.cc:1175] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 21 status { last_received { term: 20 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:09.667326 18045 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:09.830653 18012 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.607s	user 0.007s	sys 0.004s
I20260709 12:15:09.712329 18070 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:09.847923 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60706.6us
I20260709 12:15:09.825821 18000 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:09.799712 17935 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.794988 17969 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.790611 18025 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.787698 17869 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.868248 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:09.877903 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.875080 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.869225 18000 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:09.843897 17954 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.891403 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:09.880811 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 90908.1us
I20260709 12:15:09.894490 11327 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:09.900888 11324 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:09.856745 18012 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:09.856323 18084 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:15:09.873917 18064 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:09.895982 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:09.890311 18081 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:15:09.896656 18054 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:09.938848 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.975883 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:09.977445 11326 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 21 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:15:09.905774 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:10.021226 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 115526us
I20260709 12:15:09.927464 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:10.022083 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:09.934463 11323 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:15:10.022835 11326 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:09.923421 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.023869 11929 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 12:15:10.024380 11331 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:10.025475 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:09.903944 17987 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:09.844689 17893 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:09.979795 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:09.904735 17992 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:09.985100 17969 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:09.985297 18088 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:15:09.940636 18028 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:15:09.986896 17954 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:09.988099 18075 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.225s	user 0.000s	sys 0.000s
I20260709 12:15:09.907349 17805 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:09.989984 18091 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:15:09.991703 17869 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:09.992378 18080 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:15:09.992957 18087 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:15:10.011790 17935 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.013523 18012 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:09.899094 11328 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:09.913254 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:10.029623 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:10.121393 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:10.122164 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 92627.6us
I20260709 12:15:10.030153 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.031126 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.032938 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.048924 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.056012 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:10.062108 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:10.125131 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.070354 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:10.071914 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.013837 18011 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.086426 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.125921 11933 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:10.088042 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.092666 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.106459 17954 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.128981 17861 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:10.129691 17861 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.129619 17954 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.131335 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.132470 17861 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.132617 17954 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.095158 17969 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.138126 17969 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.141387 17969 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.144728 11326 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:10.145489 11323 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:10.146335 11933 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:10.154173 17805 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:10.154701 17805 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:10.155149 17805 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:10.155571 11935 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:10.156445 17498 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.156771 11935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 984) took 2035 ms (client timeout 2197 ms). Trace:
I20260709 12:15:10.156970 17498 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:10.157287 11935 rpcz_store.cc:269] 0709 12:15:08.121782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.121947 (+   165us) service_pool.cc:224] Handling call
0709 12:15:10.156577 (+2034630us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b540015e0d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:10.156705 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4657412992}
I20260709 12:15:10.100533 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.103739 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.160807 17893 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:10.068186 17931 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:10.107937 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:10.089005 18048 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.004s
W20260709 12:15:10.063855 17858 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.698s	user 0.006s	sys 0.009s
I20260709 12:15:10.079692 17992 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.147977 17988 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:10.115540 11325 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:10.094468 17959 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.168493 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.235494 11180 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:10.167892 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.236590 11251 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:10.102250 18094 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:10.237690 18094 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.227506 17498 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.181252 17869 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.242905 17869 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.248277 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.186580 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:10.247357 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139506us
W20260709 12:15:10.188609 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 275435us
I20260709 12:15:10.249205 11325 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:10.249612 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.252475 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.080430 17987 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.212318 17992 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:10.218233 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.245395 17869 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.190009 17966 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.108335 18018 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.255612 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:10.312721 11177 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:10.094192 18085 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 12:15:10.259646 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.265017 11254 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:10.164245 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.254933 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:10.315068 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60234.7us
W20260709 12:15:10.214653 18086 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:15:10.229457 17889 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.262885 18039 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.215292 17340 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:10.215633 18092 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:10.280308 18042 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:10.246556 17931 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:10.326586 18095 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:15:10.283988 18000 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.246893 18048 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.254642 18025 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.255162 18064 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:10.261807 18024 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:10.293975 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:10.303447 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53811.3us
I20260709 12:15:10.251060 17948 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:10.332290 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 402430us
I20260709 12:15:10.283473 17987 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.344587 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.370277 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:10.370977 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.254145 18074 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:10.365885 18064 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.406387 17941 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.347357 18092 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.346355 18042 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.342578 17340 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.411231 18025 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:10.336881 17657 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.320853 17811 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:15:10.411777 18047 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:10.413722 11250 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:10.424607 18042 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.425168 18000 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.426638 18092 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.428522 17340 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.426746 18048 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:10.429592 17717 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.004s
I20260709 12:15:10.409863 17931 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.406548 18064 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:10.362159 11177 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:10.458109 18028 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:10.479056 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:10.432206 17864 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:10.478240 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 98501.8us
I20260709 12:15:10.463850 17657 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.486652 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.489810 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.494465 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.495219 11935 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:10.496498 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:10.497108 11935 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:15:10.497529 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:10.498329 11176 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:10.487629 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.502390 11174 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:10.505944 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.506002 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.473055 17869 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.507619 11931 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:10.507616 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:10.515401 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.506062 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.512331 11174 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:10.472508 17966 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.516041 11179 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:10.519877 11174 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:10.514272 11935 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:10.482996 17717 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.514881 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:10.483040 18042 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:10.524796 11934 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:15:10.522109 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 435795us
W20260709 12:15:10.527405 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.528714 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:10.529764 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:10.530833 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.531879 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.533125 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.536338 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.536966 11174 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:10.537670 11177 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:10.497805 17889 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:10.542482 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.543334 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.543696 11174 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:10.544813 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.517861 18000 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.550109 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.547006 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.547677 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.538892 17340 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.522084 18092 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:10.553835 11180 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:10.446563 18074 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.480165 17992 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.507244 18048 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:15:10.554872 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.556140 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:10.565188 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 57107.5us
I20260709 12:15:10.558815 11935 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:10.559830 11900 leader_election.cc:400] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20.
I20260709 12:15:10.561981 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:10.540936 18011 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.566828 11174 raft_consensus.cc:606] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.565917 11176 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:10.578902 11174 raft_consensus.cc:614] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:10.579434 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.580109 11253 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:10.582172 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.559477 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.541512 18090 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:10.548993 17988 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:10.586679 18101 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:10.588007 18011 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.543902 18028 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:10.581913 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:10.565963 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.585067 11900 leader_election.cc:403] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:15:10.541252 17935 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:10.524632 17966 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.585011 18090 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.646888 18090 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:10.523175 17869 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.648376 17966 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.546052 18042 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.649592 17869 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.600951 18024 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:10.605729 17935 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:10.649989 17948 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:10.650497 18024 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:10.650833 18042 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.651054 17935 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.651510 18024 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.607393 17805 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.579607 18098 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.611290 17858 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:10.654405 17858 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:10.654114 18098 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.567294 17864 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:10.614079 18100 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.199s	user 0.007s	sys 0.000s
I20260709 12:15:10.558751 17954 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:10.619076 18011 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.621213 17657 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.663149 17858 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.663852 18094 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.664337 17858 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:10.665297 18098 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.624502 18018 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:10.626295 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:10.627357 18039 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.560254 17340 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.630158 11254 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:10.631429 18074 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.631776 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2826) took 1103 ms. Trace:
I20260709 12:15:10.635900 17803 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20.
W20260709 12:15:10.636807 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 77107us
W20260709 12:15:10.677578 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95479.5us
I20260709 12:15:10.678741 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.643781 18070 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.603869 18028 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.558284 17969 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:10.569725 18085 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:10.697052 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:10.658903 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 104118us
I20260709 12:15:10.697820 11331 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:10.698170 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.699158 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.563622 18053 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.622303 17608 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:15:10.660744 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 173335us
I20260709 12:15:10.715152 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.585427 17717 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.556057 17889 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.718601 17889 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.666327 11173 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:10.628461 17992 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:10.669684 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43484.5us
I20260709 12:15:10.559934 18000 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.722527 18000 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.650498 17948 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:10.678643 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.590209 17861 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:10.676983 11250 rpcz_store.cc:276] 0709 12:15:09.528376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.531157 (+  2781us) service_pool.cc:224] Handling call
0709 12:15:09.531447 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:10.464922 (+933475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:10.631515 (+166593us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b5400068e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:10.631715 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020199168}
I20260709 12:15:10.703290 11323 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:10.724030 17948 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.672645 18034 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:10.660548 17864 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.726078 11178 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:15:10.727244 17864 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.707314 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.728109 11249 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:10.709084 18025 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.728950 18025 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.731770 18025 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.708473 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.676919 18098 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.709733 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.711074 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.725358 18034 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.733405 18098 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.734097 18034 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:10.735991 18098 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.708210 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:10.739733 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:10.740902 18094 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:10.741899 18094 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:10.742380 18094 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.742864 18094 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:10.746896 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:10.748360 17948 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:10.722575 17340 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:10.679287 18047 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:10.680546 18042 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:10.771155 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50690.3us
W20260709 12:15:10.771473 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92910.4us
I20260709 12:15:10.774507 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.783912 18090 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:10.786486 18090 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:10.772677 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:10.849944 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 77379.1us
I20260709 12:15:10.791771 11331 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:10.852675 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:10.853683 11331 raft_consensus.cc:2368] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21.
I20260709 12:15:10.854841 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.855892 11331 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:15:10.677021 18039 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.710264 17782 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:10.793483 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 78422us
I20260709 12:15:10.794623 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.709679 18074 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:10.768810 18053 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.795464 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.860343 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:10.795804 17805 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:10.787875 18047 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.798729 18085 raft_consensus.cc:2764] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 12:15:10.800313 17992 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:10.768319 18032 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:15:10.801783 18098 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:10.801362 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:10.803193 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.805848 17340 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.905556 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 104285us
I20260709 12:15:10.907052 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.909049 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.714279 17954 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:10.793658 17864 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 12:15:10.810289 18107 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:15:10.810660 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.913990 11935 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:10.811590 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.794100 17969 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:10.794202 17803 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:10.813966 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.815464 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:10.918506 11249 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:10.772071 18042 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:10.814960 17893 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.004s
I20260709 12:15:10.822073 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.822465 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.824076 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.793152 18011 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:10.830011 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:10.830379 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.831794 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.833108 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.833863 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:10.938784 11180 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:10.836556 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:10.842000 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 257299us
I20260709 12:15:10.844533 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.940557 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.844333 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:10.847697 17869 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:10.866255 18092 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.878641 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.880488 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.951843 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.952518 11177 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:10.896471 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.953972 11327 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:10.867904 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:10.953568 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.955658 11177 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:10.956991 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:10.959712 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.961033 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:10.965621 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:10.966367 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:10.898797 11328 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:10.845906 17941 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:10.903494 11932 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:10.909793 11175 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:10.814247 18101 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:15:10.931919 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.861754 17966 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.861160 18067 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:10.869138 17935 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.897114 18017 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:15:10.838277 18064 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.895205 17800 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.990096 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:10.917871 17801 raft_consensus.cc:2764] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 12:15:10.989120 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 121291us
I20260709 12:15:10.896951 18020 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:15:10.917697 17969 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:10.876219 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:10.936146 18042 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:10.867013 18025 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 12:15:11.002907 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:11.006662 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:11.019331 11249 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:11.008656 11180 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:10.935819 18018 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:10.997146 11934 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:11.010838 18084 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:10.966542 18110 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:11.042131 11248 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:11.093344 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:11.094460 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 250187us
I20260709 12:15:11.045003 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.046000 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.095443 11329 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:10.930981 17340 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.052551 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:10.993343 17657 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:15:11.087913 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46902.4us
I20260709 12:15:10.989362 18074 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:11.077901 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.080025 11900 leader_election.cc:400] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22.
I20260709 12:15:11.010548 18012 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.988631 17941 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:10.992491 18090 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.035844 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:10.946069 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:10.999845 17954 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:11.084272 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47107.4us
I20260709 12:15:11.086660 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.087584 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.121946 11148 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.090071 11934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365"
dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51168
I20260709 12:15:11.122671 11934 raft_consensus.cc:606] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.092024 11182 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:15:11.124236 11935 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:11.125274 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:11.126204 11256 raft_consensus.cc:2368] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21.
I20260709 12:15:11.090786 18084 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.043977 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.133364 11327 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:11.001967 18027 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:11.097455 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:11.044104 18073 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:15:10.953487 18011 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.966342 18092 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.104740 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.106880 11255 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:11.024168 17935 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:10.965723 17869 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:11.120512 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 185380us
I20260709 12:15:11.094455 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.093241 18110 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:11.024354 18102 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:11.023780 17966 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.100109 17657 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.099810 18074 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.045866 17800 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.141783 11148 leader_election.cc:400] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21.
I20260709 12:15:11.177431 11148 leader_election.cc:403] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:15:11.042963 17969 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:11.167846 17858 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.007s
I20260709 12:15:11.119930 18012 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:11.163744 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 67574.7us
I20260709 12:15:11.092065 18034 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.094764 17596 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:15:11.137521 18084 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:11.094094 18018 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.129608 17954 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.180506 11326 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:11.161101 18027 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:11.125761 18090 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.172554 17991 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:11.184967 11324 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:11.175715 17657 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:11.178846 18074 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:11.216573 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74943.6us
I20260709 12:15:11.216632 11328 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
I20260709 12:15:11.218503 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:11.222290 11253 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:11.229714 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:11.228353 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54613.2us
W20260709 12:15:11.232391 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52256.5us
I20260709 12:15:11.233409 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:11.234946 18119 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:15:11.235706 18115 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:15:11.224973 17851 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21.
I20260709 12:15:11.236588 17851 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:11.237053 18112 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:15:11.238716 18012 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:11.239079 17935 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.241355 18027 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.243263 18034 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:11.244110 18114 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 12:15:11.246323 18117 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:15:11.231542 17991 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:11.250983 17954 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.250852 17991 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.251411 18122 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.252009 18122 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.253182 17800 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:11.253885 18039 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.254961 17340 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.255306 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.255553 18018 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.259207 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.268731 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:11.272382 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.272760 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.273499 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.273856 18128 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:11.274639 18128 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:11.275511 18128 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.276026 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:11.277886 17340 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:11.278962 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.179122 18109 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:15:11.280565 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.277608 17954 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:11.228286 18047 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.178144 17941 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.289319 18126 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:15:11.267112 17498 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:11.295320 17498 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:11.270002 18095 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:15:11.278344 18101 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:11.270568 18094 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.518s	user 0.004s	sys 0.008s
I20260709 12:15:11.300596 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.300860 18094 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.302184 17869 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.289549 18039 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:11.298888 18073 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:11.286024 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.306167 11934 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:11.290167 11180 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:11.307190 17966 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.291097 18090 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.292479 18127 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:11.308960 18127 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:11.309233 18092 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.279014 17935 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:11.310050 18094 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:11.266747 18102 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:11.281728 17805 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:11.310654 18109 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:11.311797 17498 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:11.312757 17657 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.313805 17969 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.355877 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:11.228586 18110 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.321300 11182 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:15:11.284197 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:11.309307 18127 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:11.327544 18067 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:11.331658 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.287806 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.288214 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:11.335235 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.335399 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.336220 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.337186 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.377357 11249 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 15.
I20260709 12:15:11.335911 11930 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:15:11.337503 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.338613 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.352864 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.371286 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.372576 11174 raft_consensus.cc:606] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:11.373986 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 89856.9us
I20260709 12:15:11.380242 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.378708 11251 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:11.381075 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.388659 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.389132 11929 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:11.335189 18122 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.386029 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:11.393241 11252 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:11.384756 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.388447 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.388269 18090 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:11.352118 18025 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.402096 18025 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.403925 17966 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.404565 17966 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.405105 18025 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.405309 18053 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.406029 17954 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:11.407291 17657 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:11.407793 18064 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.382539 17935 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:11.373530 18020 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:15:11.413139 18011 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.382771 18073 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:11.416226 18073 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.416750 18073 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:11.375540 18067 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:11.333564 18084 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.424301 18084 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:11.407291 17966 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.427559 17969 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:11.428340 18054 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:11.355543 18094 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:11.431320 18094 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.354527 18109 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:11.435897 18109 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.439688 18042 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:11.444304 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64158.1us
I20260709 12:15:11.447752 11253 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:11.448099 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.451692 11932 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:15:11.455358 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.457274 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.457854 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:11.458657 11934 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:11.462150 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.463034 11257 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:11.463759 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.469266 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.374594 18098 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.476109 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.488148 11328 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:11.489104 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.494834 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:11.448594 18109 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:11.498171 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.499107 11249 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:11.476940 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.481534 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.483394 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:11.486406 11929 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:15:11.487272 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.503006 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.506129 11928 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:11.501727 11175 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:11.504143 11932 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:11.507767 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:11.501929 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 127202us
I20260709 12:15:11.516777 11252 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:11.507156 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:11.464110 18110 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:11.519573 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.471632 18064 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:11.524463 11174 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:11.519768 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.528966 11326 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:15:11.530230 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:11.537281 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:11.540719 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:11.522838 17782 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.002s
I20260709 12:15:11.484390 18122 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.505497 18120 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.544317 11173 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:11.460916 18053 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:15:11.520244 17969 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.532191 17902 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.545773 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:11.567415 11252 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 12:15:11.557670 11254 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:11.569139 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.473006 18025 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:15:11.555924 11176 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:11.572073 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.573154 11328 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:11.581239 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.583570 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.598511 11329 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:15:11.598434 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 52746.8us
I20260709 12:15:11.598037 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:11.600869 11326 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:11.602586 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:11.522395 18098 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.612501 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.608505 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.613494 11934 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:11.609453 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:11.558468 18045 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:11.616150 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.619966 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:11.597153 17717 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.597838 18094 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:11.555025 18032 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:15:11.602203 17966 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:11.605819 18000 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:11.631628 11257 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:11.616539 18042 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:11.636610 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.616828 18074 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.644765 18074 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.645571 11178 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:15:11.647881 18084 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:11.649147 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.619004 18039 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:11.652534 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.598018 17969 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.564407 18122 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.618525 18045 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:11.560480 18120 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.691491 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:11.617870 18130 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.604146 17902 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.633145 18000 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.697757 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:11.630596 17717 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.651945 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.701480 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:11.694717 17987 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.632759 17800 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
W20260709 12:15:11.726786 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74898.7us
I20260709 12:15:11.715322 17902 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:11.709821 18133 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:15:11.711923 18000 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:11.729954 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 101924us
W20260709 12:15:11.716897 18108 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.634s	user 0.000s	sys 0.004s
I20260709 12:15:11.717100 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:11.717322 18098 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:11.713172 18130 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.722163 18127 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.722755 17851 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:11.723515 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.736547 11177 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:11.728663 18084 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:11.728860 18120 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.732806 11935 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:15:11.704692 18045 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:11.787380 18122 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.789500 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:11.790469 11177 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:11.791942 18034 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:11.792436 18034 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:11.793306 18034 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.794975 18034 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:11.817793 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76330.2us
I20260709 12:15:11.819423 18084 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:11.822333 11177 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:11.822837 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:11.827196 11177 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:15:11.828073 11174 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:11.830009 11174 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:11.634923 17966 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:11.737142 17851 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:11.741802 17992 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:11.682523 18073 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.839764 18073 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.751888 17969 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:11.759696 11147 leader_election.cc:400] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21.
I20260709 12:15:11.768476 18054 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:11.841018 11180 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:11.841253 18054 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:11.733434 17987 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:11.843971 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 110208us
W20260709 12:15:11.841046 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 81564.5us
I20260709 12:15:11.775815 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1079) took 1273 ms. Trace:
I20260709 12:15:11.845454 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:11.777988 18144 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:15:11.845865 11933 rpcz_store.cc:276] 0709 12:15:10.502924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.503140 (+   216us) service_pool.cc:224] Handling call
0709 12:15:10.503589 (+   449us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:10.504074 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:10.504115 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:10.504705 (+   590us) log.cc:844] Serialized 52 byte log entry
0709 12:15:10.816689 (+311984us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:10.818971 (+  2282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:10.819098 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.716331 (+897233us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:11.775529 (+ 59198us) raft_consensus.cc:1685] finished
0709 12:15:11.775551 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.775754 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8994,"spinlock_wait_cycles":16634624,"thread_start_us":298124,"threads_started":1,"wal-append.queue_time_us":448909}
I20260709 12:15:11.730693 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:11.851209 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 120580us
I20260709 12:15:11.851884 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:11.836784 17966 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:11.773773 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:11.846865 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.854523 11329 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:11.847958 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:11.856264 11324 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:11.860311 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:11.861545 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:11.865998 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:11.872787 11177 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:15:11.879848 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 106179us
I20260709 12:15:11.882021 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:11.886497 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.855667 18000 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.871419 17498 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:11.895216 17800 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:11.895730 17800 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.896097 17800 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:11.907301 18120 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:11.913877 17992 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:11.917899 17987 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:11.920801 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:11.921835 11330 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:11.923858 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:11.853555 17861 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:15:11.858342 18020 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:11.927955 18020 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.928385 18020 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:11.930601 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3077) took 1052 ms. Trace:
I20260709 12:15:11.931053 11176 rpcz_store.cc:276] 0709 12:15:10.878125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.878335 (+   210us) service_pool.cc:224] Handling call
0709 12:15:11.930413 (+1052078us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54002571d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:11.930534 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390571904}
I20260709 12:15:11.862869 17893 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:11.935588 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53686.5us
I20260709 12:15:11.936758 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:11.943346 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74429.3us
I20260709 12:15:11.961596 11324 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:11.963053 11329 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:11.961532 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:11.964067 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:11.875624 18042 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:11.878587 17931 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:11.955585 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:11.961113 18147 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:11.966253 11223 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:11.974100 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:11.970633 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.978717 11250 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:11.970633 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:11.963407 17805 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:11.983997 11248 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:11.985571 11257 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:15:11.986016 11257 pending_rounds.cc:85] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 9
I20260709 12:15:11.986289 11257 pending_rounds.cc:107] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:15:11.981174 11935 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:11.986542 11257 raft_consensus.cc:2889] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:11.981999 17608 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:11.989111 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:11.992794 11928 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:11.971244 17858 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:12.004204 18127 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:11.979128 17861 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:11.980432 18042 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:11.965240 17782 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:15:11.972151 17931 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:12.006282 17608 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:11.978976 17893 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:12.075906 17782 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:12.076272 17893 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.076365 17782 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:12.076793 17893 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:12.078606 18042 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.999234 18039 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:12.084020 18067 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.085280 18067 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:12.010429 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.016685 18130 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.019690 11324 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:12.022954 18045 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:11.971328 18098 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 12:15:12.041725 18018 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:12.049721 18134 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:11.974509 18147 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:12.067006 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.064079 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:12.068183 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:11.988482 17805 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.074649 17931 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.009375 18000 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.087646 11327 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
W20260709 12:15:12.090843 18109 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.001s
I20260709 12:15:12.088136 17858 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:12.068284 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:12.097029 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.103456 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:12.119293 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51095.2us
I20260709 12:15:12.096592 18134 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.100710 17805 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:12.077361 17782 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.120704 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:12.117412 17787 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:12.127708 18134 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:12.129438 18054 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.099357 18147 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:12.143163 18020 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.122320 18127 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:15:12.124521 18094 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:12.168744 18094 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.146083 17931 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:12.165799 18102 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:12.097815 17991 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:15:12.097270 18028 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
W20260709 12:15:12.167301 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 191415us
I20260709 12:15:12.151346 17787 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:12.172394 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 176690us
I20260709 12:15:12.118510 17966 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:15:12.127353 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:12.186159 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:12.186338 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:12.186769 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59514.4us
I20260709 12:15:12.166033 11327 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:12.169039 17782 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:12.170560 18094 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:12.190279 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:12.195684 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.196457 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:12.196655 11325 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:12.198045 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.188634 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:12.199354 11324 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:12.206527 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58536.1us
I20260709 12:15:12.207361 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:12.210004 11328 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:12.175810 18102 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.206056 17893 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.217404 18020 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.191330 18047 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:12.193917 18034 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:12.223215 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:12.182941 18054 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.175153 17800 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.188591 17787 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:12.188798 17991 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:12.230198 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:12.230623 11324 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:12.231633 11935 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:12.205852 18048 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:12.188612 17858 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.213866 18109 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:12.226159 18102 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:12.208176 17861 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:15:12.244068 18109 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:12.250478 18109 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.255414 17800 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.238791 18034 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:12.256799 17787 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.257395 18034 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.259105 17787 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.262463 17893 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.271035 17941 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:12.275269 18053 raft_consensus.cc:993] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:12.276050 18053 raft_consensus.cc:1081] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:12.276726 18102 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.264376 18045 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.278257 18147 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.280859 18134 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.282915 18094 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.283516 17966 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:12.284142 18130 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:12.289981 18134 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.289889 17966 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:12.286101 18039 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:12.291507 17966 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.287529 18102 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.289963 18130 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:12.293251 17966 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:12.293867 18130 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.295702 18130 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:12.285254 18147 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.299903 18094 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.370572 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:12.381801 11173 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:12.384882 11256 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:15:12.437997 17902 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.439163 17902 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.444613 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:12.381317 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:12.469242 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:12.471930 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2574) took 1133 ms. Trace:
I20260709 12:15:12.472550 11253 rpcz_store.cc:276] 0709 12:15:11.337923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.340261 (+  2338us) service_pool.cc:224] Handling call
0709 12:15:12.471462 (+1131201us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b5400277254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:12.471693 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157238016}
I20260709 12:15:12.479259 18094 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:12.479986 18094 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:12.480580 18094 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:12.481242 11180 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:12.495638 11179 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:12.498507 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 298593us
I20260709 12:15:12.508118 18032 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:12.510406 18147 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.511535 18147 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:12.514335 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.516548 18130 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.518107 18025 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:12.519268 18025 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.520330 17805 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:12.524219 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87572.1us
I20260709 12:15:12.534495 18053 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:12.539387 18073 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:12.540235 18073 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.551993 11176 raft_consensus.cc:606] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:12.552637 11176 raft_consensus.cc:614] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:12.558115 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.566102 18159 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:12.567196 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:12.574932 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:12.537200 18053 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:15:12.584645 18070 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.597433 17787 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:12.598111 18166 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.597522 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:12.599552 18061 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.600907 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:12.603175 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:12.607299 11254 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:15:12.608760 11253 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:15:12.610395 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.611186 18000 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.611850 18017 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:15:12.613873 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.621935 11322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:15:12.624820 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1027) took 1126 ms. Trace:
W20260709 12:15:12.631254 18162 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.167s	user 0.003s	sys 0.000s
I20260709 12:15:12.635015 18147 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.638499 17902 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:12.630201 18165 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:15:12.517515 18130 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.653975 11178 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:12.659639 11933 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:12.662914 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 432526us
I20260709 12:15:12.666234 11931 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:12.649690 11255 rpcz_store.cc:276] 0709 12:15:11.498408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.498621 (+   213us) service_pool.cc:224] Handling call
0709 12:15:12.477852 (+979231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:12.478391 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:12.478416 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:12.478475 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:12.478510 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:12.478616 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:12.624429 (+145813us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b5400277250. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:12.624585 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2250874112}
I20260709 12:15:12.672183 18166 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.679514 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:12.679867 18130 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.680732 17966 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:12.688226 18070 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.688832 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:12.691758 18070 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.635885 18137 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:15:12.694267 18137 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:12.693948 18017 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.696571 18017 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.696252 18061 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.699016 18061 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:12.699379 18025 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.701495 18159 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:12.702178 18159 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:12.703567 11252 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:12.704643 17787 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.707265 11182 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:12.707379 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.691941 11934 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:12.692574 18000 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.712538 17851 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.713654 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.714278 11329 raft_consensus.cc:2368] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20.
I20260709 12:15:12.715783 11322 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:12.717023 11322 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:12.707535 17787 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.725845 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.589437 17801 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.127s
W20260709 12:15:12.739665 11935 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.015s
I20260709 12:15:12.742574 11328 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:12.740329 11935 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:12.694677 18137 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:12.691759 18166 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.746559 11324 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:12.747740 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.749722 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.752235 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.753139 18073 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:12.755354 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 99585.8us
I20260709 12:15:12.754436 17966 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:12.754055 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:12.759788 11182 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
W20260709 12:15:12.764385 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.767185 18137 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.768954 18137 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:12.762451 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:12.770207 11176 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:12.759279 17966 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:12.783593 11325 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:12.786718 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:12.781833 17966 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:12.782872 17805 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.810503 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:12.811367 11251 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:12.812453 18025 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.813133 18025 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.815400 18025 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.816649 18094 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.817958 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.818359 17801 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:12.818789 11329 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:12.819079 17801 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:12.819974 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.818367 17717 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.829381 17941 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.833201 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:12.833781 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 145146us
I20260709 12:15:12.833611 18085 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:12.834060 17805 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:15:12.850013 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.851508 18102 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.852172 18102 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.857942 11074 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:12.866150 18000 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:12.868012 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:12.868242 11325 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:12.869748 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.883008 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.872186 11331 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:12.872958 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:12.894671 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 48147.2us
I20260709 12:15:12.895393 17992 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:12.897264 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.898720 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.873831 18017 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:12.847313 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:12.901273 18017 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.904099 18017 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:12.893323 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:12.870790 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:12.877704 18070 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:12.877790 17941 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.883513 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:12.848770 11322 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:12.880086 17851 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:12.914156 11324 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:12.907195 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:12.919543 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:12.923337 18061 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:12.923911 18061 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.924330 18061 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:12.930428 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:12.931290 11177 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:12.933916 11322 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:12.933921 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 161934us
I20260709 12:15:12.936667 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:12.939149 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:12.939395 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:12.947857 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:12.948719 11182 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:12.954746 11934 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:12.913043 18070 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:12.949118 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 172514us
I20260709 12:15:12.951198 11331 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:12.968199 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:12.998407 11176 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:12.941289 18073 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.950719 17992 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:12.929283 17966 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.016120 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:13.021430 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.034817 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:13.032248 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:13.042552 11254 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:13.046801 11329 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:13.014995 17717 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.022859 17992 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:13.051424 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.054237 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:13.025254 17801 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:13.059633 11931 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:15:13.058678 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:13.064685 17717 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.067021 17717 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.068003 18061 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.068470 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:13.069851 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:13.070533 11179 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:13.071511 11148 leader_election.cc:400] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20.
I20260709 12:15:13.071854 11148 leader_election.cc:403] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:15:13.072742 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.062844 11178 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:13.073603 11249 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:13.082019 11330 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:13.082242 18045 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:15:13.097543 18045 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:13.098019 18045 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:13.097182 11074 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 9 to 16, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 16 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:13.098999 18045 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.100458 18045 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:13.100214 11074 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 9 to 20. New cstate: current_term: 20 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.082427 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:13.083834 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.083495 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:13.085139 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:13.029207 18073 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.085328 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:13.123368 11929 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:13.128644 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45321.4us
I20260709 12:15:13.084868 18104 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.129923 18104 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.132344 18104 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.085417 18084 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:13.085227 17498 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.113s
I20260709 12:15:13.134217 17498 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:13.134626 17498 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:13.135766 17498 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.137205 17498 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:13.139284 18102 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:15:13.140004 18102 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:13.140429 18102 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:13.140861 18102 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:13.142306 18025 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:13.057039 17992 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:13.144845 11074 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.146843 17787 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:13.147394 17787 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.147770 17787 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:13.147084 11074 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.090127 17340 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:13.087085 17941 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.126927 18159 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:13.152853 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63312.9us
I20260709 12:15:13.153837 18159 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:13.153936 17966 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:13.154349 17717 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.155001 17717 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.156368 18073 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:13.157119 17717 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.157613 17608 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.086234 18070 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.159425 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:13.159936 17805 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:13.158215 17608 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.163139 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.163932 11249 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:13.163163 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.165550 11251 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:13.160116 11180 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:13.149247 11074 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:13.165622 11078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 244) took 57437 ms (client timeout 15000 ms). Trace:
I20260709 12:15:13.178004 17803 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:13.178812 11074 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.093422 18094 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:13.184226 18085 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:15:13.174153 11078 rpcz_store.cc:269] 0709 12:14:15.727686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.742027 (+ 14341us) service_pool.cc:224] Handling call
0709 12:15:13.165338 (+57423311us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:09.680395 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:09.681081 (+   686us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:09.681137 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:15:09.681155 (+    18us) tablet.cc:662] Decoding operations
0709 12:15:09.770440 (+ 89285us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:09.774932 (+  4492us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:09.791281 (+ 16349us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:15:09.874315 (+ 83034us) tablet.cc:701] Row locks acquired
0709 12:15:09.874344 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:15:09.900897 (+ 26553us) write_op.cc:270] Start()
0709 12:15:10.041013 (+140116us) write_op.cc:276] Timestamp: P: 1783599309894364 usec, L: 0
0709 12:15:10.041046 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:15:10.241234 (+200188us) log.cc:844] Serialized 1701 byte log entry
0709 12:15:11.168709 (+927475us) op_driver.cc:464] REPLICATION: finished
0709 12:15:11.628192 (+459483us) write_op.cc:301] APPLY: starting
0709 12:15:11.632562 (+  4370us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:11.780543 (+147981us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:11.780649 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:12.010779 (+230130us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:12.010992 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:15:12.011046 (+    54us) write_op.cc:312] APPLY: finished
0709 12:15:12.011301 (+   255us) log.cc:844] Serialized 49 byte log entry
0709 12:15:12.011499 (+   198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:12.011617 (+   118us) write_op.cc:454] Released schema lock
0709 12:15:12.011663 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:12.012113 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:12.012144 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":45324,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":441974,"apply.run_cpu_time_us":8845,"apply.run_wall_time_us":393928,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20879,"num_ops":4,"prepare.queue_time_us":491302,"prepare.run_cpu_time_us":29762,"prepare.run_wall_time_us":689060,"raft.queue_time_us":473497,"raft.run_cpu_time_us":14912,"raft.run_wall_time_us":421552,"replication_time_us":1063303,"thread_start_us":1253925,"threads_started":4,"wal-append.queue_time_us":316552,"wal-append.run_cpu_time_us":35538,"wal-append.run_wall_time_us":2517952}]]}
I20260709 12:15:13.166776 11930 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:13.168464 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.169251 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:13.187572 11176 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:13.184240 11074 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 17, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 17 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:13.251772 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 98171.2us
I20260709 12:15:13.252393 11182 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:13.253659 11934 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:15:13.254195 11934 pending_rounds.cc:85] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 13
W20260709 12:15:13.255177 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 122083us
I20260709 12:15:13.168946 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.262310 11331 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:15:13.262916 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.184684 18085 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:13.182823 18084 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:13.168601 18070 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:13.254522 11934 pending_rounds.cc:107] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:15:13.268514 11178 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:13.168076 17608 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:13.280144 11934 raft_consensus.cc:2889] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:13.276774 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:13.275794 18084 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:13.304677 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:13.322029 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:13.323309 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:13.306928 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:13.305320 17851 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:13.324793 17851 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:13.326202 18073 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:13.326752 18104 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:13.327128 18073 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:13.327436 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:13.328460 18073 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:13.329933 18171 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:13.330353 11331 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:13.330507 18171 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:13.331652 18171 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:13.333067 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 462412us
I20260709 12:15:13.276714 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:13.334929 11176 raft_consensus.cc:606] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:13.336190 18134 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:15:13.337291 18134 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:13.337771 18134 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:13.337574 17902 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:13.338912 18134 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.319732 18085 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.340438 18085 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:13.332123 18061 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.339203 18053 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:13.347536 17992 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.325850 17851 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.348812 11928 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:13.272962 18025 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:15:13.354781 18102 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.296233 11254 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:13.314009 17340 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20.
I20260709 12:15:13.296713 11074 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 11 to 16, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 16 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:13.370483 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:13.375589 11324 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:13.378216 11331 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:13.384025 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:13.384902 11322 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:13.385958 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:13.393443 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:13.394593 11253 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:13.395197 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:13.396739 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:13.398057 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:13.401933 18134 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:13.405858 17803 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:13.393460 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:13.374085 18102 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:13.407994 11074 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:13.423532 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:15:13.419672 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:13.423961 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:13.424702 11176 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:13.417618 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:13.409487 17992 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:13.420022 18025 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:13.412873 18017 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:13.436131 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.409149 17608 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:15:13.433056 18104 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:13.423962 17931 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:13.449544 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:13.442515 11074 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 17, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 17 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:13.440075 18153 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:13.479852 11178 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:13.453222 17931 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:13.453914 17941 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:13.485144 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:13.453085 18017 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:13.485352 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:13.489692 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:13.491268 11322 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:13.475293 18061 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:13.494511 17931 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.510277 11327 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:13.488333 11074 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.494518 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:13.521191 11329 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:13.552368 11074 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 20, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:13.520334 17787 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:13.581789 18177 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 12:15:13.589931 18053 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:13.548039 18000 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:13.603399 11176 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:13.609614 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1114) took 1237 ms. Trace:
W20260709 12:15:13.610868 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.534704 18147 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:13.572618 17931 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:13.610442 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:13.620702 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.621454 11254 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:13.622733 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.554149 18054 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:13.639988 18104 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:13.640518 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:13.641203 17782 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:13.642177 17902 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:13.642602 17902 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.643002 17902 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:13.640733 18104 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.645355 18104 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.647598 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:13.602643 17717 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.549252 18061 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.658257 11074 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.667492 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:13.670471 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 310629us
I20260709 12:15:13.659471 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.672443 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:13.610177 11173 rpcz_store.cc:276] 0709 12:15:12.372650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.383447 (+ 10797us) service_pool.cc:224] Handling call
0709 12:15:12.384424 (+   977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:12.739220 (+354796us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:15:12.998842 (+259622us) log.cc:844] Serialized 14 byte log entry
0709 12:15:13.392233 (+393391us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:13.448797 (+ 56564us) log.cc:844] Serialized 26 byte log entry
0709 12:15:13.448964 (+   167us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:13.448998 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:13.449093 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:13.603558 (+154465us) raft_consensus.cc:1685] finished
0709 12:15:13.603577 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:13.609513 (+  5936us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22162,"prepare.queue_time_us":296714,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3598,"raft.queue_time_us":184204,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":304,"thread_start_us":877394,"threads_started":3,"wal-append.queue_time_us":500207}
I20260709 12:15:13.637761 17941 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:13.679420 11249 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:15:13.676959 11074 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 10 to 22, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 22 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:13.610069 18053 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:13.682973 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 265820us
I20260709 12:15:13.621572 18084 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.692358 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:13.684543 11074 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 8 to 20, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:13.660413 18147 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:13.704687 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.707634 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.677764 18153 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:13.712194 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.668696 17717 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.718168 11329 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:13.683004 18025 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:13.730963 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:13.732280 11254 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:13.696482 18104 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:13.732708 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:13.768677 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:13.672470 18017 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.773325 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:13.681667 17902 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.774413 11254 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:13.786111 18174 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
W20260709 12:15:13.787845 18183 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 12:15:13.789523 18187 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:15:13.791447 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:13.740913 18179 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:13.767164 18181 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:15:13.810134 11074 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 15 to 16, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 16 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.737133 18171 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:13.785769 18185 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:15:13.774507 18104 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.823424 18104 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:13.831643 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:13.831696 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.832609 11250 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:13.778282 18045 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:13.707958 17941 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.866029 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:13.870071 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:13.870960 11329 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:15:13.821214 18205 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.804883 18102 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:13.890314 17861 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:13.890813 17861 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.891170 17861 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:13.894045 18204 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:15:13.897782 18198 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:13.904820 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:13.906175 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:13.855046 18045 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.908970 11253 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:13.907428 11074 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 7 to 15, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 15 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.910714 11250 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:13.912024 11252 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:13.911670 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 179050us
I20260709 12:15:13.863667 18130 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:13.872417 18147 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.933794 11254 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:13.873917 17787 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.924643 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:13.939893 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 201437us
I20260709 12:15:13.918506 18045 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:13.916496 18017 raft_consensus.cc:2764] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:15:13.917369 18094 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:13.860949 18179 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.969198 11074 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:13.976126 18070 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.696s	user 0.006s	sys 0.007s
I20260709 12:15:13.966691 18147 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:15:13.966889 18130 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:13.955528 18102 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.998176 18208 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:13.999114 18208 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.001166 18201 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:14.001772 18201 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.002362 18201 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:14.006742 18061 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:13.923931 18205 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.958489 18025 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.010668 18205 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:14.014098 18203 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:15:13.985728 18217 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.012619 18205 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:13.911931 18153 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:13.984068 18084 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:14.027032 11074 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:14.028038 18217 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.029806 18196 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.030589 18196 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.041194 11329 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:13.984402 18094 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.048678 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.058027 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.062037 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.064075 17941 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.064851 17941 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:14.065286 17941 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:14.066406 17941 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.068804 18179 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:14.072877 18134 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.002s
I20260709 12:15:14.074415 18054 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:14.075065 18094 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:14.080735 18104 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.081954 11173 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:14.094765 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 172449us
I20260709 12:15:14.047935 18208 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:14.156442 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 191532us
I20260709 12:15:14.051756 17902 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.206379 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:14.054031 18102 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:14.056154 18217 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.215798 11327 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:14.222012 17805 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:14.230700 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.098760 18054 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:14.230002 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.248906 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:14.218493 18094 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.246302 11327 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:15:14.246444 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.265146 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:14.263090 11180 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:14.262243 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.269495 18199 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:14.270283 18199 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.272817 18199 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.273821 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.273958 17805 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:14.274511 11255 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:15:14.274775 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.275486 11182 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:14.275177 17805 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.275810 11182 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:14.276170 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.276630 11182 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.264672 18134 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:15:14.277697 11182 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:14.225457 18147 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:14.279089 18147 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.282130 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.283118 11173 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:14.285805 11176 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:14.232168 18130 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.289875 11331 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:14.291352 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:14.294613 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:14.298674 11255 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:14.300036 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.286804 18147 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.299204 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3110) took 1564 ms. Trace:
W20260709 12:15:14.308287 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.227407 17861 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.316880 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.319514 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.317126 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:14.323218 11328 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:14.324465 11182 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:14.291930 18134 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:14.329999 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.333981 11329 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:14.337450 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.338604 11328 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:14.309891 17498 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.348776 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.313970 11326 rpcz_store.cc:276] 0709 12:15:12.730647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.753234 (+ 22587us) service_pool.cc:224] Handling call
0709 12:15:12.753602 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:12.784877 (+ 31275us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:13.031906 (+247029us) log.cc:844] Serialized 14 byte log entry
0709 12:15:13.405147 (+373241us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b1800296c80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:13.731253 (+326106us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b1800296c80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:15:13.741161 (+  9908us) log.cc:844] Serialized 14 byte log entry
0709 12:15:14.290679 (+549518us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b1800296c80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:15:14.290900 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 12:15:14.294357 (+  3457us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:14.294417 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:14.294470 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.294539 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:14.294662 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":460918,"prepare.queue_time_us":1115422,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":136989,"raft.run_cpu_time_us":469,"raft.run_wall_time_us":33837,"spinlock_wait_cycles":46904576,"thread_start_us":614883,"threads_started":3,"wal-append.queue_time_us":487461}
I20260709 12:15:14.358294 11180 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
W20260709 12:15:14.358784 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67543.4us
W20260709 12:15:14.441498 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48303.9us
I20260709 12:15:14.366998 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:14.442625 11931 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:14.365366 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:14.382017 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.361250 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.320699 18094 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.339334 18134 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.380330 18045 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.361372 17498 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.368114 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:14.363705 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:14.384158 18070 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:14.470165 18070 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.385075 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.366297 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:14.361231 18229 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:14.341507 18199 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:14.380470 18180 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.377022 11932 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:15:14.485939 18180 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.379525 18054 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.488282 18180 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:14.488773 18054 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.387933 18201 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.490023 17782 raft_consensus.cc:993] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: : Instructing follower fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:15:14.490567 17782 raft_consensus.cc:1081] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Signalling peer fecfc03de52a4308bc9b045cad5aac1e to start an election
I20260709 12:15:14.491240 18201 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:14.391727 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.492594 18070 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:14.386114 18196 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.367496 11175 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
W20260709 12:15:14.389025 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.373549 18213 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:14.396819 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71762.3us
I20260709 12:15:14.394986 17717 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.438544 18179 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:14.500618 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:14.501348 18201 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:14.500883 18179 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.445158 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:14.502681 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57581.8us
I20260709 12:15:14.503376 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:14.503814 18025 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:14.504268 18179 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.359112 11176 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:14.505219 18025 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:14.505956 18025 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:14.506337 18025 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:14.507411 18224 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.507750 11323 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:15:14.507972 18224 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.505281 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:14.509758 18130 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:14.365540 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.476240 11930 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:14.478565 17498 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.480549 17801 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.511956 17801 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.390173 17861 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.515384 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:14.515844 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:14.516693 11254 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:14.516955 17801 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.517522 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.517577 18179 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.446884 18137 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.522006 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.522346 18137 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.524883 18137 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.525336 18179 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.527444 18179 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.529237 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.361531 18104 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.496760 11178 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:14.501751 18201 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.530699 11929 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:14.531000 18201 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:14.485414 11927 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:14.521524 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.531812 11323 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:15:14.496112 11931 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:14.533051 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.509478 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:14.533985 11329 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:15:14.474995 11331 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
W20260709 12:15:14.540683 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.498879 18190 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.547629 18190 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.534003 11254 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:15:14.567348 11329 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:14.549520 18094 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.574980 11929 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:14.582912 11930 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
W20260709 12:15:14.584185 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55147.1us
I20260709 12:15:14.597025 11926 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:14.603153 18196 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.603904 18196 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.605854 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.607229 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.608007 11331 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:14.609036 18053 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.609894 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.612040 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.578603 18179 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:14.583609 17782 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.619192 11927 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:14.619513 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.615422 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:14.598325 18070 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.628429 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.628609 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.629278 11175 raft_consensus.cc:2368] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20.
I20260709 12:15:14.629980 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.629309 11180 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:14.631141 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.609983 18094 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.612884 18197 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:14.637858 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.638962 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.638005 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1100) took 1785 ms. Trace:
I20260709 12:15:14.638532 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.640501 11926 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:14.644421 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.645506 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.649320 11179 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:14.639581 11935 rpcz_store.cc:276] 0709 12:15:12.867056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.867275 (+   219us) service_pool.cc:224] Handling call
0709 12:15:12.867702 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:12.868114 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:13.058523 (+190409us) log.cc:844] Serialized 14 byte log entry
0709 12:15:13.447658 (+389135us) log.cc:844] Serialized 14 byte log entry
0709 12:15:14.291276 (+843618us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b1800488c20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:15:14.468580 (+177304us) log.cc:844] Serialized 14 byte log entry
0709 12:15:14.621618 (+153038us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:14.621706 (+    88us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:14.621771 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.622147 (+   376us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:14.631424 (+  9277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87626,"prepare.queue_time_us":649795,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":2327,"raft.queue_time_us":872945,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":11598,"spinlock_wait_cycles":31548672,"thread_start_us":872904,"threads_started":4,"wal-append.queue_time_us":507699}
I20260709 12:15:14.649545 11928 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:14.570598 18229 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.639564 11933 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:14.624397 18053 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:14.651772 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.641384 18070 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.648538 18104 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.644310 18094 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.648975 18025 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.644462 18197 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.686707 18104 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.690260 18025 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.691300 18197 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:14.691131 18229 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.691708 18104 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.692420 11927 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:14.627856 18179 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.694753 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:14.689117 18094 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:14.695992 18229 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:14.696202 18179 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:14.697221 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.697381 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:14.697804 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.632894 18147 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.618080 18190 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.698690 18147 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.701269 18147 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.643656 17782 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.702730 18245 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:14.715379 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.704182 18239 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.719976 11181 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:14.720557 18218 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:14.721863 18234 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.722512 18234 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.709822 18137 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:14.725293 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.726104 11926 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:14.730127 18215 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.730996 18215 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:14.732328 18192 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.687s	user 0.002s	sys 0.001s
I20260709 12:15:14.732555 11182 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
I20260709 12:15:14.726624 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.768389 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.714143 18196 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:14.710995 18231 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:15:14.769194 11179 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:14.780157 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.776880 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.781361 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.782801 11930 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:14.721249 18218 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.778435 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.783918 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.780686 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:14.786499 11328 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:15:14.779686 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.711055 11176 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
W20260709 12:15:14.786957 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 478754us
I20260709 12:15:14.789412 11928 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:14.792356 11180 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:14.796013 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:14.797348 11932 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:14.712678 18201 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:14.794203 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 477135us
I20260709 12:15:14.802371 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:14.805020 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 189659us
I20260709 12:15:14.800338 18134 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.785764 17931 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.814131 11180 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:14.816538 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:14.774175 18053 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.801553 18200 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:14.824115 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:14.801656 18045 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.772223 18239 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.845867 11328 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:15:14.846165 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:14.835824 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:14.846951 11255 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:14.851461 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:14.854007 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.855139 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:14.856981 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.857676 11329 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:14.855736 11928 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:14.862229 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.862670 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:14.863746 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:14.864301 11251 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:14.829890 18200 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.826623 18134 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:14.865828 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3119) took 1391 ms. Trace:
I20260709 12:15:14.865881 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:14.874414 11328 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:15:14.866505 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.867480 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:14.826455 18220 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:14.869262 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.870951 11250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362"
dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:33798
I20260709 12:15:14.881249 11250 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:14.881685 11250 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:14.878994 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:14.883770 11253 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:14.879489 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.879711 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.884213 11176 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:14.822531 17931 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.872948 11322 rpcz_store.cc:276] 0709 12:15:13.474596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.492287 (+ 17691us) service_pool.cc:224] Handling call
0709 12:15:13.515684 (+ 23397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:13.519847 (+  4163us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:14.045074 (+525227us) log.cc:844] Serialized 14 byte log entry
0709 12:15:14.288144 (+243070us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:14.341022 (+ 52878us) log.cc:844] Serialized 26 byte log entry
0709 12:15:14.352713 (+ 11691us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:14.352753 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.352827 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:14.865485 (+512658us) raft_consensus.cc:1685] finished
0709 12:15:14.865517 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:14.865744 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":260492,"prepare.queue_time_us":772294,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":2301,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":14443,"thread_start_us":237774,"threads_started":2,"wal-append.queue_time_us":251193}
I20260709 12:15:14.859685 17805 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:14.852149 17861 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:14.882790 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:14.897421 11932 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:14.898319 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.875365 18200 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:14.853524 17717 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:15:14.904165 18224 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.900516 18179 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:14.905966 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.872799 18134 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:14.905033 18179 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.907618 11176 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:14.933935 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.934412 11328 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:14.910683 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:14.913136 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.914765 18053 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.915217 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.915876 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:14.916676 18215 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.917101 17931 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.917675 18134 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:14.901732 18025 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:14.971243 11932 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:14.921305 18239 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.887236 18054 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:14.922410 11181 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:14.897148 17805 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.938557 11180 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:14.936733 11323 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 15.
W20260709 12:15:14.940763 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 144777us
I20260709 12:15:14.983047 17805 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:14.982988 18054 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.947952 11900 leader_election.cc:400] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20.
I20260709 12:15:14.969475 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:14.985957 11900 leader_election.cc:403] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:15:14.986212 11178 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:14.988610 18257 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:15:14.970270 11175 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:14.989684 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:14.990091 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2730) took 1082 ms. Trace:
I20260709 12:15:14.895365 18196 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:15:14.972026 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:14.991120 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:14.991606 18254 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:14.992087 18254 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:14.992213 18249 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:14.992538 18254 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:14.993837 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:14.994017 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:14.994215 18045 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.990509 11330 rpcz_store.cc:276] 0709 12:15:13.908151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.916493 (+  8342us) service_pool.cc:224] Handling call
0709 12:15:13.916883 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:14.025682 (+108799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:14.025804 (+   122us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:14.040228 (+ 14424us) log.cc:844] Serialized 26 byte log entry
0709 12:15:14.756883 (+716655us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b18003b02c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:14.804107 (+ 47224us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:14.804623 (+   516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.804734 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:14.989772 (+185038us) raft_consensus.cc:1685] finished
0709 12:15:14.989795 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:14.990024 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38423,"raft.queue_time_us":176546,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":7689,"spinlock_wait_cycles":14145792,"thread_start_us":147909,"threads_started":2,"wal-append.queue_time_us":817696}
I20260709 12:15:14.994863 18212 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:14.994727 18045 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.996930 18045 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:14.998535 17805 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:14.974184 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:15.001917 11177 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:15.003005 17805 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:14.967713 11250 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.004766 18212 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:14.994701 11181 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:15.007366 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:15.009095 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 016411114c6d42039bd04025378dfc98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:15.010700 11181 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:15.011286 18234 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.012570 18054 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.013305 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:15.013605 11175 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:15.015810 11180 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:15:15.017349 11180 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:15:15.018106 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.018714 11330 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:15.020215 18196 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.020107 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:15.021039 11180 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:15.021402 11180 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:14.938257 18246 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:15.022179 11180 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.022749 18201 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.020941 18196 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:14.975925 11253 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:14.980371 11255 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:15.028864 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.023159 11180 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:15.024796 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 85038.9us
I20260709 12:15:15.032512 11251 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:15.066134 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.066273 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:15.068460 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.069089 11323 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:15:15.050567 11178 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:15.069381 11328 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:15.071229 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:15.071962 11175 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:15.067332 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:15.073544 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:15.040935 18252 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:15:15.076843 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:15.077374 11328 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:15.058014 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.083786 11330 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:15.085616 11177 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:15.049039 17931 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20.
I20260709 12:15:15.094551 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.090742 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.041976 18179 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:15.104948 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.083348 18234 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:15.119526 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:15.122629 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 236654us
I20260709 12:15:15.048415 18025 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.089445 18188 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:15.089717 18218 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.022204 18246 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.113832 18094 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:15.090117 18201 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:15:15.112429 17931 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:15.148605 11175 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:15.144105 18253 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:15.160494 18256 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:15.141618 18234 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:15.165148 11180 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:15.110782 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:15.127306 18205 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:15:15.195238 11328 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:15.206287 18137 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.207109 18137 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:15.220535 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 132263us
W20260709 12:15:15.221004 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 110352us
I20260709 12:15:15.222136 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.227006 11934 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:15.226296 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:15.189926 18253 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:15.229352 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72351.1us
I20260709 12:15:15.145907 18094 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.235965 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:15.225322 18190 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:15:15.220681 18261 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:15:15.279997 18094 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:15.280834 18196 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.236006 18188 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:15.288138 18188 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.288546 18188 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:15.255621 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:15:15.293372 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67398.8us
I20260709 12:15:15.237183 18253 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:15.327852 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 91966.7us
I20260709 12:15:15.329741 18263 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:15.331662 18054 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:15.329210 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:15.331985 18272 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:15.332430 18054 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:15.332659 18263 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:15.333209 18263 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:15.334195 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 78761.8us
I20260709 12:15:15.236413 18248 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.289s
I20260709 12:15:15.233112 17931 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:15.235837 17498 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.090s
I20260709 12:15:15.236282 18229 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.095s
I20260709 12:15:15.342711 18025 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:15.346663 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.349365 17498 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:15.349876 17498 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:15.350829 17498 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.354370 17861 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.355238 17861 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.356081 18254 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.337361 17805 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:15.358506 17805 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:15.359176 18262 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:15:15.359746 17805 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.360719 17941 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:15.362782 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.405s	user 0.019s	sys 0.029s
I20260709 12:15:15.365190 18061 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:15.365989 18229 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:15.366420 18229 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:15.367369 18229 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.341075 18263 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.334200 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:15.373656 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:15.299502 18201 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:15:15.377774 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:15.382546 18253 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.332667 18272 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:15.374073 18197 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.002s
I20260709 12:15:15.386101 18197 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.386613 11250 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.387377 11174 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:15.388352 18264 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:15:15.389394 17782 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:15.368625 18234 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.393252 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:15.394150 18197 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:15.394471 18246 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:15.395007 18137 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.396668 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:15.397308 11931 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:15.400528 11928 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:15.300810 18190 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:15.371863 11222 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:15.402760 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.445s	user 0.019s	sys 0.029s
I20260709 12:15:15.404048 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:15.397210 18137 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:15.414193 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:15.415064 11926 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:15.373175 18248 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:15.336413 18134 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.372298 17931 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:15.433885 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69132.6us
I20260709 12:15:15.296869 18200 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.424566 18245 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:15.417035 18201 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:15.333590 18054 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.450762 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.452291 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:15.459589 11926 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:15.423161 17861 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.465504 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.474088 17861 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:15.476644 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.478297 17861 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:15.485617 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50568.8us
W20260709 12:15:15.489040 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.424863 18248 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:15.436133 18266 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:15.481753 18264 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:15:15.497821 18254 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.498494 18272 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:15.502480 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:15.505551 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:15.520074 11929 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:15.499190 18245 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:15.489697 18201 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:15.523540 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:15.506127 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:15.533253 11926 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:15.535272 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:15.539527 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.540515 11330 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
W20260709 12:15:15.541646 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.547417 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:15.547689 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:15.551465 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 44544.4us
I20260709 12:15:15.552711 11250 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:15.554524 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:15.555480 18198 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:15.548900 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:15.558569 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:15.560019 18225 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:15.561046 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:15.562650 18226 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:15.563576 18226 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:15.566120 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 66962.2us
I20260709 12:15:15.567304 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.552369 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.571077 18263 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:15.571421 18094 raft_consensus.cc:2764] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:15:15.511194 17931 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.580574 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.526994 18179 raft_consensus.cc:2764] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:15:15.586982 11252 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:15:15.587589 11252 pending_rounds.cc:85] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 11
I20260709 12:15:15.587888 11252 pending_rounds.cc:107] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:15:15.588197 11252 raft_consensus.cc:2889] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:15.520486 18190 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:15.601042 11147 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.598007 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:15.602212 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.602885 11330 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
W20260709 12:15:15.575953 18261 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:15:15.554158 17782 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:15.605340 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:15.610042 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:15.617750 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:15.618587 11926 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:15.555126 18017 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:15.516713 18266 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.579386 18198 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:15.590849 18225 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:15.593994 18070 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.595851 18226 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:15.595674 18200 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:15.613704 18213 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:15.627379 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.633785 11325 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:15.634845 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.635525 11323 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:15.604019 18245 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.612526 17902 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:15:15.626047 18256 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:15.640741 18256 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:15.642055 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:15.641881 18256 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.642729 11256 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:15.648219 18134 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:15.625947 18137 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:15:15.650132 18270 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.373s	user 0.000s	sys 0.002s
I20260709 12:15:15.650960 18266 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:15.651131 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:15.654292 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1108) took 1048 ms. Trace:
W20260709 12:15:15.649812 18268 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.169s	user 0.000s	sys 0.004s
I20260709 12:15:15.656894 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:15.654726 11934 rpcz_store.cc:276] 0709 12:15:14.605855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.610735 (+  4880us) service_pool.cc:224] Handling call
0709 12:15:15.226792 (+616057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:15.227438 (+   646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.654088 (+426650us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54004c2550. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:15.654214 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416072448}
I20260709 12:15:15.622641 18017 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:15.678053 11256 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:15.624419 18225 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.659927 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:15.681151 11251 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:15:15.677379 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 110204us
I20260709 12:15:15.683580 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:15.684806 18266 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:15.685994 11296 leader_election.cc:400] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:15.686472 11296 leader_election.cc:403] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:15:15.598430 18201 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.697062 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.681340 18017 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:15.687885 18266 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
W20260709 12:15:15.794044 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 96719.5us
I20260709 12:15:15.595985 18248 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:15.787963 11175 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:15:15.738595 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:15.688288 18225 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:15.800618 17340 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:15.800945 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 103960us
I20260709 12:15:15.688467 18025 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:15:15.700981 18104 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:15:15.820742 11178 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:15.822428 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:15.815423 11177 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:15.813246 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:15.824421 11178 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:15.828016 11926 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:15.831401 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:15.833187 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:15.835773 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:15.835795 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:15.838003 18200 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:15.853379 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:15.810196 18282 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:15:15.859593 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:15.840183 18266 raft_consensus.cc:2764] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:15:15.866415 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:15.867513 11178 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:15.869946 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:15.870846 11175 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:15.894330 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:15.894999 11323 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:15.895108 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.898487 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:15.900964 18245 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:15.901793 18254 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:15.905563 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.868217 18200 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:15.906339 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:15.908943 11177 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:15.908533 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:15.913311 11325 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:15:15.914146 11326 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:15.888788 18054 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:15.917611 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:15.919582 18205 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:15.903012 18209 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:15.906013 18070 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:15.903615 18094 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:15.902809 18180 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:15.884793 18280 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:15.916515 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:15.931227 11175 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:15.964370 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.934110 18205 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:15.933851 18209 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:15.967244 17941 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:15.961970 18025 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:15.963737 18094 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:15.981611 11928 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:15.981840 11257 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:15.964761 18280 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:15.989542 11325 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:15:15.990358 18283 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:15:15.984310 18209 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:16.003130 18286 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:15:16.004046 18209 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:16.000118 17941 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:16.007400 18188 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.005s
I20260709 12:15:16.007085 17941 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.007836 18188 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.009034 11927 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:15.996898 17787 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.009495 18198 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:16.010217 18094 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.010310 17787 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.012009 18198 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:16.015133 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:15.974761 17902 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:15.994081 18284 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:15:16.001444 18254 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.035807 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:16.036886 11174 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:16.047292 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 211577us
I20260709 12:15:16.048105 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:16.000377 18134 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:16.051721 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 135307us
W20260709 12:15:16.058519 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.999626 18025 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.028568 17902 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.054875 18153 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.022012 18200 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:16.085873 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77742.8us
I20260709 12:15:16.050827 18254 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.087123 18282 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:16.095811 11180 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:15:16.088161 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:16.098289 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 1126) took 1474 ms (client timeout 1900 ms). Trace:
W20260709 12:15:16.098897 11182 rpcz_store.cc:269] 0709 12:15:14.615568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.622274 (+  6706us) service_pool.cc:224] Handling call
0709 12:15:16.090226 (+1467952us) spinlock_profiling.cc:243] Waited 93.1 ms on lock 0x7b5400347bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:16.090350 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214073216}
I20260709 12:15:16.089462 18153 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.073446 18134 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.086925 18190 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.084980 18188 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.088697 18054 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:16.112357 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64332.2us
I20260709 12:15:16.052686 18205 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.119841 18292 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:16.120801 18094 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:16.121048 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:16.121284 18094 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:16.122359 18094 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.123716 18094 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:16.127552 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.130277 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.130522 11174 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:16.133206 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.133960 11932 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:16.134156 18188 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:16.135393 18070 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:16.136023 18070 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.138753 17902 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.138000 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.139591 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.139868 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:16.140185 18025 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:16.141523 18070 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.144943 18258 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:16.145416 18258 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:16.146275 18258 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.135562 18188 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.151839 18188 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.151894 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.152210 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.152988 11182 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:16.151582 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.153503 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.154176 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.155810 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.157544 18134 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:16.151682 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.146385 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.160562 11932 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:16.158284 17787 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:16.161476 18293 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:16.161963 18293 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:16.162387 18293 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:16.162082 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:16.163784 18247 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:16.166867 18247 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:16.167382 18247 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:16.167778 18247 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:16.170578 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:16.173810 18070 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:16.174535 18070 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:16.096683 18280 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.175762 18070 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.177573 18266 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:16.178143 18266 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:16.179044 18266 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.180284 18266 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:16.176411 18263 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:16.160247 11928 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:16.186137 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.187294 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:16.107209 18282 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:16.201042 11323 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:16.208325 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.215525 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.223500 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.223251 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.147409 18258 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:16.229314 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.230557 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.232542 11177 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:15:16.233855 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.234628 11182 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:16.223251 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.224119 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.226543 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.228055 11257 raft_consensus.cc:2368] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18.
I20260709 12:15:16.240370 11174 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:16.243897 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.245150 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.196882 18263 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:16.213541 18254 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:16.253408 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 150261us
I20260709 12:15:16.112319 18190 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.258414 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.259536 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:16.256325 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50278.6us
W20260709 12:15:16.256680 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4157450909999625s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 12:15:16.261916 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3536450909998621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 12:15:16.262463 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3536450909998621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 12:15:16.263067 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2654450909999468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 12:15:16.263514 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2492450909999206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 12:15:16.266809 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2168450909998683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 12:15:16.267280 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1259450909999487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260709 12:15:16.267683 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1259450909999487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260709 12:15:16.268117 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1133450909999283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:15:16.268512 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0233450910000101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
I20260709 12:15:16.264339 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:16.274698 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.279307 11181 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:16.280081 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:16.258770 18094 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:16.285837 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.286473 11254 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:16.284040 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:16.287408 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:16.288246 18302 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:16.304908 18190 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.253268 18225 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.313943 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.318136 18153 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.318768 18147 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:16.262478 18054 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.288614 18304 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:16.320526 18304 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:16.320905 18304 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:16.322624 11176 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 19. Ops: [17.9-17.9]
I20260709 12:15:16.324096 18225 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:16.326265 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.327338 18253 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.327504 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:16.332566 11324 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:16.292498 18282 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.328647 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.336273 11176 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:16.331547 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:16.338362 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:16.336679 11181 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:16.341558 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:16.342182 11323 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:16.351567 18247 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.356256 18247 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.356387 11324 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:15:16.361971 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.367724 11323 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:16.349440 18294 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
W20260709 12:15:16.371476 18017 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.003s
I20260709 12:15:16.371941 18017 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.373227 11251 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:16.338928 18296 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:16.373492 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.374840 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.374920 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.375560 11253 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:16.376145 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:16.376439 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.376901 11251 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:16.377074 11253 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:16.378870 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:16.369532 17498 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:16.379799 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:16.380893 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:16.381774 18085 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:16.382408 18070 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:16.381855 18190 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:16.383581 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:16.289328 18094 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.384764 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:16.382788 18070 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:16.426167 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:16.429925 18070 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:16.483958 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57912.5us
W20260709 12:15:16.486558 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.383093 18280 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:16.430934 18085 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:15:16.498804 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 211788us
I20260709 12:15:16.507872 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:16.489970 18084 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:16.490634 18296 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:16.514998 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:16.546209 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:16.516199 18293 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.518011 18304 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.520220 18263 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.542339 18229 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:16.504961 18179 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:16.563313 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.511667 18084 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:16.595578 18084 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:16.596017 11254 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:16.513025 18280 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:16.596868 18280 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:16.598610 11250 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:16.614205 18017 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:16.615967 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69834.5us
I20260709 12:15:16.556509 18296 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.549405 18302 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:16.625686 18282 raft_consensus.cc:2764] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:15:16.653206 18302 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:16.602015 18094 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:16.654544 18190 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.655184 18190 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.547721 18220 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.656320 18220 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.657605 18190 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.614964 18179 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.660895 18179 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:16.650106 11323 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 10. (term mismatch)
I20260709 12:15:16.661703 11323 pending_rounds.cc:85] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:15:16.662038 11323 pending_rounds.cc:107] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 21.10
I20260709 12:15:16.662364 11323 raft_consensus.cc:2889] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:16.664592 18220 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:16.654763 18302 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.648051 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:16.671190 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:16.673087 18282 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:16.673669 18282 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:16.674146 18282 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:16.680513 18070 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.717090 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:16.681078 18179 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:16.681403 18280 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:16.647548 18313 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:15:16.722960 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:16.691139 11296 leader_election.cc:400] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20.
I20260709 12:15:16.703390 18084 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.705729 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.720525 11932 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:16.726014 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:16.726604 11934 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:16.715644 18220 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.727414 11928 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:16.725438 11249 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:16.731401 18225 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:16.732168 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.732805 11926 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:16.735687 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:16.736009 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.737093 11253 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
W20260709 12:15:16.737586 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64877us
I20260709 12:15:16.738469 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.745962 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:16.746636 11249 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:16.748919 18070 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:16.752053 18201 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.752657 18201 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.754803 18201 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:16.755672 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.729615 18179 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:16.756938 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:16.757156 18179 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.757929 18070 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:16.758306 18070 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:16.758641 18070 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:16.761417 11249 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:15:16.769701 18070 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.770540 11331 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:16.771106 11256 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:16.723060 18253 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:16.761655 11251 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:15:16.775506 11249 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 12:15:16.776008 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:16.782275 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:16.785835 11253 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:16.774533 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:16.787317 18253 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:16.798228 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:16.792810 18094 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:16.802183 11253 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:16.786533 18190 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:15:16.818414 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2931) took 1282 ms. Trace:
I20260709 12:15:16.818883 11255 rpcz_store.cc:276] 0709 12:15:15.535696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.535830 (+   134us) service_pool.cc:224] Handling call
0709 12:15:15.536138 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.168613 (+632475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.168653 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.168699 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:16.168730 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.818109 (+649379us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54000f8cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:16.818168 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.818345 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1454078336}
I20260709 12:15:16.802416 18253 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:16.737115 18220 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:16.829445 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 104358us
I20260709 12:15:16.823313 18094 raft_consensus.cc:2764] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:15:16.836416 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:16.836728 18292 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:16.837436 18292 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:16.839897 18292 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:16.840224 18201 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:16.836105 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:16.843475 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 016411114c6d42039bd04025378dfc98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:16.836797 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.860877 11323 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:15:16.863389 11324 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:15:16.864524 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.870659 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.871258 11331 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:15:16.873178 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:16.882515 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:16.886636 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:16.887601 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:16.894165 18200 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:16.894963 17861 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:16.895259 18200 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:16.896129 18200 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:16.897255 18070 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.899243 18220 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.900532 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:16.900852 18220 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:16.901746 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65735.1us
I20260709 12:15:16.903280 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:16.904364 18299 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.688s	user 0.000s	sys 0.002s
I20260709 12:15:16.906138 18084 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:16.907187 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:16.907799 11249 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:16.909574 11255 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 15. Current term is 16. Ops: []
W20260709 12:15:16.912137 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 76753us
I20260709 12:15:16.858989 18292 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:16.920341 11249 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:16.912889 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:16.922767 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:16.924077 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:16.924402 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:16.912020 18190 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.930713 18190 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.933007 18190 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:16.921212 18306 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:16.936218 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2942) took 1060 ms. Trace:
I20260709 12:15:16.937184 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:16.937915 18323 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:16.939033 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:16.939620 11254 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:16.937422 18318 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:15:16.908694 18296 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:16.948020 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:16.936570 11248 rpcz_store.cc:276] 0709 12:15:15.875688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.875917 (+   229us) service_pool.cc:224] Handling call
0709 12:15:15.899009 (+ 23092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.911200 (+ 12191us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:16.095012 (+183812us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.839108 (+744096us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b18003801a0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a
0709 12:15:16.839320 (+   212us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:16.839795 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.839940 (+   145us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:16.839975 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.840041 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.935919 (+ 95878us) raft_consensus.cc:1685] finished
0709 12:15:16.936015 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.936156 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11747,"prepare.queue_time_us":332233,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":38572,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":18951,"spinlock_wait_cycles":25768064,"thread_start_us":269611,"threads_started":2,"wal-append.queue_time_us":185356}
I20260709 12:15:16.928375 18213 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:16.901463 18220 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:16.927546 18320 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:16.963479 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.956652 18266 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:16.968104 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.003316 11254 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:17.005716 11296 leader_election.cc:400] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18.
I20260709 12:15:17.006177 11296 leader_election.cc:403] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:17.006834 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.007527 11178 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
W20260709 12:15:17.004833 18324 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:15:16.975075 18205 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:17.002815 18227 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.011118 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.015643 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:16.974277 18213 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.016299 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:16.956743 18327 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:17.018131 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:17.019557 11180 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:15:17.021175 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2904) took 2788 ms. Trace:
I20260709 12:15:17.024914 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.025586 11252 rpcz_store.cc:276] 0709 12:15:14.224837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.224985 (+   148us) service_pool.cc:224] Handling call
0709 12:15:14.230068 (+  5083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:15.588787 (+1358719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.011129 (+1422342us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400109854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:17.013527 (+  2398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3095890688}
I20260709 12:15:17.040618 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.043812 11175 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:15:17.011888 18292 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:17.050055 11180 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:15:17.016556 18179 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18.
I20260709 12:15:17.010018 18227 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.026475 18213 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:17.058000 11331 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 12:15:17.057358 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 106488us
I20260709 12:15:16.974153 18282 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.026158 18327 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:16.973521 18200 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.040481 18229 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.023654 18094 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:17.054900 18320 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:17.076954 11180 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:15:17.080255 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3133) took 2070 ms. Trace:
I20260709 12:15:17.080695 11179 rpcz_store.cc:276] 0709 12:15:15.009905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.010670 (+   765us) service_pool.cc:224] Handling call
0709 12:15:16.090444 (+1079774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.090633 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.090657 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.090695 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:16.090725 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.090786 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:17.080068 (+989282us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400347bd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:17.080186 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482480896}
I20260709 12:15:17.084271 18200 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:17.075297 18229 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.106279 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:17.132184 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.097510 18293 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:17.092026 18314 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.465s	user 0.000s	sys 0.001s
I20260709 12:15:17.098865 18226 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:17.095618 18254 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:17.091832 11325 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:17.142663 11325 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:17.075575 18263 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.090780 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.101536 11331 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:17.149237 11331 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:17.092514 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.112635 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2759) took 1514 ms. Trace:
I20260709 12:15:17.119740 11182 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:15:17.120855 11180 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:15:17.107576 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:17.063802 18199 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:17.121505 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:17.124703 11323 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:17.109804 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:17.100383 18322 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:15:17.108017 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.105454 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:17.107255 18304 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.111634 18220 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.131568 18190 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:17.157922 18190 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.158305 18190 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:17.162120 17498 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:15:17.142233 18308 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:15:17.162684 17498 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:17.163090 18303 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:15:17.163388 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.163692 17498 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.140700 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:17.164806 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.145306 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:17.166568 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:17.167749 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 58026.9us
I20260709 12:15:17.167953 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.146200 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:17.137390 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:17.155733 18199 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.165376 11330 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:17.069288 18327 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:17.170786 11322 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:15:17.168738 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:17.150450 11326 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:17.143436 11325 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.176151 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:17.173156 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 65645.1us
W20260709 12:15:17.179242 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 154375us
I20260709 12:15:17.182024 11928 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 12:15:17.139608 18242 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:17.193270 11256 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:17.189368 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:17.187644 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:17.193105 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70200.3us
I20260709 12:15:17.197355 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.198980 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:17.202224 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:17.201316 11328 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:15:17.203276 11328 pending_rounds.cc:85] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 15
I20260709 12:15:17.203083 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.203548 11328 pending_rounds.cc:107] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:15:17.203804 11328 raft_consensus.cc:2889] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:17.205566 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 58904.1us
I20260709 12:15:17.207024 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:17.150959 11327 rpcz_store.cc:276] 0709 12:15:15.598377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.599667 (+  1290us) service_pool.cc:224] Handling call
0709 12:15:15.599956 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.382687 (+782731us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:15:16.423433 (+ 40746us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.551663 (+128230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.551736 (+    73us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:16.551804 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.112411 (+560607us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b5400429ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:17.112458 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:17.112579 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41161,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":97,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"spinlock_wait_cycles":1799632128,"thread_start_us":127843,"threads_started":1,"wal-append.queue_time_us":219264}
I20260709 12:15:17.211493 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.214205 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.214728 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.215687 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.217073 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.164832 17498 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:17.217751 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.213598 18212 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:15:17.204368 11331 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.234933 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.178417 18293 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:17.140604 18188 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:15:17.186015 18247 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:17.255484 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 109362us
I20260709 12:15:17.256524 11325 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:17.208129 18134 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:17.205092 18017 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:17.261312 11182 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:17.213788 18296 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:15:17.257351 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:17.215337 18297 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:17.207013 18094 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:17.270962 18242 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:17.237382 18234 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:15:17.273675 18220 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:15:17.298687 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 111134us
I20260709 12:15:17.312999 11325 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:15:17.281129 18084 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.314807 11926 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:17.315135 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:17.315028 18084 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:17.303328 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59862.5us
I20260709 12:15:17.316170 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:17.317615 18084 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:17.319720 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:17.320123 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 83161.8us
I20260709 12:15:17.320843 11926 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:17.321439 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:17.322830 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:17.322846 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 113601us
I20260709 12:15:17.324751 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:17.324925 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:17.279847 18134 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:17.306877 18085 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:17.282897 18070 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:17.262434 18265 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:15:17.260260 18190 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.308039 18094 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.368856 18094 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.371886 18266 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:17.301244 18292 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:17.345454 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:17.349179 18332 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 12:15:17.352492 18346 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:17.352422 18084 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:15:17.358909 18220 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:17.363991 18282 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:17.364858 18253 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.000s
I20260709 12:15:17.365324 18227 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.365937 18085 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:17.366494 18199 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:17.372699 18342 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:17.373909 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46461.2us
I20260709 12:15:17.435061 18134 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.441834 18084 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:17.446224 11254 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:17.447481 18190 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:17.447520 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.442466 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.452622 11252 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:15:17.457525 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:17.384362 18323 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:17.459530 11249 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:17.374799 18347 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:17.342268 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:17.384157 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66679.7us
W20260709 12:15:17.395696 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74424.3us
W20260709 12:15:17.463347 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:17.282802 18017 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.365176 18242 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:17.406334 11330 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 12:15:17.468427 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 126267us
I20260709 12:15:17.462540 18342 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.404304 18212 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:17.471081 18212 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:17.471530 18212 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:17.471230 18213 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.472076 18342 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:17.410259 11934 raft_consensus.cc:2368] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:17.474886 11322 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:15:17.475260 11322 pending_rounds.cc:85] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 12
I20260709 12:15:17.475471 11322 pending_rounds.cc:107] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:15:17.475692 11322 raft_consensus.cc:2889] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:17.409168 18346 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.477721 18296 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:15:17.430051 11331 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:17.493999 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.421593 18253 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.418975 18220 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:17.434224 18085 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.465528 18353 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.469846 18344 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:17.528900 11931 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:17.477764 18302 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:17.531805 18017 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:17.468443 18242 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.533843 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:17.534443 11182 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:17.534993 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.536202 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.419853 18229 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:17.470352 18070 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:17.543651 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.545059 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.547175 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:17.496241 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.495608 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:17.473404 18213 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.566884 11331 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:15:17.501382 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.561296 18328 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.516882 18346 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.498659 18084 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:15:17.518097 18258 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.518600 18353 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.525976 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.526614 18347 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:17.538616 11255 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:17.540280 11926 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
W20260709 12:15:17.495430 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 252690us
I20260709 12:15:17.556638 11182 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:17.559293 18348 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:15:17.513224 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 301795us
I20260709 12:15:17.541178 18344 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:17.571314 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:17.571382 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70120.4us
I20260709 12:15:17.587185 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.591471 11931 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:17.569720 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.541087 18094 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.593729 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.605718 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.553175 18300 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 12:15:17.579176 18196 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:17.603731 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:17.575722 18296 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.605341 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:17.610064 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:17.611553 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.586786 18253 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:17.618638 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.622612 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.578109 18045 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:15:17.602247 18280 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:17.641582 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.644623 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.629009 18179 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:17.630774 18300 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:17.636461 18188 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:17.640585 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.595492 18347 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:17.578714 18328 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.606964 17861 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:17.584635 18227 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:15:17.606444 18171 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:17.686893 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64344.7us
I20260709 12:15:17.625753 18085 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:17.700910 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.591513 18258 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.643641 18327 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.602820 18094 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:17.695370 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45063.6us
I20260709 12:15:17.728183 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.728269 11931 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
W20260709 12:15:17.713248 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 99790.3us
I20260709 12:15:17.647017 18353 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:17.687980 18297 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:15:17.695250 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60254.2us
I20260709 12:15:17.647337 18352 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:15:17.763008 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:17.734603 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:17.736802 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:17.739643 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:17.685868 18282 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:17.694751 18342 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:17.698793 18302 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:17.693778 18300 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:17.781124 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:17.781523 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:17.765982 11176 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:17.765188 17498 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:17.698577 18188 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:17.700692 18196 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:17.780999 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:17.782346 17498 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.697993 18266 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:17.647578 18348 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.096s
W20260709 12:15:17.736868 18205 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.002s
I20260709 12:15:17.734551 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:17.792259 11173 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:17.627168 18296 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.764003 18190 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.763797 18297 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.744709 18085 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:17.776432 18179 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:17.723416 18265 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:15:17.718405 17861 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.853686 11930 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:17.694875 18220 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.848917 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.867245 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.868410 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:17.820346 18070 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:17.820633 18346 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:17.878556 11927 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:17.854727 18297 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:17.789494 18282 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:17.876673 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65493.8us
W20260709 12:15:17.933061 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 205921us
I20260709 12:15:18.013870 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:18.021181 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 144973us
I20260709 12:15:18.023537 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:18.025367 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:17.855822 18190 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:17.762560 18344 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.076362 11222 leader_election.cc:400] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:18.010025 18253 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:18.009568 18280 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:17.872438 18179 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:17.863440 18302 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.147s
I20260709 12:15:17.930397 18199 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:18.098766 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 71808.8us
I20260709 12:15:17.928618 18256 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.150s
W20260709 12:15:18.097390 18357 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:15:18.143170 11928 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:18.143549 11222 leader_election.cc:403] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:15:18.149740 11322 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.008s	sys 0.012s
I20260709 12:15:18.098449 18355 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:18.157207 11322 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.012s
W20260709 12:15:18.098461 18369 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:15:18.082340 18134 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:15:18.142848 18153 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:18.144431 18205 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:15:18.182390 18363 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 12:15:18.276477 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 200277us
I20260709 12:15:18.210071 11928 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:18.158339 18253 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:17.928056 18347 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.144481 18352 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:18.277776 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:18.198151 18320 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.076s	user 0.009s	sys 0.005s
I20260709 12:15:18.012099 18300 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.078964 18188 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:18.200192 18362 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 12:15:18.284426 18094 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:18.284857 18382 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
W20260709 12:15:18.285596 18372 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:15:18.085201 18085 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.163208 18348 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.121s
W20260709 12:15:18.278208 18358 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.495s	user 0.000s	sys 0.001s
I20260709 12:15:18.099287 18196 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:18.285599 18365 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:15:18.289680 18337 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:18.298795 18205 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:15:18.291558 18327 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.292665 18258 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.299636 18242 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.303272 18323 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.301836 18320 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:18.352017 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74573.9us
I20260709 12:15:18.306870 18328 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.313776 18153 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:18.362967 18386 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:15:18.368659 18381 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:18.370200 18371 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.096s
I20260709 12:15:18.374328 18190 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.379001 17861 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:18.380326 18367 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 12:15:18.430963 11329 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:18.343678 18084 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.381227 18188 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:18.411090 18212 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.939s	user 0.008s	sys 0.004s
I20260709 12:15:18.384961 18323 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:18.384416 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:18.481178 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 96846.4us
I20260709 12:15:18.409474 18393 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.333519 18355 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
W20260709 12:15:18.458494 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.419842 18300 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:18.414552 18153 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:18.484740 18365 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:15:18.478140 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.468793 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2964) took 1018 ms. Trace:
I20260709 12:15:18.418445 18337 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.361477 18220 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.431552 18282 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:18.495437 11220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.003s	sys 0.006s
W20260709 12:15:18.510356 11220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.003s	sys 0.006s
W20260709 12:15:18.507671 11145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.003s	sys 0.006s
W20260709 12:15:18.512645 11145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.003s	sys 0.006s
I20260709 12:15:18.503054 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.515719 11930 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:18.501101 11251 rpcz_store.cc:276] 0709 12:15:17.450210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.461310 (+ 11100us) service_pool.cc:224] Handling call
0709 12:15:17.461780 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.462041 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.462066 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.494377 (+ 32311us) log.cc:844] Serialized 26 byte log entry
0709 12:15:17.802711 (+308334us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:17.803096 (+   385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.803500 (+   404us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.468526 (+665026us) raft_consensus.cc:1685] finished
0709 12:15:18.468552 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.468724 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":236024,"threads_started":1,"wal-append.queue_time_us":438844}
W20260709 12:15:18.531862 18360 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.490s	user 0.001s	sys 0.000s
I20260709 12:15:18.383539 18242 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.467108 18381 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.467772 18190 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.439987 18386 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:15:18.498482 18393 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:18.532402 18368 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.410s	user 0.000s	sys 0.001s
W20260709 12:15:18.542893 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.038s	sys 0.070s
I20260709 12:15:18.508096 18390 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.568233 18390 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.430773 18253 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:18.570474 18387 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:15:18.570732 18390 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.530609 18391 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:18.571775 18359 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.941s	user 0.000s	sys 0.009s
I20260709 12:15:18.571581 18391 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.529933 18395 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.572799 18395 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.573240 18054 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:18.560848 18328 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.574556 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1308) took 1119 ms. Trace:
I20260709 12:15:18.574420 18328 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.574903 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1352) took 1130 ms. Trace:
I20260709 12:15:18.575021 18395 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.575821 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3178) took 1034 ms. Trace:
I20260709 12:15:18.576519 18328 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:18.181921 18199 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.578632 18199 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.547160 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.494402 18212 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:18.516307 18213 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.005s	sys 0.004s
W20260709 12:15:18.581375 18213 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.005s	sys 0.005s
I20260709 12:15:18.581101 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.581835 11254 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:18.583853 18212 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:18.576200 11330 rpcz_store.cc:276] 0709 12:15:17.541604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.541918 (+   314us) service_pool.cc:224] Handling call
0709 12:15:17.542221 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.542537 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.542566 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.567070 (+ 24504us) log.cc:844] Serialized 26 byte log entry
0709 12:15:17.783687 (+216617us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:17.783798 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.783879 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.575623 (+791744us) raft_consensus.cc:1685] finished
0709 12:15:18.575644 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.575769 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7520,"raft.queue_time_us":9265,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5521,"thread_start_us":184185,"threads_started":1,"wal-append.queue_time_us":294938}
I20260709 12:15:18.514955 18282 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.587460 18282 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.587857 18391 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.561586 18381 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.512226 18337 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.589756 18242 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.591504 18337 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:18.591698 18381 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.592167 18190 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.594269 18190 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.530216 18153 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.545159 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:18.595969 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50953.1us
I20260709 12:15:18.575307 11326 rpcz_store.cc:276] 0709 12:15:17.444955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.445126 (+   171us) service_pool.cc:224] Handling call
0709 12:15:17.445416 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.445941 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.445970 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.552214 (+106244us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b2c00126d18. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:17.552585 (+   371us) log.cc:844] Serialized 26 byte log entry
0709 12:15:17.736010 (+183425us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:17.782840 (+ 46830us) log.cc:844] Serialized 14 byte log entry
0709 12:15:17.783076 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.783174 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.552765 (+769591us) raft_consensus.cc:1685] finished
0709 12:15:18.574701 (+ 21936us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.574845 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47144,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":774,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":33597,"spinlock_wait_cycles":128690560,"thread_start_us":132781,"threads_started":1,"wal-append.queue_time_us":294446}
I20260709 12:15:18.596966 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:18.479671 18084 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.573608 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:18.561568 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.039s	sys 0.070s
W20260709 12:15:18.599133 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.00737e+06us
I20260709 12:15:18.564675 18378 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:18.565971 18385 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.566900 18393 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.567431 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.571693 18179 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.573702 18054 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:18.531134 18388 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.108s
I20260709 12:15:18.563788 18386 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.562777 18402 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:18.604267 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.628562 11934 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:18.615747 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.632104 11326 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:18.630306 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:18.581894 11325 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:18.574939 11932 rpcz_store.cc:276] 0709 12:15:17.455319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.456174 (+   855us) service_pool.cc:224] Handling call
0709 12:15:17.456644 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.466881 (+ 10237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.466921 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.467481 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:15:17.557209 (+ 89728us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:17.557250 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.568343 (+ 11093us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.571098 (+1002755us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:18.574270 (+  3172us) raft_consensus.cc:1685] finished
0709 12:15:18.574302 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.574494 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":907,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":30427,"spinlock_wait_cycles":42069376,"thread_start_us":89337,"threads_started":1,"wal-append.queue_time_us":121855}
W20260709 12:15:18.732595 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73466.5us
I20260709 12:15:18.648898 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:18.758255 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.738183 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:18.742795 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 94050.6us
I20260709 12:15:18.760152 11331 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:18.629181 18355 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.766059 18355 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.768483 18355 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.731519 18393 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:18.598600 18282 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.758653 18402 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.771931 18402 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:18.772532 18205 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:18.758445 18386 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.567461 18201 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.773929 18201 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.774050 18054 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.774806 18386 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.731173 18213 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:15:18.777079 18201 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:18.777482 18323 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.778092 18084 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:18.778503 18383 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:18.778990 18383 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.779372 18383 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:18.779837 18323 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:18.780126 18320 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:18.780586 18320 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:18.781266 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:18.785277 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3176) took 1244 ms. Trace:
I20260709 12:15:18.758941 17861 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:15:18.785781 11322 rpcz_store.cc:276] 0709 12:15:17.540913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.544149 (+  3236us) service_pool.cc:224] Handling call
0709 12:15:17.544445 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.157833 (+613388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:18.157883 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:18.188637 (+ 30754us) log.cc:844] Serialized 26 byte log entry
0709 12:15:18.484297 (+295660us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:18.484341 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.484421 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.785022 (+300601us) raft_consensus.cc:1685] finished
0709 12:15:18.785045 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.785206 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30860,"raft.queue_time_us":44650,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":13681,"thread_start_us":758573,"threads_started":2,"wal-append.queue_time_us":499324}
I20260709 12:15:18.786996 18399 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:18.787592 18399 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:18.787981 18399 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:18.791409 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 161187us
I20260709 12:15:18.793787 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.795194 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.786964 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1346) took 1649 ms. Trace:
I20260709 12:15:18.793530 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:18.740314 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:18.790670 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56179.6us
I20260709 12:15:18.815871 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.796300 11329 rpcz_store.cc:276] 0709 12:15:17.137688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.202352 (+ 64664us) service_pool.cc:224] Handling call
0709 12:15:17.202633 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:18.431688 (+1229055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.786659 (+354971us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400428bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:18.786853 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2809598208}
W20260709 12:15:18.825914 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 86350.9us
I20260709 12:15:18.827045 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.827848 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 16.
I20260709 12:15:18.828908 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:18.826086 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:18.627951 18346 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:18.591514 17805 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:18.835157 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.837282 11322 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:15:18.841836 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.836601 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:18.844251 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.846608 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.846876 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.847319 11325 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:18.850819 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:18.732472 18328 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.825440 18196 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:18.855394 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.014s	sys 0.018s
W20260709 12:15:18.858016 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.014s	sys 0.018s
I20260709 12:15:18.858577 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.811025 18084 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.858217 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.860133 11325 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:18.832166 18201 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.839514 18402 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:18.834784 18346 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.834872 17805 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:18.863248 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.825735 18242 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.834859 18337 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.859256 18212 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:18.858731 18282 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:18.881289 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:18.882217 11927 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
W20260709 12:15:18.883258 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.011s	sys 0.031s
I20260709 12:15:18.890311 18344 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.854693 18386 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:18.894886 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.781472 18320 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.923980 11934 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:18.627317 18385 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.742098 18378 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:18.879426 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:18.860936 18292 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.853600 18328 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.895169 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.873700 18084 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.868649 18196 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.876771 18402 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.878809 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:18.882663 18346 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:18.882879 18395 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:18.932013 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60490.4us
W20260709 12:15:18.900930 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.011s	sys 0.032s
W20260709 12:15:18.933279 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.901322 18212 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:18.950378 11934 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:15:18.950556 18302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.002s	sys 0.027s
I20260709 12:15:18.952947 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:18.914716 18364 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.968556 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.969237 11934 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:18.876312 18201 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:18.934165 18188 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.897543 18242 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.934165 18393 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:18.976039 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:18.941810 18402 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:18.978140 18410 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:18.978507 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.978739 17805 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.974942 18364 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:18.964073 18302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.002s	sys 0.027s
I20260709 12:15:18.899533 18337 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.965044 18212 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:18.996016 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 54894.5us
I20260709 12:15:18.928570 18282 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.923616 18344 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.937297 18292 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:18.999629 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.712s (SENT) [suppressed 3 similar messages]
I20260709 12:15:18.987819 18253 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:19.052640 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:18.965500 18395 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.065898 11931 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:19.064076 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:19.070053 11256 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:19.073526 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.074486 11256 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:19.076817 18412 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:19.080202 11256 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:19.090876 11251 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:19.092324 18412 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.094962 18292 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:19.051604 17805 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:18.986056 18188 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:19.095803 18382 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:15:19.055756 18302 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:19.114293 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.094990 18412 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:19.119436 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.118006 11256 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:19.119112 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.131735 11251 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
W20260709 12:15:19.115239 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 115582us
I20260709 12:15:19.141299 11250 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:19.144946 18364 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.145179 18201 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.099006 18253 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.059938 18378 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.011439 18410 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.156279 18353 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:19.169220 11250 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:19.160928 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:19.171486 11249 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:15:19.172925 18351 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.768s	user 0.005s	sys 0.008s
W20260709 12:15:19.174190 18405 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:15:19.175380 18398 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:19.176083 18398 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.178764 11253 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:15:19.180477 18409 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:15:19.181005 11322 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:19.181483 11322 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:19.172971 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.183614 11251 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:19.121582 18302 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:19.186769 18416 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:15:19.182312 11322 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.198947 11322 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:19.200739 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:19.202901 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:19.204056 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.203365 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.207331 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.204698 11256 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:19.211707 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:19.213514 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.214452 11256 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:19.218063 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:19.225311 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.227383 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.198459 18227 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:19.202051 18328 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.163527 18201 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:15:19.209836 18386 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:19.245532 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.233431 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.250746 11253 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:19.166456 18353 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.215881 18199 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:19.210220 18385 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.253010 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.255471 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.261741 11251 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:19.227506 18205 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:19.260082 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.264303 11250 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:19.232043 18227 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:19.239686 18253 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.270138 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:19.276650 18427 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:15:19.276902 18429 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:19.277645 18429 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.278852 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3189) took 1630 ms. Trace:
I20260709 12:15:19.279278 11182 rpcz_store.cc:276] 0709 12:15:17.648649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.650083 (+  1434us) service_pool.cc:224] Handling call
0709 12:15:17.650361 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.728747 (+ 78386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.728779 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:17.728829 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:17.728862 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.278575 (+1549713us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b54002b7354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:19.278625 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.278791 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179820928}
I20260709 12:15:19.276556 18410 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.332928 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.333607 11256 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:15:19.258642 18386 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.333002 18344 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:19.338941 18413 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:15:19.339356 18386 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:19.340102 18428 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:19.269953 18337 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.271622 18393 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.392880 18344 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:19.404760 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.404407 18344 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.407305 18344 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:19.392944 18414 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
W20260709 12:15:19.382884 18297 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.373s	user 0.008s	sys 0.007s
I20260709 12:15:19.422511 18395 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.425915 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1312) took 1691 ms. Trace:
I20260709 12:15:19.426510 11928 rpcz_store.cc:276] 0709 12:15:17.734144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.734292 (+   148us) service_pool.cc:224] Handling call
0709 12:15:19.425592 (+1691300us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540034f3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:19.425792 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625811328}
I20260709 12:15:19.434921 18353 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.436477 18302 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.382028 18084 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.447423 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:19.437500 18419 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 12:15:19.353888 18429 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:19.446455 18383 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.648s	user 0.013s	sys 0.000s
I20260709 12:15:19.454850 18343 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:15:19.457736 18394 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:19.412598 18398 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.462772 18094 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:19.463165 18188 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.272894 18227 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:19.480643 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.384847 18393 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.482401 11175 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:19.480794 18094 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:19.483215 18394 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:19.485776 18394 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:19.485821 18094 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.486999 18094 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:19.487939 18393 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.453886 18344 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:15:19.446983 18297 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:19.484704 18399 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.005s
I20260709 12:15:19.468854 18383 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.486946 18212 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:19.563232 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.609640 11178 raft_consensus.cc:2393] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 17: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:19.503547 18346 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.002s
W20260709 12:15:19.630019 11328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.004s	sys 0.009s
W20260709 12:15:19.630447 11328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.004s	sys 0.009s
W20260709 12:15:19.649266 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.010s	sys 0.006s
W20260709 12:15:19.649771 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.010s	sys 0.006s
I20260709 12:15:19.650475 11255 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:19.654829 11327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.036s	sys 0.065s
W20260709 12:15:19.655205 11327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.036s	sys 0.065s
I20260709 12:15:19.656710 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.503875 18399 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.665347 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.671692 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.672056 11175 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:19.672561 11325 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
W20260709 12:15:19.665968 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.674819 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.657378 11177 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:19.650455 18429 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:19.681952 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.689924 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.691294 11330 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:19.692269 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.692842 11330 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:19.697726 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.699287 11330 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:19.625344 18346 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:19.677735 17805 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.570s	user 0.010s	sys 0.003s
I20260709 12:15:19.661513 18302 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:19.705361 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.669219 18398 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.706173 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.673964 18393 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.671099 18266 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:19.711614 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.709475 11325 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:15:19.712751 11935 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.025s	sys 0.040s
W20260709 12:15:19.713092 11935 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.025s	sys 0.040s
I20260709 12:15:19.713246 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.714140 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.717327 11330 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:15:19.715853 11257 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.007s	sys 0.014s
I20260709 12:15:19.714838 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:19.719671 11257 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.007s	sys 0.015s
I20260709 12:15:19.714968 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:19.729966 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.732527 11174 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:19.735152 11329 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:19.736742 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.737890 11175 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:19.738919 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.740554 11175 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:19.750571 11175 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:19.759616 11173 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3179) took 2627 ms (client timeout 1712 ms). Trace:
W20260709 12:15:19.764786 11173 rpcz_store.cc:269] 0709 12:15:17.104512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.105207 (+   695us) service_pool.cc:224] Handling call
0709 12:15:19.732164 (+2626957us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54001f36d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:19.732313 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439080832}
I20260709 12:15:19.703684 18429 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.770946 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.775972 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:19.776886 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.005s	sys 0.019s
W20260709 12:15:19.777410 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.005s	sys 0.019s
I20260709 12:15:19.776950 11177 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:19.720350 18407 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:19.729944 18408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.015s	sys 0.044s
W20260709 12:15:19.772552 18402 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.739s	user 0.005s	sys 0.008s
I20260709 12:15:19.762588 18344 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:19.777048 18397 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:15:19.789203 11296 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.055s	sys 0.077s
W20260709 12:15:19.789559 11296 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.055s	sys 0.077s
W20260709 12:15:19.787482 11930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.009s	sys 0.018s
W20260709 12:15:19.789896 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 541198us
I20260709 12:15:19.788401 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:19.712325 18302 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:19.782497 18408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.015s	sys 0.044s
I20260709 12:15:19.793157 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.793864 11256 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:19.728788 18266 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.793159 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:19.795450 11928 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:19.729266 18171 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:15:19.798405 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:19.799711 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:19.789960 11930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.010s	sys 0.018s
I20260709 12:15:19.729256 18393 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:19.729984 18398 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.809336 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:19.889511 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:19.886647 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:19.910092 11256 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 20 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 12:15:19.909217 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 110906us
I20260709 12:15:19.820611 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.888044 11326 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:15:19.783609 11252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.019s	sys 0.023s
W20260709 12:15:19.912142 11252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.019s	sys 0.024s
I20260709 12:15:19.911957 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:19.790210 18344 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:19.821293 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.878479 18294 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.915174 18344 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:19.915799 11323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.024s	sys 0.026s
I20260709 12:15:19.916088 11928 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 16: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:19.916389 11323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.024s	sys 0.027s
W20260709 12:15:19.916584 11147 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.003s	sys 0.011s
W20260709 12:15:19.916877 11147 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.003s	sys 0.011s
I20260709 12:15:19.915470 18294 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:19.917191 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 660914us
I20260709 12:15:19.919097 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:19.919777 11933 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:19.925653 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 1028) took 1996 ms. Trace:
I20260709 12:15:19.927857 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.926117 11927 rpcz_store.cc:276] 0709 12:15:17.929783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.930164 (+   381us) service_pool.cc:224] Handling call
0709 12:15:18.004685 (+ 74521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:18.900700 (+896015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.924853 (+1024153us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b5400415fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:19.925560 (+   707us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2017502080}
I20260709 12:15:19.788106 18402 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.883136 18403 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:19.878176 18333 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:19.932782 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52776.6us
I20260709 12:15:19.921887 18343 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:19.922717 18212 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.940734 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:19.938577 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:19.944314 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:19.945040 11325 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:19.930928 18378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.027s	sys 0.023s
I20260709 12:15:19.948787 18349 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:19.949048 18294 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.949446 18349 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:19.950001 18349 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:19.953672 18201 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:19.954295 18201 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.954794 18201 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:19.942943 18347 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:15:19.957612 17805 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:19.958140 17805 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:19.958626 17805 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:19.958974 18398 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.960268 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:19.960985 11330 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:19.959821 18355 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:19.967181 18355 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.970223 18346 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.960350 18403 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:19.941057 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:15:19.994419 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53443.7us
I20260709 12:15:19.932926 18383 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:19.937469 18297 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:19.961210 18378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.028s	sys 0.023s
W20260709 12:15:20.014752 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 75497.8us
W20260709 12:15:19.996520 18437 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:15:19.949051 18302 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:20.028544 18403 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:19.995332 18344 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:20.032371 11182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.011s	sys 0.028s
W20260709 12:15:20.033064 11182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.011s	sys 0.029s
I20260709 12:15:19.969542 18355 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:20.044839 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1154) took 2514 ms. Trace:
W20260709 12:15:20.070288 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.000408 18171 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:15:20.080830 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:20.088063 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:20.115159 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.049171 11179 rpcz_store.cc:276] 0709 12:15:17.530070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.530183 (+   113us) service_pool.cc:224] Handling call
0709 12:15:17.530515 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.530859 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:15:17.530887 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.411207 (+1880320us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b2c00013008. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:19.411639 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 12:15:19.856492 (+444853us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:19.856529 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.856604 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.044451 (+187847us) raft_consensus.cc:1685] finished
0709 12:15:20.044478 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.044740 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26942,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":99258624,"thread_start_us":444523,"threads_started":1,"wal-append.queue_time_us":630912}
I20260709 12:15:19.999954 18303 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:20.101296 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1080) took 2606 ms. Trace:
W20260709 12:15:20.107476 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.202219 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:19.941715 18333 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:20.117327 11257 rpcz_store.cc:276] 0709 12:15:17.494870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.495003 (+   133us) service_pool.cc:224] Handling call
0709 12:15:17.516832 (+ 21829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.551733 (+ 34901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.551760 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.557786 (+  6026us) log.cc:844] Serialized 26 byte log entry
0709 12:15:19.206491 (+1648705us) spinlock_profiling.cc:243] Waited 9.57 ms on lock 0x7b18004c16e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:19.720146 (+513655us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:19.720186 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.720419 (+   233us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.033704 (+313285us) raft_consensus.cc:1685] finished
0709 12:15:20.033727 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.101181 (+ 67454us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34515,"raft.queue_time_us":34913,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":10936,"spinlock_wait_cycles":22003456,"thread_start_us":513438,"threads_started":1,"wal-append.queue_time_us":2369362}
I20260709 12:15:20.235563 11330 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:20.267524 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:20.269275 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47227.2us
I20260709 12:15:20.296924 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:20.298671 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:20.197765 18337 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:20.196714 18346 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:20.346534 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:20.211813 18171 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:20.297369 18347 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.296643 18429 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:20.204741 18303 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.331648 18403 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:20.338218 18266 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:20.338930 18399 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:20.486562 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:20.487149 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:20.488394 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:20.476082 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 1031) took 1872 ms. Trace:
I20260709 12:15:20.494521 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:20.495282 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:20.398134 18395 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:20.494418 11179 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:20.494350 11930 rpcz_store.cc:276] 0709 12:15:18.603298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.603942 (+   644us) service_pool.cc:224] Handling call
0709 12:15:18.739734 (+135792us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.740253 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:18.740297 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:18.790157 (+ 49860us) log.cc:844] Serialized 26 byte log entry
0709 12:15:19.911234 (+1121077us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:19.911273 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.911337 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.475778 (+564441us) raft_consensus.cc:1685] finished
0709 12:15:20.475809 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.476002 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18335,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":15123,"thread_start_us":1069983,"threads_started":1,"wal-append.queue_time_us":1588370}
I20260709 12:15:20.559073 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:20.575531 11326 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:20.396308 18346 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:20.561435 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.348457 18337 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:20.499374 17805 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.540s	user 0.005s	sys 0.007s
I20260709 12:15:20.463232 18296 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:20.560684 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2966) took 3023 ms. Trace:
I20260709 12:15:20.531605 18350 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:20.442683 18347 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:20.578512 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.560043 18393 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
W20260709 12:15:20.595810 18446 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:15:20.449666 18403 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.600862 11248 rpcz_store.cc:276] 0709 12:15:17.537354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.539106 (+  1752us) service_pool.cc:224] Handling call
0709 12:15:17.539417 (+   311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:17.568427 (+ 29010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.568455 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:17.576007 (+  7552us) log.cc:844] Serialized 52 byte log entry
0709 12:15:17.795890 (+219883us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:18.086532 (+290642us) log.cc:844] Serialized 14 byte log entry
0709 12:15:20.498582 (+2412050us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54001b2720. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6667733 00007f3ec66ecdbf 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c
0709 12:15:20.498665 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.519243 (+ 20578us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.519300 (+    57us) raft_consensus.cc:1685] finished
0709 12:15:20.559584 (+ 40284us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.560611 (+  1027us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109089,"prepare.queue_time_us":56381,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":222087,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":16474,"spinlock_wait_cycles":51414912,"thread_start_us":716320,"threads_started":3,"wal-append.queue_time_us":191354,"wal-append.run_cpu_time_us":6797,"wal-append.run_wall_time_us":1766325}
I20260709 12:15:20.574366 18266 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:20.613417 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.573182 18395 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:20.621279 18451 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:15:20.591158 18171 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:20.604460 18402 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:20.622850 18349 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.672s	user 0.003s	sys 0.008s
I20260709 12:15:20.605599 17805 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:20.626195 18361 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:15:20.625316 18201 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.000s
I20260709 12:15:20.632301 11179 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:15:20.634063 18454 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.635934 18153 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:20.637308 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3187) took 2990 ms. Trace:
W20260709 12:15:20.638338 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:15:20.642148 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.025s	sys 0.030s
W20260709 12:15:20.655587 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.025s	sys 0.031s
W20260709 12:15:20.655987 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 711711us
I20260709 12:15:20.644718 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:20.646682 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:20.647831 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:20.657728 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:20.658589 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:20.619299 18393 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:20.650597 18463 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:20.640960 18448 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.594s	user 0.001s	sys 0.001s
I20260709 12:15:20.656314 11249 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:20.656744 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:20.655138 11180 rpcz_store.cc:276] 0709 12:15:17.647023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.647356 (+   333us) service_pool.cc:224] Handling call
0709 12:15:17.647722 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.787455 (+139733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.787487 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.808538 (+2021051us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b2c00013008. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:19.879917 (+ 71379us) log.cc:844] Serialized 26 byte log entry
0709 12:15:20.306294 (+426377us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:20.306348 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.306452 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.636999 (+330547us) raft_consensus.cc:1685] finished
0709 12:15:20.637026 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.637244 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74895,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":6422272,"thread_start_us":526424,"threads_started":2,"wal-append.queue_time_us":696400}
I20260709 12:15:20.662487 11248 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:20.651285 18227 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.160s	user 0.009s	sys 0.004s
I20260709 12:15:20.663314 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:20.728287 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:20.638909 18266 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:20.729379 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 382906us
I20260709 12:15:20.659092 18459 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:20.659487 18454 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:20.745887 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:20.741026 18460 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:20.732347 18227 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:20.742017 18469 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:20.743095 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.747474 11248 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 12:15:20.762600 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3193) took 1292 ms. Trace:
I20260709 12:15:20.647399 18349 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.661294 18402 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:20.661015 18393 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:20.763254 11182 rpcz_store.cc:276] 0709 12:15:19.473739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.473915 (+   176us) service_pool.cc:224] Handling call
0709 12:15:19.474263 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:19.474514 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:15:19.474539 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.475205 (+   666us) log.cc:844] Serialized 26 byte log entry
0709 12:15:20.033588 (+558383us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:20.033661 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.033739 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.762286 (+728547us) raft_consensus.cc:1685] finished
0709 12:15:20.762319 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.762516 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":543137,"threads_started":1,"wal-append.queue_time_us":960036}
I20260709 12:15:20.745251 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:20.762599 11248 pending_rounds.cc:85] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 8
I20260709 12:15:20.731922 18242 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:20.750929 18094 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.260s	user 0.008s	sys 0.006s
I20260709 12:15:20.654083 18201 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.786549 11248 pending_rounds.cc:107] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:15:20.798481 11248 raft_consensus.cc:2889] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:20.799124 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.788537 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:20.743283 18462 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:20.801360 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1084) took 3069 ms. Trace:
I20260709 12:15:20.802927 11254 rpcz_store.cc:276] 0709 12:15:17.731674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.731992 (+   318us) service_pool.cc:224] Handling call
0709 12:15:17.775304 (+ 43312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:18.582270 (+806966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.801051 (+2218781us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b5400108954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:20.801278 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854426496}
W20260709 12:15:20.806633 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 77900.1us
W20260709 12:15:20.807255 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 62052.9us
I20260709 12:15:20.760413 18337 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:20.728260 18463 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.779858 18469 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.768862 18460 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.768327 18303 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:20.813601 11257 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:20.815814 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.024039 11176 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:21.024381 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:20.843698 11931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1318) took 1960 ms (client timeout 1504 ms). Trace:
I20260709 12:15:20.828948 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3196) took 1995 ms. Trace:
I20260709 12:15:20.985232 17498 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:20.987222 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1139) took 2032 ms. Trace:
I20260709 12:15:21.031294 11934 rpcz_store.cc:276] 0709 12:15:18.954426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.976532 (+ 22106us) service_pool.cc:224] Handling call
0709 12:15:18.976847 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.107423 (+130576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.107456 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.107503 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:19.107536 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.986947 (+1879411us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400617954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:20.986997 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.987149 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299740416}
W20260709 12:15:21.023967 11178 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3182) took 3383 ms (client timeout 4245 ms). Trace:
I20260709 12:15:20.790294 18393 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:21.055670 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.852257 18347 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:20.818334 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2967) took 3280 ms. Trace:
W20260709 12:15:21.037614 11178 rpcz_store.cc:269] 0709 12:15:17.640159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.640301 (+   142us) service_pool.cc:224] Handling call
0709 12:15:20.986121 (+3345820us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54002e45d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:21.023900 (+ 37779us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4177145728}
W20260709 12:15:21.025619 11931 rpcz_store.cc:269] 0709 12:15:18.883350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.883521 (+   171us) service_pool.cc:224] Handling call
0709 12:15:20.843404 (+1959883us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54005afd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:20.843583 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299784576}
I20260709 12:15:21.026855 11327 rpcz_store.cc:276] 0709 12:15:18.834042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.960122 (+126080us) service_pool.cc:224] Handling call
0709 12:15:18.980222 (+ 20100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:19.078429 (+ 98207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.078457 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.079014 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:15:19.655543 (+576529us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:19.655585 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.828500 (+1172915us) spinlock_profiling.cc:243] Waited 97.8 ms on lock 0x7b54003318d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:20.828555 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.828607 (+    52us) raft_consensus.cc:1685] finished
0709 12:15:20.828622 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.828877 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17849,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5260,"spinlock_wait_cycles":224939392,"thread_start_us":551839,"threads_started":1,"wal-append.queue_time_us":682958,"wal-append.run_cpu_time_us":2173,"wal-append.run_wall_time_us":1168940}
I20260709 12:15:20.797926 18242 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:20.825421 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.809021 18462 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:20.814435 18227 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:20.858054 18394 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.372s	user 0.004s	sys 0.010s
I20260709 12:15:21.064520 18337 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:15:20.861454 18266 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:20.953322 18466 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.154s	user 0.003s	sys 0.004s
I20260709 12:15:21.044924 18403 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:20.848701 18445 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.881s	user 0.000s	sys 0.001s
I20260709 12:15:20.760869 18459 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.977363 18320 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:20.848994 18450 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.618s	user 0.003s	sys 0.000s
W20260709 12:15:21.036625 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 229067us
I20260709 12:15:20.789049 18402 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:20.816052 18297 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:20.817833 18472 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:21.023748 18343 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:20.853094 18361 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:21.045609 18395 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:20.813120 18201 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:20.850734 18346 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:21.313642 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.182169 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:21.154553 11255 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:21.189257 17498 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.318007 17498 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:18.552066 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:20.820628 18094 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:21.338639 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:21.334501 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.011s	sys 0.010s
I20260709 12:15:21.339530 11253 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
W20260709 12:15:21.339736 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.011s	sys 0.010s
W20260709 12:15:21.321241 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.013s	sys 0.010s
W20260709 12:15:21.340207 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 683480us
W20260709 12:15:21.340579 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.013s	sys 0.011s
I20260709 12:15:21.092818 11252 rpcz_store.cc:276] 0709 12:15:17.537648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.538372 (+   724us) service_pool.cc:224] Handling call
0709 12:15:17.573260 (+ 34888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.577418 (+  4158us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:18.930950 (+1353532us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b18004d2ae0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:15:19.099990 (+169040us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.925475 (+825485us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.926001 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:15:19.926142 (+   141us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:20.397227 (+471085us) log.cc:844] Serialized 14 byte log entry
0709 12:15:20.397591 (+   364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.397706 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.818070 (+420364us) raft_consensus.cc:1685] finished
0709 12:15:20.818094 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.818266 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36380,"prepare.queue_time_us":961483,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":105,"raft.queue_time_us":19711,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":25809,"spinlock_wait_cycles":6214912,"thread_start_us":1419738,"threads_started":3,"wal-append.queue_time_us":829788}
I20260709 12:15:21.340899 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:21.340916 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 530192us
I20260709 12:15:21.180481 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2639) took 3985 ms. Trace:
I20260709 12:15:21.158670 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:21.343179 11174 rpcz_store.cc:276] 0709 12:15:17.194607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.195839 (+  1232us) service_pool.cc:224] Handling call
0709 12:15:19.732293 (+2536454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.732938 (+   645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.180229 (+1447291us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b54001f36d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:21.180399 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5785267968}
I20260709 12:15:21.229676 17805 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.228432 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1161) took 1520 ms. Trace:
I20260709 12:15:21.332620 17498 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.346733 18412 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:21.228067 11075 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:21.312273 18471 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:15:21.362149 18460 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:21.361893 18412 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.219221 18463 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.220736 18454 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.345180 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:21.278859 18371 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:21.081532 18421 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.359081 11175 rpcz_store.cc:276] 0709 12:15:19.707978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.785586 (+ 77608us) service_pool.cc:224] Handling call
0709 12:15:19.787403 (+  1817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.973522 (+186119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.973567 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.973648 (+    81us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:19.973691 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.228133 (+1254442us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54002e40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:21.228196 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:21.228365 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425141120}
W20260709 12:15:21.360267 11256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.000s	sys 0.024s
I20260709 12:15:21.337603 18094 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:21.385186 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:21.385537 11222 connection.cc:256] RPC call timeout handler was delayed by -1.0856678569998621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 12:15:21.386144 11222 connection.cc:256] RPC call timeout handler was delayed by -1.0604678569998214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
I20260709 12:15:21.389213 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1358) took 2424 ms. Trace:
I20260709 12:15:21.369062 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:21.313419 18297 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:21.380723 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 510789us
I20260709 12:15:21.107220 18469 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:21.326117 18343 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:21.216248 18085 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:21.383360 11256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.000s	sys 0.025s
I20260709 12:15:21.178875 18349 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.312043 18196 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.385366 18200 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.177366 18386 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.389623 11324 rpcz_store.cc:276] 0709 12:15:18.959079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.134586 (+175507us) service_pool.cc:224] Handling call
0709 12:15:19.153419 (+ 18833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.220649 (+ 67230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.220694 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.220757 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:19.220795 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.383601 (+2162806us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b54004597d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:21.383661 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:21.383829 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153971456}
W20260709 12:15:21.394887 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 56336.9us
W20260709 12:15:21.395831 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55015.1us
I20260709 12:15:21.409128 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:21.444236 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:21.444921 11330 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:21.411083 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:21.383507 18442 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:21.378685 18201 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.465983 18201 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.437485 18229 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.467468 18229 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.468858 18201 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.416971 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.420874 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:21.401117 18344 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.025s
I20260709 12:15:21.346093 18346 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:21.382683 18371 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:21.390552 18094 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:21.385120 18265 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.446630 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1356) took 2488 ms. Trace:
I20260709 12:15:21.447050 11322 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:21.445504 11327 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:21.394081 18134 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.522224 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:21.403580 18297 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:21.346074 18472 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.405457 18435 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:21.406529 18225 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:21.506814 18347 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:15:21.457227 18442 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.466457 18300 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.586743 18300 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.149024 18242 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.592007 18242 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:21.147512 18394 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.499440 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.593925 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.594667 18394 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:21.595750 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.596588 11934 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:15:21.474249 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.475008 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.491176 11928 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:21.490291 11248 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
W20260709 12:15:21.501785 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.436429 18395 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:15:21.602722 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:21.603334 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.506194 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.611069 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.612761 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.614455 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:21.617887 11928 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:21.618736 11932 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:21.620852 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:21.622063 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.623090 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.624027 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.624825 11254 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:21.626101 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.566730 18229 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.511315 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.436316 18463 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.441416 18227 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.638279 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:21.515075 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.516358 11249 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:21.510488 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66120.4us
I20260709 12:15:21.519677 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.522894 11251 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:21.498853 18344 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.008s	sys 0.025s
I20260709 12:15:21.517195 11323 rpcz_store.cc:276] 0709 12:15:18.958325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.133129 (+174804us) service_pool.cc:224] Handling call
0709 12:15:19.152682 (+ 19553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:19.153077 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.153107 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.153569 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:15:19.916815 (+763246us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:19.916857 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.916926 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.847941 (+931015us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:21.436107 (+588166us) raft_consensus.cc:1685] finished
0709 12:15:21.436132 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:21.446530 (+ 10398us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":822,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3551,"spinlock_wait_cycles":4750592,"thread_start_us":710672,"threads_started":1,"wal-append.queue_time_us":803548}
I20260709 12:15:21.394292 18421 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.436570 18085 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.516721 18371 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:21.608084 18337 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:21.451390 18200 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.470057 18386 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.574074 11326 pending_rounds.cc:85] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 15
I20260709 12:15:21.648838 11326 pending_rounds.cc:107] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:15:21.649227 11326 raft_consensus.cc:2889] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:21.650115 18171 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.034s	sys 0.043s
W20260709 12:15:21.650655 18171 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.034s	sys 0.043s
I20260709 12:15:21.582165 18225 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:21.550716 18179 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:21.583992 18462 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.503607 18349 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:15:21.534209 18134 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.610929 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:21.521202 18346 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.634104 11930 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:21.533986 18265 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.640704 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:21.643333 11253 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:21.607611 18395 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.457494 11927 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:21.619331 18282 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.629572 18459 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:21.654732 11932 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:21.641650 18463 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.651330 18300 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.644256 18227 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.652396 18337 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.656546 18225 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:21.753858 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3192) took 2283 ms. Trace:
I20260709 12:15:21.775975 11933 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:15:21.717589 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:21.714838 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.509198 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:21.656141 18179 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:21.786425 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 145818us
W20260709 12:15:21.709115 11180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.008s	sys 0.011s
W20260709 12:15:21.787531 11180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.009s	sys 0.011s
I20260709 12:15:21.783353 11251 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:21.787364 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:21.781407 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 272340us
I20260709 12:15:21.779592 11181 rpcz_store.cc:276] 0709 12:15:19.470038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.470151 (+   113us) service_pool.cc:224] Handling call
0709 12:15:19.470524 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:19.479225 (+  8701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.479274 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.479788 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:15:21.002054 (+1522266us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b18002f9920. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:21.403299 (+401245us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:21.403369 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.403478 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:21.753375 (+349897us) raft_consensus.cc:1685] finished
0709 12:15:21.753413 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:21.753759 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":424,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3163,"spinlock_wait_cycles":12588416,"thread_start_us":401005,"threads_started":1,"wal-append.queue_time_us":2267201}
I20260709 12:15:21.792428 18470 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:21.789373 18361 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:21.793926 18361 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.794339 18361 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:21.796236 18054 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:21.801242 18333 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.804615 18442 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.807662 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:21.812268 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2644) took 4069 ms. Trace:
I20260709 12:15:21.812695 11176 rpcz_store.cc:276] 0709 12:15:17.742400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.792843 (+ 50443us) service_pool.cc:224] Handling call
0709 12:15:21.023856 (+3231013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.025181 (+  1325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.812063 (+786882us) spinlock_profiling.cc:243] Waited 3.2 s on lock 0x7b54002e45d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:21.812205 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7360476928}
I20260709 12:15:21.818145 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2974) took 1172 ms. Trace:
I20260709 12:15:21.818588 11257 rpcz_store.cc:276] 0709 12:15:20.645886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.646011 (+   125us) service_pool.cc:224] Handling call
0709 12:15:21.817932 (+1171921us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400276854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:21.818079 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354566016}
I20260709 12:15:21.784696 18229 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:21.822175 18229 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:21.824115 11074 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 172) took 16004 ms (client timeout 15000 ms). Trace:
I20260709 12:15:21.824579 18229 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.737613 18085 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:21.751228 18421 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:21.755501 18472 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.826200 11075 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:21.827247 18179 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.828352 18179 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:21.828001 11075 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:21.830101 18421 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.830806 18421 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.833537 18421 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.834600 18463 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.835451 18200 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.732170 18328 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 12:15:21.842823 18328 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:21.843415 18328 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:21.688474 18412 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:21.845953 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 58670.8us
I20260709 12:15:21.790922 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:21.733562 18386 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:21.873383 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:21.873919 11255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 2976) took 1226 ms (client timeout 1519 ms). Trace:
I20260709 12:15:21.872752 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1167) took 2079 ms. Trace:
W20260709 12:15:21.886026 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95179.1us
I20260709 12:15:21.891795 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:21.885859 11255 rpcz_store.cc:269] 0709 12:15:20.647455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.647577 (+   122us) service_pool.cc:224] Handling call
0709 12:15:21.873741 (+1226164us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b5400277c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:21.873857 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141834112}
I20260709 12:15:21.886349 11179 rpcz_store.cc:276] 0709 12:15:19.793231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.607665 (+814434us) service_pool.cc:224] Handling call
0709 12:15:20.608024 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:20.649394 (+ 41370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.872511 (+1223117us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5400256cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:21.872690 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26019200}
I20260709 12:15:21.779282 18395 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:21.875578 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:21.896956 11248 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:21.817286 18297 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.807487 18094 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.844548 18328 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:22.666390 11256 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.001s	sys 0.010s
W20260709 12:15:21.981062 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:22.719059 11256 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.001s	sys 0.011s
W20260709 12:15:22.719630 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:21.898766 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:22.720142 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:22.720618 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.955238 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:21.932504 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:22.113724 11929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.105s	sys 0.132s
I20260709 12:15:22.023692 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:22.722510 11929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.360s	user 0.162s	sys 0.289s
W20260709 12:15:22.268500 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.014s	sys 0.002s
W20260709 12:15:22.723198 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.335s	user 0.014s	sys 0.006s
I20260709 12:15:21.898046 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:22.723666 11180 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
W20260709 12:15:22.666867 11249 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.003s	sys 0.007s
I20260709 12:15:22.722015 11174 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 16.
W20260709 12:15:22.725718 11249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.005s	sys 0.008s
I20260709 12:15:22.727396 11325 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
W20260709 12:15:22.728845 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.933754 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:21.944408 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:22.566198 11931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.008s	sys 0.004s
I20260709 12:15:22.731029 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
W20260709 12:15:22.731381 11931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.009s	sys 0.005s
I20260709 12:15:21.978286 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:22.733021 11327 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:22.471387 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.425s	user 0.003s	sys 0.024s
W20260709 12:15:22.734251 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.687s	user 0.005s	sys 0.025s
W20260709 12:15:22.734805 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 1.68808e+06us
I20260709 12:15:22.736210 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:22.737854 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:21.958089 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:21.944316 11176 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:22.716511 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.005s	sys 0.011s
I20260709 12:15:21.932170 18200 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:21.785445 18296 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.743312 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:22.714927 11178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.128s	user 0.005s	sys 0.016s
W20260709 12:15:22.812592 11178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.005s	sys 0.016s
I20260709 12:15:22.715301 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:22.813557 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3207) took 1670 ms. Trace:
I20260709 12:15:22.715924 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:22.814919 11175 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:22.814018 11178 rpcz_store.cc:276] 0709 12:15:21.142747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.156648 (+ 13901us) service_pool.cc:224] Handling call
0709 12:15:21.157036 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.157267 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:21.586742 (+429475us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.813140 (+1226398us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:22.813210 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:22.813270 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.813353 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.813494 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58171,"prepare.queue_time_us":119066,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"thread_start_us":1551258,"threads_started":2}
W20260709 12:15:22.747766 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:22.723546 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 1.33571e+06us
I20260709 12:15:22.816749 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:22.722304 11177 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:15:21.870357 18320 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.002s
I20260709 12:15:22.759478 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:22.369863 18371 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.706s	user 0.007s	sys 0.004s
I20260709 12:15:22.429757 18134 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:22.741717 18389 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:22.381475 18179 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.004s
W20260709 12:15:22.486640 18482 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:22.323339 18265 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:22.852028 18265 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:22.853720 18484 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:22.854013 18265 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:22.769722 18328 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:22.372320 18393 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.268s	user 0.014s	sys 0.000s
W20260709 12:15:22.772506 18225 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.982s	user 0.007s	sys 0.007s
I20260709 12:15:22.774533 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:22.667887 18054 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.871s	user 0.009s	sys 0.001s
W20260709 12:15:22.783135 18395 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.827s	user 0.010s	sys 0.001s
I20260709 12:15:22.758050 18399 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:22.278482 18463 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:21.872022 18475 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:21.858084 18470 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:21.862920 18227 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:21.913341 18085 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:21.977805 18403 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:21.888605 18337 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:22.242817 18421 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:22.874137 18463 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:21.869468 18361 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:22.810313 18429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.155s	user 0.003s	sys 0.019s
W20260709 12:15:22.811110 18344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.122s	user 0.010s	sys 0.004s
W20260709 12:15:22.810874 18394 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.009s	sys 0.012s
W20260709 12:15:22.811282 18229 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.014s	sys 0.045s
I20260709 12:15:22.768325 18094 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:22.854600 18484 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:22.771688 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1333) took 1128 ms. Trace:
I20260709 12:15:22.831600 18302 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.863483 18320 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:22.846781 18371 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.874101 18470 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:22.860633 18265 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:22.877290 18085 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:22.879269 18421 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:21.872637 18343 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.867820 18200 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:22.881577 18463 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:22.884872 18429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.003s	sys 0.024s
I20260709 12:15:22.915894 18421 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:22.872292 18475 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:22.885085 18344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.012s	sys 0.006s
I20260709 12:15:22.918675 18200 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:22.919113 18475 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:22.863802 18393 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:22.897802 18484 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:22.894076 11933 rpcz_store.cc:276] 0709 12:15:21.643418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.644951 (+  1533us) service_pool.cc:224] Handling call
0709 12:15:21.775642 (+130691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.779296 (+  3654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.771361 (+992065us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54004c2550. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:22.771604 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299748608}
W20260709 12:15:22.889591 18394 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.010s	sys 0.016s
W20260709 12:15:22.890003 18229 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.052s
I20260709 12:15:22.881587 18361 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:22.871233 18395 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.872925 18399 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.910214 18085 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:22.904353 18470 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:22.915485 18343 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:22.866647 18225 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:22.921023 18475 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:22.921022 18346 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:22.927213 18498 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.900825 18320 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:22.828909 18296 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.903229 18302 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.964855 18389 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:22.966360 18485 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:15:22.966619 18402 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:22.969815 18399 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:22.972388 18489 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:15:22.975121 18470 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.977429 18470 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:22.979461 18393 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:22.982372 18371 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:22.984613 18395 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:22.952862 18498 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:22.992551 18498 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:22.993460 18389 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:23.010525 18484 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:22.778745 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:15:23.206542 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 427694us
W20260709 12:15:21.824522 11074 rpcz_store.cc:269] 0709 12:15:05.819467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.821790 (+  2323us) service_pool.cc:224] Handling call
0709 12:15:06.439608 (+617818us) spinlock_profiling.cc:243] Waited 7.94 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:21.824009 (+15384401us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:14.903534 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:14.904264 (+   730us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:14.904326 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:15:14.904351 (+    25us) tablet.cc:662] Decoding operations
0709 12:15:14.998487 (+ 94136us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:14.998593 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:14.998625 (+    32us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:15:15.000499 (+  1874us) tablet.cc:701] Row locks acquired
0709 12:15:15.000557 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:15:15.000772 (+   215us) write_op.cc:270] Start()
0709 12:15:15.000953 (+   181us) write_op.cc:276] Timestamp: P: 1783599315000683 usec, L: 0
0709 12:15:15.000995 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:15:15.070448 (+ 69453us) log.cc:844] Serialized 7877 byte log entry
0709 12:15:15.558282 (+487834us) op_driver.cc:464] REPLICATION: finished
0709 12:15:15.610649 (+ 52367us) write_op.cc:301] APPLY: starting
0709 12:15:15.610802 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:16.015799 (+404997us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:16.015965 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:16.624060 (+608095us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:16.624624 (+   564us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 12:15:16.624709 (+    85us) write_op.cc:312] APPLY: finished
0709 12:15:16.625383 (+   674us) log.cc:844] Serialized 180 byte log entry
0709 12:15:16.890844 (+265461us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:16.891064 (+   220us) write_op.cc:454] Released schema lock
0709 12:15:16.891141 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:16.891555 (+   414us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:16.891589 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":356677,"reported_tablets":32,"spinlock_wait_cycles":18271488,"child_traces":[["op",{"apply.queue_time_us":51930,"apply.run_cpu_time_us":83344,"apply.run_wall_time_us":1280974,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87887,"num_ops":20,"prepare.queue_time_us":233847,"prepare.run_cpu_time_us":67063,"prepare.run_wall_time_us":393494,"raft.queue_time_us":121149,"raft.run_cpu_time_us":8849,"raft.run_wall_time_us":107959,"replication_time_us":519981,"thread_start_us":530244,"threads_started":5,"wal-append.queue_time_us":625991,"wal-append.run_cpu_time_us":13486,"wal-append.run_wall_time_us":2305732}]]}
I20260709 12:15:23.046241 11173 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:22.766172 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:22.717267 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 999689us
I20260709 12:15:23.211345 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.048640 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1168) took 1157 ms. Trace:
I20260709 12:15:23.056245 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:22.773180 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:22.816203 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:22.805403 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:23.077246 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:22.386348 11075 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 10 to 13, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 13 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:15:23.079510 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.077324 11179 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
W20260709 12:15:22.807011 11250 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.115s	user 0.012s	sys 0.010s
I20260709 12:15:23.090056 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.090085 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:22.829836 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3005) took 1296 ms. Trace:
I20260709 12:15:23.091521 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.091555 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.093775 11255 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:15:23.096133 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:22.862435 11322 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:22.813918 11323 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:23.106660 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3214) took 1083 ms. Trace:
I20260709 12:15:22.825443 11325 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
W20260709 12:15:22.716337 11326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.008s	sys 0.006s
I20260709 12:15:23.330495 11181 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:23.330829 11326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.680s	user 0.008s	sys 0.006s
I20260709 12:15:23.118387 11328 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:22.824463 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2787) took 4008 ms. Trace:
I20260709 12:15:23.333734 11329 rpcz_store.cc:276] 0709 12:15:18.816495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.819096 (+  2601us) service_pool.cc:224] Handling call
0709 12:15:18.819472 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.749085 (+929613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.824032 (+3074947us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54002855d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:22.824380 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096525184}
W20260709 12:15:22.809548 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.005s	sys 0.012s
I20260709 12:15:23.338164 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3212) took 1382 ms. Trace:
I20260709 12:15:23.338217 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3223) took 1215 ms. Trace:
I20260709 12:15:23.338418 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.338860 11323 rpcz_store.cc:276] 0709 12:15:22.123238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.123598 (+   360us) service_pool.cc:224] Handling call
0709 12:15:23.338005 (+1214407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65664}
I20260709 12:15:23.339598 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.126466 11327 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
W20260709 12:15:23.340785 11327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3209) took 1444 ms (client timeout 1812 ms). Trace:
I20260709 12:15:23.132964 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1319) took 4236 ms. Trace:
W20260709 12:15:23.341224 11327 rpcz_store.cc:269] 0709 12:15:21.896720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.896879 (+   159us) service_pool.cc:224] Handling call
0709 12:15:23.340693 (+1443814us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.342875 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.341729 11929 rpcz_store.cc:276] 0709 12:15:18.896856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.897007 (+   151us) service_pool.cc:224] Handling call
0709 12:15:18.897357 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.902046 (+  4689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:18.902075 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:18.902616 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:15:21.362559 (+2459943us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b180039ef20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:22.723218 (+1360659us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:22.723258 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.723454 (+   196us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.132232 (+408778us) raft_consensus.cc:1685] finished
0709 12:15:23.132278 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.132675 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6700,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":12424,"spinlock_wait_cycles":4402432,"thread_start_us":1360434,"threads_started":1,"wal-append.queue_time_us":3979063}
I20260709 12:15:23.134479 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1141) took 1741 ms. Trace:
I20260709 12:15:23.344434 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.344365 11931 rpcz_store.cc:276] 0709 12:15:21.393439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.404789 (+ 11350us) service_pool.cc:224] Handling call
0709 12:15:21.412345 (+  7556us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:21.670965 (+258620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.671011 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:21.692486 (+ 21475us) log.cc:844] Serialized 52 byte log entry
0709 12:15:22.731926 (+1039440us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:22.731983 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.892661 (+160678us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b540053a7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:22.892745 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.133705 (+240960us) raft_consensus.cc:1685] finished
0709 12:15:23.133750 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.134227 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":782,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":7952,"spinlock_wait_cycles":593285888,"thread_start_us":941804,"threads_started":1,"wal-append.queue_time_us":1139326}
I20260709 12:15:23.135869 18225 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:22.955708 18296 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:23.346231 11075 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:23.166476 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.349216 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.170742 18463 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:23.350384 18463 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.171831 18486 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.177717 18320 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:23.180429 18297 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:22.942256 18265 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.190232 11331 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:15:23.192397 11175 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:23.199128 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:23.201543 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.330901 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3226) took 1087 ms. Trace:
I20260709 12:15:23.201987 18396 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.209489 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 1181) took 1283 ms. Trace:
W20260709 12:15:23.233074 11173 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 1165) took 3437 ms (client timeout 2009 ms). Trace:
I20260709 12:15:23.212949 11176 rpcz_store.cc:276] 0709 12:15:21.891219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.931675 (+ 40456us) service_pool.cc:224] Handling call
0709 12:15:21.944136 (+ 12461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.047817 (+1103681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.048364 (+   547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.257159 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 1179) took 1358 ms. Trace:
I20260709 12:15:23.361841 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:23.258189 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.271021 18302 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:23.277443 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.363727 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 164910us
I20260709 12:15:23.274366 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1175) took 1377 ms. Trace:
I20260709 12:15:23.271865 18484 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.276229 18470 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.365893 18470 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.286620 18328 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:15:23.308750 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 97908.9us
I20260709 12:15:23.369408 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:23.361806 11174 rpcz_store.cc:276] 0709 12:15:21.926561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.726214 (+799653us) service_pool.cc:224] Handling call
0709 12:15:23.209243 (+483029us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.283789 18498 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:23.372618 18498 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:23.362114 11173 rpcz_store.cc:269] 0709 12:15:19.783025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.926446 (+143421us) service_pool.cc:224] Handling call
0709 12:15:23.212046 (+3285600us) spinlock_profiling.cc:243] Waited 3.12 s on lock 0x7b54000c0b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:23.220472 (+  8426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7166572928}
I20260709 12:15:23.329969 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3213) took 1215 ms. Trace:
W20260709 12:15:23.326444 11250 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.595s	user 0.012s	sys 0.010s
I20260709 12:15:23.364723 11179 rpcz_store.cc:276] 0709 12:15:21.895460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.049361 (+1153901us) service_pool.cc:224] Handling call
0709 12:15:23.076346 (+ 26985us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:15:23.076755 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.270085 (+193330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.272387 (+  2302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55722368}
I20260709 12:15:23.374282 18396 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.374416 11182 rpcz_store.cc:276] 0709 12:15:22.079091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.772847 (+693756us) service_pool.cc:224] Handling call
0709 12:15:23.287022 (+514175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.328068 11324 rpcz_store.cc:276] 0709 12:15:22.022808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.024918 (+  2110us) service_pool.cc:224] Handling call
0709 12:15:23.104749 (+1079831us) spinlock_profiling.cc:243] Waited 984 us on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7eee 000000000036eb39 00007f3ecb63b1e0 00007f3ecb7040c4 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:15:23.105249 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.105886 (+   637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.105923 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.106016 (+    93us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:23.106067 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.106190 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.106403 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290816}
I20260709 12:15:23.378980 18265 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:23.353453 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1376) took 1462 ms. Trace:
I20260709 12:15:23.381266 18498 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:23.331048 11249 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.010s	sys 0.018s
I20260709 12:15:23.336424 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:23.411327 11249 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.012s	sys 0.019s
I20260709 12:15:23.412106 11329 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:23.356494 18486 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:23.340801 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 263986us
I20260709 12:15:23.365033 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:23.371881 11175 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:23.361131 11331 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 13
I20260709 12:15:23.374397 18225 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:23.382108 18399 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:15:23.390483 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.345327 11327 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:23.386471 18470 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:23.394117 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.410127 11256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.000s	sys 0.025s
W20260709 12:15:23.388423 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.004s	sys 0.010s
I20260709 12:15:23.362279 11180 rpcz_store.cc:276] 0709 12:15:21.896449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.724503 (+828054us) service_pool.cc:224] Handling call
0709 12:15:23.055815 (+331312us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6b20a0 00007f3ecb6b1984 00007f3ec4c2c6d4 00007f3ec4c2c662
0709 12:15:23.255238 (+199423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45559040}
W20260709 12:15:23.395758 18473 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.066s	sys 0.128s
I20260709 12:15:23.390924 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:23.400745 11178 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 12:15:23.393776 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.000s	sys 0.012s
I20260709 12:15:23.361569 11325 rpcz_store.cc:276] 0709 12:15:22.243676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.759138 (+515462us) service_pool.cc:224] Handling call
0709 12:15:23.330635 (+571497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.333451 11253 rpcz_store.cc:276] 0709 12:15:21.532977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.788038 (+255061us) service_pool.cc:224] Handling call
0709 12:15:21.788331 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.887177 (+ 98846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.887206 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:21.887249 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:21.887282 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.829559 (+942277us) spinlock_profiling.cc:243] Waited 98.6 ms on lock 0x7b54002a67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:22.829607 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.829777 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226856704}
I20260709 12:15:23.354483 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.414053 11257 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:23.338723 11330 rpcz_store.cc:276] 0709 12:15:21.955900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.956161 (+   261us) service_pool.cc:224] Handling call
0709 12:15:22.023205 (+ 67044us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:22.023472 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:22.023494 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:22.024059 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:15:23.337433 (+1313374us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:23.337552 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.337658 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.337729 (+    71us) raft_consensus.cc:1685] finished
0709 12:15:23.337740 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.337979 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1312695,"threads_started":1,"wal-append.queue_time_us":825935}
I20260709 12:15:23.402660 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1373) took 1994 ms. Trace:
I20260709 12:15:23.414647 11251 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:23.415762 11331 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:15:23.393872 11252 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:23.387091 11328 rpcz_store.cc:276] 0709 12:15:21.891352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.891501 (+   149us) service_pool.cc:224] Handling call
0709 12:15:21.891780 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.350610 (+1458830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.353085 (+  2475us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400285fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:23.353381 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2819332480}
I20260709 12:15:23.417505 18225 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.435846 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.436250 18225 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:23.436491 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:23.425098 11248 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:23.438746 11253 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:15:23.433544 11331 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:23.435225 18296 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:23.435519 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:23.437897 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.440002 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.004s	sys 0.011s
I20260709 12:15:23.430768 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3227) took 1157 ms. Trace:
I20260709 12:15:23.355304 18463 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:23.451478 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.001s	sys 0.013s
I20260709 12:15:23.453025 11327 rpcz_store.cc:276] 0709 12:15:22.262504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.344042 (+1081538us) service_pool.cc:224] Handling call
0709 12:15:23.420169 (+ 76127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.454895 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 1182) took 1462 ms. Trace:
W20260709 12:15:23.458676 11256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.000s	sys 0.026s
I20260709 12:15:23.433027 11326 rpcz_store.cc:276] 0709 12:15:21.407795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.408691 (+   896us) service_pool.cc:224] Handling call
0709 12:15:21.443719 (+ 35028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.649936 (+206217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.649985 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:21.650544 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:15:23.401409 (+1750865us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:23.401509 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.402208 (+   699us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400285ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:23.402264 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.402312 (+    48us) raft_consensus.cc:1685] finished
0709 12:15:23.402326 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.402595 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299064704,"thread_start_us":1703976,"threads_started":1,"wal-append.queue_time_us":1262353}
W20260709 12:15:23.460476 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 95588.2us
I20260709 12:15:23.468209 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.468098 18396 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:23.470547 11323 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:23.470667 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.471701 18328 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:23.473973 11325 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:23.475627 18227 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:23.475863 11075 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:23.477272 18402 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:23.472836 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.481016 18471 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:23.481927 18471 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.482502 18399 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:23.484365 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:23.484751 18471 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.486217 18486 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:23.455447 11178 rpcz_store.cc:276] 0709 12:15:21.992150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.815845 (+823695us) service_pool.cc:224] Handling call
0709 12:15:23.438392 (+622547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12160}
W20260709 12:15:23.468581 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 747781us
I20260709 12:15:23.492138 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3003) took 1960 ms. Trace:
I20260709 12:15:23.492651 11249 rpcz_store.cc:276] 0709 12:15:21.532305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.646530 (+114225us) service_pool.cc:224] Handling call
0709 12:15:21.646881 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.647204 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.647234 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:21.647646 (+   412us) log.cc:844] Serialized 26 byte log entry
0709 12:15:22.726159 (+1078513us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:23.411944 (+685785us) log.cc:844] Serialized 14 byte log entry
0709 12:15:23.412425 (+   481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.412554 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.491848 (+ 79294us) raft_consensus.cc:1685] finished
0709 12:15:23.491865 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.492047 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":254778,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":615,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":225,"thread_start_us":1763509,"threads_started":2,"wal-append.queue_time_us":1778219}
I20260709 12:15:23.493727 18403 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 12:15:23.495111 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 2999) took 1972 ms. Trace:
W20260709 12:15:23.464816 18473 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.066s	sys 0.129s
I20260709 12:15:23.496063 18403 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:23.496440 18263 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:23.496567 18403 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:23.477937 18402 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:23.501672 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.018s	sys 0.021s
I20260709 12:15:23.503281 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:23.503923 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 766108us
I20260709 12:15:23.508284 18328 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:23.503886 18402 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.511137 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:23.514349 18402 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:23.514925 18402 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:23.515369 18402 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:23.507274 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.495535 11250 rpcz_store.cc:276] 0709 12:15:21.522461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.639812 (+117351us) service_pool.cc:224] Handling call
0709 12:15:21.651556 (+ 11744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.651715 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:23.374903 (+1723188us) log.cc:844] Serialized 14 byte log entry
0709 12:15:23.478566 (+103663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.490448 (+ 11882us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:23.490629 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.494843 (+  4214us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.495055 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5349,"prepare.queue_time_us":1275068,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":18147,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":9996,"thread_start_us":1767818,"threads_started":2}
I20260709 12:15:23.518944 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.519961 11327 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:23.520989 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.522531 18263 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:23.523141 18263 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:23.524073 18389 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.014s
I20260709 12:15:23.513444 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:23.524646 18389 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.525508 18302 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:23.525934 18302 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:23.526361 18302 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:23.528457 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1359) took 4555 ms. Trace:
I20260709 12:15:23.528651 18403 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.530520 18403 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:23.531474 11322 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
W20260709 12:15:23.532186 11322 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3218) took 1507 ms (client timeout 1996 ms). Trace:
W20260709 12:15:23.532542 11322 rpcz_store.cc:269] 0709 12:15:22.024378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.774229 (+749851us) service_pool.cc:224] Handling call
0709 12:15:23.532106 (+757877us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.504382 11328 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:23.533843 18389 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.531603 11327 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:23.534538 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.535435 11328 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:23.536656 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.507535 18470 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:15:23.539043 18470 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:23.539463 18470 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:23.540573 18470 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.541612 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.542623 11325 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 19: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:23.543107 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.543773 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.507903 11075 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:23.544436 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.529065 11331 rpcz_store.cc:276] 0709 12:15:18.966184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.169143 (+202959us) service_pool.cc:224] Handling call
0709 12:15:19.206949 (+ 37806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.473467 (+4266518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.521355 (+ 47888us) spinlock_profiling.cc:243] Waited 3.98 s on lock 0x7b5400323cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:23.521605 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9158719744}
I20260709 12:15:23.535704 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.545256 11075 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 12 to 15, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 15 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:23.546252 11327 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:23.546554 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.547446 11324 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:23.547302 11075 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:23.547995 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3225) took 1389 ms. Trace:
I20260709 12:15:23.548426 11324 rpcz_store.cc:276] 0709 12:15:22.158388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.546153 (+1387765us) service_pool.cc:224] Handling call
0709 12:15:23.547936 (+  1783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81152}
I20260709 12:15:23.549654 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.549808 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:23.550349 11324 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:23.550853 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.552043 11330 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:15:23.552378 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.552950 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.553392 18347 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:23.542496 18470 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:23.523788 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.553591 11330 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:23.556526 18484 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.559522 18496 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:15:23.561940 11147 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:23.562294 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:23.564126 18496 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:23.564589 18496 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:23.565253 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3219) took 1540 ms. Trace:
I20260709 12:15:23.565676 11322 rpcz_store.cc:276] 0709 12:15:22.024735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.564178 (+1539443us) service_pool.cc:224] Handling call
0709 12:15:23.564624 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.564921 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.564945 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.564986 (+    41us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:23.565014 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.565077 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.565174 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.560627 18484 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.567166 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.567916 11322 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:23.568805 18484 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:23.569532 18496 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.572806 11329 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:23.571610 18344 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:23.573014 18496 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:23.578761 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3217) took 1551 ms. Trace:
I20260709 12:15:23.537524 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3229) took 1211 ms. Trace:
I20260709 12:15:23.571297 18302 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.581938 18396 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:23.582458 18396 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:23.582932 18396 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:23.558096 18347 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:23.583916 18302 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.584133 18347 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.586750 18347 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:23.587257 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 116314us
I20260709 12:15:23.587525 18347 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:23.545311 11326 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:23.579607 11181 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:23.557399 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:23.558094 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44832.5us
I20260709 12:15:23.580279 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3216) took 1550 ms. Trace:
I20260709 12:15:23.412098 18085 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.579926 11328 rpcz_store.cc:276] 0709 12:15:22.326448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.536285 (+1209837us) service_pool.cc:224] Handling call
0709 12:15:23.537465 (+  1180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.546593 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.599190 11331 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:23.593683 11330 rpcz_store.cc:276] 0709 12:15:22.023668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.560639 (+1536971us) service_pool.cc:224] Handling call
0709 12:15:23.561256 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.561592 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.561623 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.573607 (+ 11984us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:23.573665 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.573725 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.573874 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.579486 11329 rpcz_store.cc:276] 0709 12:15:22.024007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.336002 (+1311995us) service_pool.cc:224] Handling call
0709 12:15:23.574932 (+238930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.604962 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3214) took 1393 ms. Trace:
I20260709 12:15:23.605079 11257 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:23.605706 11181 rpcz_store.cc:276] 0709 12:15:22.211289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.805085 (+593796us) service_pool.cc:224] Handling call
0709 12:15:23.600186 (+795101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.556578 11075 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 14 to 16, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 16 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:23.600096 18227 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:23.615351 18227 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.550912 18297 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:23.558703 18343 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.588176 18347 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:23.625144 11323 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:23.636574 11327 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:23.638281 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3004) took 2105 ms. Trace:
W20260709 12:15:23.640003 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.638741 11256 rpcz_store.cc:276] 0709 12:15:21.532657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.646976 (+114319us) service_pool.cc:224] Handling call
0709 12:15:21.647238 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.647431 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:22.719668 (+1072237us) log.cc:844] Serialized 14 byte log entry
0709 12:15:23.605254 (+885586us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.621704 (+ 16450us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:23.632037 (+ 10333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.638032 (+  5995us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.638212 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2879,"prepare.queue_time_us":1255367,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":1301,"raft.queue_time_us":96765,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":17893,"thread_start_us":1811451,"threads_started":2,"wal-append.queue_time_us":633371}
I20260709 12:15:23.644170 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:23.646181 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.647758 18344 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:23.647996 11327 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:23.648726 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3220) took 1569 ms. Trace:
I20260709 12:15:23.648478 18344 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.649088 11327 rpcz_store.cc:276] 0709 12:15:22.079091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.637715 (+1558624us) service_pool.cc:224] Handling call
0709 12:15:23.647780 (+ 10065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.648463 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.648665 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.650516 18343 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.651161 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3222) took 1536 ms. Trace:
I20260709 12:15:23.651542 11327 rpcz_store.cc:276] 0709 12:15:22.115069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.649972 (+1534903us) service_pool.cc:224] Handling call
0709 12:15:23.650394 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.650767 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.650795 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.650884 (+    89us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:23.650940 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.650995 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.651110 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.652254 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:23.652446 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:23.653067 18344 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:23.654728 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:23.657747 18475 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:23.658221 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3221) took 1563 ms. Trace:
I20260709 12:15:23.657864 18347 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.660648 18475 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:23.661101 18475 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:23.661784 11326 rpcz_store.cc:276] 0709 12:15:22.095067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.642095 (+1547028us) service_pool.cc:224] Handling call
0709 12:15:23.657648 (+ 15553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.657902 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.657927 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.657967 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:23.658015 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.658067 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.658167 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.664194 18389 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.665133 18389 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.670845 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:23.671181 18505 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:23.672999 18501 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:23.619184 18297 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:23.674088 18297 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:23.675102 18501 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:23.675666 18501 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:23.676100 18501 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:23.690671 18320 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.691427 18389 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:23.661444 18475 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:23.695453 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.697420 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.698355 11325 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:23.698853 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3224) took 1563 ms. Trace:
I20260709 12:15:23.699266 11327 rpcz_store.cc:276] 0709 12:15:22.128977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.652513 (+1523536us) service_pool.cc:224] Handling call
0709 12:15:23.652958 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.653225 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.653251 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.653310 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:23.653341 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.653395 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.692693 (+ 39298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.690142 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3228) took 1411 ms. Trace:
I20260709 12:15:23.701514 11326 rpcz_store.cc:276] 0709 12:15:22.279067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.663098 (+1384031us) service_pool.cc:224] Handling call
0709 12:15:23.663651 (+   553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.689568 (+ 25917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.689610 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.689714 (+   104us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:23.689758 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.689836 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.690044 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.702955 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.704974 11331 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:23.714135 11323 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:23.715706 11324 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:23.716262 11322 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:23.722493 11328 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:23.723839 11329 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:23.698318 18343 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:23.732263 11075 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:23.744197 11075 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 6 to 13, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 13 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:15:23.729585 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 245346us
I20260709 12:15:23.748735 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1378) took 1018 ms. Trace:
I20260709 12:15:23.749809 11329 rpcz_store.cc:276] 0709 12:15:22.729777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.686891 (+957114us) service_pool.cc:224] Handling call
0709 12:15:23.720538 (+ 33647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.724363 (+  3825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.747390 (+ 23027us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b54000c1cd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:23.747595 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71968000}
I20260709 12:15:23.745846 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.759109 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.704389 18263 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:23.786028 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 224223us
I20260709 12:15:23.795734 11330 raft_consensus.cc:2410] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 22 because replica has last-logged OpId of term: 20 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 16.
I20260709 12:15:23.799482 11327 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
W20260709 12:15:23.794796 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 206489us
W20260709 12:15:23.805126 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.805706 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:23.807672 18514 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:23.809522 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:23.758507 18328 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:23.723392 18225 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.826373 18225 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:23.830229 18320 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.740796 18302 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:23.823316 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1177) took 1927 ms. Trace:
I20260709 12:15:23.813068 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:23.823854 11180 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:23.822881 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:23.824318 18263 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:23.823360 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 177188us
W20260709 12:15:23.836211 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.842238 11323 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:23.840474 11331 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:23.818305 11075 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 14 to 16, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 16 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:23.844192 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 173476us
I20260709 12:15:23.847229 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2812) took 1067 ms. Trace:
I20260709 12:15:23.847882 11331 rpcz_store.cc:276] 0709 12:15:22.776565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.759834 (+983269us) service_pool.cc:224] Handling call
0709 12:15:23.840193 (+ 80359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.843886 (+  3693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.844174 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.850375 11329 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:15:23.851613 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2809) took 1075 ms. Trace:
I20260709 12:15:23.852174 11329 rpcz_store.cc:276] 0709 12:15:22.775710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.751576 (+975866us) service_pool.cc:224] Handling call
0709 12:15:23.820869 (+ 69293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:23.851158 (+ 30289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.851511 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.853924 11328 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:15:23.846053 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:23.839546 11174 rpcz_store.cc:276] 0709 12:15:21.895952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.370978 (+1475026us) service_pool.cc:224] Handling call
0709 12:15:23.371512 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:23.372137 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.372173 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:23.372833 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 12:15:23.607301 (+234468us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:23.607361 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.607461 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.823096 (+215635us) raft_consensus.cc:1685] finished
0709 12:15:23.823118 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.823250 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4474,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":233931,"threads_started":1,"wal-append.queue_time_us":421176}
I20260709 12:15:23.858171 11330 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:23.858290 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2806) took 1128 ms. Trace:
I20260709 12:15:23.860383 11328 rpcz_store.cc:276] 0709 12:15:22.730192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.731945 (+1001753us) service_pool.cc:224] Handling call
0709 12:15:23.833825 (+101880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.854407 (+ 20582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.858205 (+  3798us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.863474 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2815) took 1086 ms. Trace:
I20260709 12:15:23.863875 11330 rpcz_store.cc:276] 0709 12:15:22.777391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.817280 (+1039889us) service_pool.cc:224] Handling call
0709 12:15:23.854626 (+ 37346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.863250 (+  8624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.863427 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:23.868613 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 311293us
I20260709 12:15:23.869619 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1174) took 1920 ms. Trace:
I20260709 12:15:23.870419 11326 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:23.872493 18347 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:15:23.874609 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71950.7us
I20260709 12:15:23.874774 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.876991 11901 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:23.877543 18396 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.878598 11931 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:23.870163 11177 rpcz_store.cc:276] 0709 12:15:21.895203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.049373 (+1154170us) service_pool.cc:224] Handling call
0709 12:15:23.050252 (+   879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.050942 (+   690us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:15:23.286136 (+235194us) log.cc:844] Serialized 14 byte log entry
0709 12:15:23.814327 (+528191us) log.cc:844] Serialized 14 byte log entry
0709 12:15:23.814520 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.814568 (+    48us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:15:23.814614 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.814691 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.815343 (+   652us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":307603,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":93,"raft.queue_time_us":93058,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":53450752,"thread_start_us":717890,"threads_started":3,"wal-append.queue_time_us":487659}
I20260709 12:15:23.880200 18526 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:23.880606 18396 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:23.881032 18526 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:23.883709 18527 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:23.885295 18527 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:23.818967 18402 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:23.886466 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63760.2us
I20260709 12:15:23.849539 18085 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:23.857846 11075 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 12 to 14, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 14 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:15:23.896323 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57032.8us
I20260709 12:15:23.889818 18527 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:23.882696 18526 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.888809 18514 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:23.905840 18473 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:23.914224 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2816) took 1136 ms. Trace:
I20260709 12:15:23.917084 18402 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.916579 18527 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.921309 11325 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:23.924686 11323 rpcz_store.cc:276] 0709 12:15:22.778086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.845092 (+1067006us) service_pool.cc:224] Handling call
0709 12:15:23.898497 (+ 53405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.913814 (+ 15317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.913853 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.913933 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:23.913971 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.914035 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.914166 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.925805 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.928961 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:23.930289 11323 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:23.933751 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.935667 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61133.5us
I20260709 12:15:23.936352 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:23.939421 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:23.927729 18473 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:23.940393 11323 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:23.941572 18344 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:23.941924 11323 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:23.941481 18473 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.942103 18344 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:23.942775 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2818) took 1164 ms. Trace:
I20260709 12:15:23.943176 11323 rpcz_store.cc:276] 0709 12:15:22.778698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.941351 (+1162653us) service_pool.cc:224] Handling call
0709 12:15:23.941743 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.942483 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.942688 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.944928 11323 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:23.946632 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.947592 11075 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:23.910071 18399 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:23.953270 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2819) took 1170 ms. Trace:
I20260709 12:15:23.954231 11323 rpcz_store.cc:276] 0709 12:15:22.778983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.944205 (+1165222us) service_pool.cc:224] Handling call
0709 12:15:23.944669 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.945355 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.949445 (+  4090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:23.956048 18501 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.958153 18525 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:23.958017 18532 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:23.957485 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66597us
I20260709 12:15:23.959831 11327 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 18. (term mismatch)
I20260709 12:15:23.957367 18463 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:23.963290 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:23.983515 18463 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:23.963618 18527 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:23.962258 18502 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:15:23.964344 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:23.985607 11329 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:23.988086 11329 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:23.986208 18486 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:23.990139 18486 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:23.989885 18463 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:23.991231 18486 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.943295 18344 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.991973 18463 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:24.004263 11322 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:24.004436 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2814) took 1227 ms. Trace:
I20260709 12:15:24.004964 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2821) took 1218 ms. Trace:
I20260709 12:15:24.005394 11322 rpcz_store.cc:276] 0709 12:15:22.786502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.972929 (+1186427us) service_pool.cc:224] Handling call
0709 12:15:24.004087 (+ 31158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:24.004741 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:24.004917 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:24.006271 18344 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:24.007208 11326 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:23.925943 18347 consensus_queue.cc:1243] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:15:23.965467 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.966128 18516 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:15:23.972404 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2817) took 1194 ms. Trace:
I20260709 12:15:23.970714 18525 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:23.975481 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:23.972100 11327 pending_rounds.cc:85] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 17
I20260709 12:15:24.030140 11327 pending_rounds.cc:107] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 20.18
I20260709 12:15:24.030488 11327 raft_consensus.cc:2889] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:23.982164 18532 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.982513 18297 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:23.990688 18501 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:23.993223 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:24.004791 11325 rpcz_store.cc:276] 0709 12:15:22.777113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.809063 (+1031950us) service_pool.cc:224] Handling call
0709 12:15:23.822069 (+ 13006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.934105 (+112036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:24.004167 (+ 70062us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b5400322dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:24.004352 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227683712}
I20260709 12:15:23.999292 11075 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:15:23.964385 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:24.039181 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0469055830001253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 12:15:24.039680 11147 connection.cc:256] RPC call timeout handler was delayed by -1.0064055830000598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:15:23.999761 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:23.992733 18486 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:23.994031 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 56557.6us
I20260709 12:15:24.020346 11331 rpcz_store.cc:276] 0709 12:15:22.778393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.862466 (+1084073us) service_pool.cc:224] Handling call
0709 12:15:23.871084 (+  8618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:23.871586 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:23.900534 (+ 28948us) log.cc:844] Serialized 14 byte log entry
0709 12:15:23.970430 (+ 69896us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.972066 (+  1636us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:23.972127 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.972211 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.972338 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":84295,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":68,"raft.queue_time_us":112,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10175,"thread_start_us":88192,"threads_started":2}
I20260709 12:15:23.951431 18399 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:24.046043 18399 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:24.047403 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72026.3us
I20260709 12:15:24.047715 18473 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:24.048353 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:24.031145 11328 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:24.047269 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54162.3us
W20260709 12:15:24.049961 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:24.031085 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.043565 18470 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:24.048399 18473 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:24.043692 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:24.054067 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54450.4us
W20260709 12:15:24.054301 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.031625 18525 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:24.059636 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2823) took 1272 ms. Trace:
I20260709 12:15:24.060101 11323 rpcz_store.cc:276] 0709 12:15:22.787155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.972960 (+1185805us) service_pool.cc:224] Handling call
0709 12:15:24.044178 (+ 71218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:24.053135 (+  8957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:24.053163 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:24.059343 (+  6180us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:24.059386 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:24.059451 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:24.059576 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:24.060281 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.052390 18517 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:24.068137 18522 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.067818 18501 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:24.068334 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.068768 18522 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.069411 18501 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:24.070119 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.071884 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2820) took 1292 ms. Trace:
I20260709 12:15:24.072412 11327 rpcz_store.cc:276] 0709 12:15:22.779268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.958878 (+1179610us) service_pool.cc:224] Handling call
0709 12:15:23.959393 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:24.071518 (+112125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:24.071805 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:24.070274 18501 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:24.079921 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 109715us
I20260709 12:15:24.058734 18403 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:24.060396 18328 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:24.088424 18517 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:24.092231 18496 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:24.108676 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.070958 18470 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.110507 11223 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:24.097189 18328 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.102279 18265 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:15:24.126603 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 81569.9us
W20260709 12:15:24.138634 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 90403.9us
I20260709 12:15:24.079401 18347 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:15:24.119676 18263 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:24.148780 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.114593 18320 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:24.171988 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 101987us
I20260709 12:15:24.172544 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.134055 18517 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:24.175685 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:24.176553 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:24.181425 18526 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:24.181938 18526 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:24.182336 18526 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:24.183136 18540 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:15:24.184257 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.170737 18265 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:24.195205 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:24.198673 11257 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:24.175038 18263 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:24.202121 18535 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:15:24.188664 18396 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:24.208467 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:24.172293 18501 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.187002 18522 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:24.210180 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71161.2us
I20260709 12:15:24.215895 18541 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:15:24.216667 18227 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:24.216414 18265 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.217146 18227 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:24.217890 18328 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.218226 18227 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.219205 18328 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:24.219550 18227 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:24.208197 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.520s (SENT) [suppressed 8 similar messages]
I20260709 12:15:24.208930 18263 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:24.226343 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.228022 18526 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:24.228892 18537 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:15:24.189363 18473 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.230626 18526 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:24.215895 18534 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:15:24.234002 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54327.4us
I20260709 12:15:24.211822 18514 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:24.239763 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2807) took 1504 ms. Trace:
I20260709 12:15:24.240449 11324 rpcz_store.cc:276] 0709 12:15:22.730450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.746063 (+1015613us) service_pool.cc:224] Handling call
0709 12:15:23.816257 (+ 70194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:23.897817 (+ 81560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:23.897843 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:23.953396 (+ 55553us) log.cc:844] Serialized 26 byte log entry
0709 12:15:24.112259 (+158863us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:24.112323 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:24.112442 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:24.234428 (+121986us) raft_consensus.cc:1685] finished
0709 12:15:24.234443 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:24.234549 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28727,"raft.run_cpu_time_us":3829,"raft.run_wall_time_us":33558,"thread_start_us":71053,"threads_started":1,"wal-append.queue_time_us":151871}
I20260709 12:15:24.222821 18297 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:24.226230 18403 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:24.207176 18475 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.254184 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.255188 11929 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:15:24.252844 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:24.225410 18396 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:24.223917 18402 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:15:24.263343 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.249416 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:24.256615 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.265349 11322 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:24.220293 18389 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:15:24.240377 18514 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:24.272378 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 47606.7us
I20260709 12:15:24.273839 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:24.224119 18522 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:15:24.238663 18532 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:24.279026 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 79334us
I20260709 12:15:24.283295 11927 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:24.284983 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.285506 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.286461 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.287665 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.288110 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.262339 18403 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.270799 18402 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:24.289579 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:24.289961 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:24.290243 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:24.290958 18538 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:15:24.292197 11175 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:15:24.287706 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.308828 11181 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:24.284983 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:24.322865 11173 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:24.322855 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:24.326712 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52959.9us
I20260709 12:15:24.298802 18514 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:24.335186 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:24.250878 18399 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.307996 18402 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:24.289062 18396 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.325944 18470 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:24.348678 18521 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:24.349288 18521 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:24.347229 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.348800 18403 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.337170 18545 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:24.337492 18541 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:24.360447 11257 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:24.356571 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.358886 18522 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.367152 11173 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:24.369284 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.379385 18545 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.374420 18541 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.382359 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.382711 18541 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.380867 18545 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.371585 18516 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:24.390853 18516 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.391256 18516 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:24.393863 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.389825 18297 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:15:24.399448 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 191261us
I20260709 12:15:24.360929 18470 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:24.356587 18521 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:24.370442 18475 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:24.374259 18532 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:15:24.419391 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.397662 18302 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:24.431535 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.432976 11252 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:24.435103 18263 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.439509 11928 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:24.441025 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.410470 18470 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:24.444599 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 93108.4us
W20260709 12:15:24.446780 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 108144us
I20260709 12:15:24.449038 11933 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:24.404814 18522 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.456115 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.454277 11173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:24.457469 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.457952 18263 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:24.457489 11173 raft_consensus.cc:606] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.460911 11173 raft_consensus.cc:614] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:24.444859 18532 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.465710 11932 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:24.467717 18496 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:24.470057 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.456328 11900 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:24.470907 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.484309 11250 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:24.471081 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.471879 11255 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:24.471879 11248 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:24.473418 18522 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:24.477572 18470 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.487560 11180 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:24.488409 11252 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:24.491582 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.485569 11253 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:24.493645 18555 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:15:24.494182 18470 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:24.494342 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.502772 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:24.511525 11249 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:24.453675 18553 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:15:24.513391 11929 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:24.515596 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.516085 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:24.517899 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55033.1us
I20260709 12:15:24.526180 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.525864 18540 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:15:24.530789 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.512396 18501 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:24.531525 11176 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:24.538954 18556 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:24.539273 11251 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:24.514461 18328 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:15:24.544615 18554 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:24.545107 18554 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.546200 18554 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.548143 18554 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:24.548581 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.550886 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.551350 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.551751 11180 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:24.551676 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.552834 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:24.553700 11928 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:24.554682 11178 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 12:15:24.555085 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:24.556401 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.518703 18320 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:24.565557 11173 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
W20260709 12:15:24.570730 18544 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:15:24.577783 18532 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.580042 18541 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:24.580572 18541 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.582226 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.583353 11926 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:24.590194 18525 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 12:15:24.590852 18525 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.599346 11324 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:15:24.600392 18463 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:24.602275 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.602481 18347 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:24.601348 18463 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:24.603018 18347 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.603240 18463 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:24.604143 18463 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.604775 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:24.605312 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.607687 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.604079 18347 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.608960 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.609555 18347 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:24.609591 11248 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:24.611536 18517 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:24.611961 18517 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.612891 18517 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.618278 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.619228 18517 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:24.620236 18522 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:24.620028 18516 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.605508 18463 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:24.622071 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.622701 11257 raft_consensus.cc:2376] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22. Re-sending same reply.
I20260709 12:15:24.622896 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.623564 11256 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:24.539688 18556 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.623859 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:24.624639 11931 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:24.626488 11322 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:24.628093 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.630249 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.631088 11933 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:24.637370 11327 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:24.613472 18402 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.639629 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.640838 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.642922 18402 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:24.643570 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.644281 11179 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:24.645596 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.645401 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.647406 11331 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:24.650813 18521 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.613404 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.687234 11329 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:24.687876 18227 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:24.616427 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:24.688402 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.688875 11181 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:24.689157 18520 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:24.616765 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.617527 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.614078 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.690034 11324 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:24.690305 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.690485 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.654543 18556 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:24.655441 18347 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:24.690860 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.690932 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.691697 11251 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:24.691817 18347 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:24.692276 18347 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.692453 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.692786 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.657589 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.693703 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:24.658802 18542 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:15:24.659343 18553 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:24.660647 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.694502 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.660800 18540 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:24.695166 18403 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:24.618177 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:24.695464 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.662719 18555 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:24.695768 18403 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.695962 11182 raft_consensus.cc:606] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:24.696735 11324 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:24.693274 18347 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.697100 11324 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:24.688218 18498 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:15:24.696910 18403 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.698212 18347 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:24.698002 11324 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.698486 18403 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:24.698997 11324 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:24.657474 18541 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.668900 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.723803 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.726457 18541 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:24.707394 18521 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:24.670023 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:24.670511 18486 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.004s
I20260709 12:15:24.670562 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.672283 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.674088 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.674460 18516 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:24.675441 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.675643 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:24.675980 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.738366 11931 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:24.678193 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.593321 18399 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:24.679177 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.744616 11179 raft_consensus.cc:2410] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:15:24.593341 18525 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:24.679854 11222 connection.cc:256] RPC call timeout handler was delayed by -1.4841751429999022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 12:15:24.745522 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3617751429999316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 12:15:24.746071 11222 connection.cc:256] RPC call timeout handler was delayed by -1.1628751430000648s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1996s for a response
I20260709 12:15:24.659660 18514 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.620028 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:24.680692 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.750700 11173 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:24.681595 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.681988 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.682781 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.683465 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:24.684146 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.688134 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:24.689420 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.693910 11251 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:24.667624 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.695858 18540 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:24.700186 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.691004 11926 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:24.695547 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.658030 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:24.764771 11322 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:24.666453 11331 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:24.691067 11249 raft_consensus.cc:2376] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21. Re-sending same reply.
I20260709 12:15:24.689554 18520 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:24.703110 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.709715 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.710281 18263 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:24.711176 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.685186 11250 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:24.769567 11254 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:15:24.712577 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.770207 11254 pending_rounds.cc:85] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 13
I20260709 12:15:24.770581 11254 pending_rounds.cc:107] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:15:24.713980 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.770882 11254 raft_consensus.cc:2889] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:24.716857 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.772449 11181 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
W20260709 12:15:24.721932 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.773777 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:24.771380 11182 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:24.737704 18501 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.740667 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:24.754850 11933 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:24.793803 11248 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:24.793771 11255 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:24.796237 11325 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:24.798316 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:24.799355 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 179382us
I20260709 12:15:24.800096 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.793599 11934 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:24.799238 11180 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:24.808178 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:24.809955 11182 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:24.813408 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:24.814898 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.762326 18540 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:24.779829 18470 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:24.826046 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.827723 11173 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:24.828321 11324 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:15:24.829003 11324 pending_rounds.cc:85] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:15:24.829340 11324 pending_rounds.cc:107] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:15:24.830339 11324 raft_consensus.cc:2889] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:24.828219 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87635.7us
I20260709 12:15:24.823413 11930 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:24.833858 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.837419 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.838229 11181 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:24.845455 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44255us
I20260709 12:15:24.846377 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:24.838229 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:24.848526 11933 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:24.786638 18389 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
W20260709 12:15:24.848755 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.822809 18344 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:24.841997 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:24.857663 11180 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:15:24.858893 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.829744 18556 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:24.860415 11178 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:24.832629 18470 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:24.863811 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.806550 18501 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:24.926529 11249 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:24.927462 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:24.927937 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.931026 11326 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:15:24.872874 11257 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:24.937196 11180 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:24.943463 11248 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:15:24.945820 11180 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:15:24.838347 18496 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:24.947034 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:24.949788 11324 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:15:24.950467 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95699.7us
I20260709 12:15:24.869753 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.870434 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:24.954654 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 92914.9us
I20260709 12:15:24.872663 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:24.953214 11932 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:24.927114 18540 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:24.869575 18297 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:15:24.958156 11252 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:24.980527 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:24.960029 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:25.004927 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 158659us
W20260709 12:15:25.087502 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.846778 18525 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 12:15:24.859124 18526 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:24.863907 18399 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:25.014814 11176 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:24.925297 18396 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
W20260709 12:15:25.004894 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 162741us
I20260709 12:15:25.002252 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:25.078294 11255 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:24.879590 18470 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:24.993422 18513 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:24.959990 11256 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 16. Current term is 17. Ops: [13.8-16.10]
I20260709 12:15:25.008460 11326 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:24.855343 18520 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.000352 18522 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:25.038193 18347 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:15:25.103889 11322 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:15:25.109010 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 106891us
I20260709 12:15:25.110176 11322 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:15:25.115424 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.115221 11323 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:25.084223 18496 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:15:25.097363 18399 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.126987 11175 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:25.100698 18514 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:25.097985 18526 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:25.149811 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.155987 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.106169 18389 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:15:25.144130 18399 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:25.157929 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:25.160094 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:25.163101 11180 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:25.164064 11179 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:25.166324 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.145416 18526 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:25.149179 18545 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:25.173226 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:25.154844 18522 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:25.179459 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:25.183305 11929 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 17: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:25.185017 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:25.159032 18347 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.190913 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:25.195119 11180 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:25.157004 18297 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:15:25.176544 18501 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:25.191043 18566 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.180s	user 0.001s	sys 0.002s
I20260709 12:15:25.179330 18545 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.182597 18522 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:25.203858 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:25.212347 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46104.9us
I20260709 12:15:25.213917 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.199548 18347 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.200176 18576 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:25.195097 18514 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:25.203130 18525 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:15:25.254511 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47550.2us
W20260709 12:15:25.257987 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 109320us
W20260709 12:15:25.257042 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 43938.1us
I20260709 12:15:25.275024 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.211135 18545 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:25.278407 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.281250 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:25.283053 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74940.2us
I20260709 12:15:25.256029 18328 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:25.218710 18576 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.289719 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.290970 11178 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:25.209020 18555 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:15:25.300642 18525 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:25.300911 18578 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:15:25.302476 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:15:25.215931 18522 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.306243 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.280335 18263 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:25.310297 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:25.272243 18514 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:25.317273 18514 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:25.316934 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:25.318053 18525 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:25.295760 18576 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:25.328181 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.328828 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.205516 18513 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:15:25.329356 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.329573 11255 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:25.332581 11256 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:25.321476 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.323362 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:25.330149 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.330837 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.334196 11252 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:25.334285 11250 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 17.
I20260709 12:15:25.288786 18554 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:15:25.336272 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.337272 11253 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:25.338943 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.340945 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:25.341158 18579 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:15:25.341938 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.345386 18554 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:25.345566 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.345906 18554 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:25.346781 18554 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.347468 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.348028 18554 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:25.349241 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:25.350186 18541 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.004s
I20260709 12:15:25.350901 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.350085 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.348073 11255 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:25.359225 11256 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:15:25.358584 11251 raft_consensus.cc:2410] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:15:25.351760 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.351946 11174 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:25.364377 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.357300 11250 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
W20260709 12:15:25.364902 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66004.4us
I20260709 12:15:25.369170 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.318595 18587 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 12:15:25.379503 11296 connection.cc:256] RPC call timeout handler was delayed by -3.3632659969998713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 12:15:25.383858 11296 connection.cc:256] RPC call timeout handler was delayed by -3.1283659969999462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 12:15:25.384372 11296 connection.cc:256] RPC call timeout handler was delayed by -3.1274659969999448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 12:15:25.384706 11296 connection.cc:256] RPC call timeout handler was delayed by -2.0582659969998076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2997s for a response
I20260709 12:15:25.363265 18541 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:15:25.350986 18577 consensus_queue.cc:1175] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fecfc03de52a4308bc9b045cad5aac1e" responder_term: 22 status { last_received { term: 21 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:25.390812 18581 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:25.398247 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 76846.9us
I20260709 12:15:25.398860 18265 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:25.399609 18513 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:15:25.400102 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.369654 18590 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:25.401439 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.402227 18403 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:25.405022 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.368963 18347 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.408830 18545 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.410007 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.411221 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.411808 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1347) took 1003 ms. Trace:
I20260709 12:15:25.412497 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.413272 18532 raft_consensus.cc:993] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:25.375025 18328 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:25.417006 11935 raft_consensus.cc:1217] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Deduplicated request from leader. Original: 17.16->[18.17-20.18]   Dedup: 18.17->[20.18-20.18]
I20260709 12:15:25.448314 11935 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [18.17-20.18]
W20260709 12:15:25.419139 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.451750 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:25.452589 11327 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:25.419567 11177 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:25.421277 11255 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:25.426697 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.428898 11178 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:25.431367 18581 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.432473 11252 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:25.435048 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.437994 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:25.405252 18470 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.445797 11253 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:25.454337 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:25.439816 11927 rpcz_store.cc:276] 0709 12:15:24.408597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.408748 (+   151us) service_pool.cc:224] Handling call
0709 12:15:24.429779 (+ 21031us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:24.431436 (+  1657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:24.431466 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:24.449920 (+ 18454us) log.cc:844] Serialized 52 byte log entry
0709 12:15:24.820153 (+370233us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:24.820227 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:24.820329 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:25.408840 (+588511us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b24000311d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363dfb0 00007f3ec62b4488 00007f3ec62b4be8 00007f3ec628c398 00007f3ec628776a 00007f3ec6286bb6
0709 12:15:25.411416 (+  2576us) raft_consensus.cc:1685] finished
0709 12:15:25.411440 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:25.411724 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":9591,"spinlock_wait_cycles":18015488,"thread_start_us":314481,"threads_started":1,"wal-append.queue_time_us":619489}
I20260709 12:15:25.432663 18265 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:25.436239 18590 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.467402 18587 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:25.479033 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.479261 18470 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:25.481406 18584 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:25.467687 18525 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:15:25.482071 18584 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:25.482259 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 54474.5us
I20260709 12:15:25.438874 18403 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:25.484275 11181 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:25.447888 18328 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.453869 18541 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:25.458751 18347 raft_consensus.cc:1081] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:25.498044 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.498965 11250 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:25.496198 11254 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:25.464030 18389 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:25.509461 11933 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:25.477444 18587 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.484238 18590 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:25.518307 18265 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:25.519608 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92994.4us
I20260709 12:15:25.483070 18584 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.520349 18576 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.494005 18328 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:25.522209 18399 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.524349 18399 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:25.522209 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 232593us
I20260709 12:15:25.488050 18402 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:25.515286 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.520412 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:25.537533 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.530210 18587 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:25.563967 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 43629.4us
I20260709 12:15:25.564821 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.531687 18402 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:25.539410 18541 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:25.570027 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 58721.2us
I20260709 12:15:25.546355 18320 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:25.574729 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:25.574752 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.576335 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:25.561746 18265 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:25.585258 11326 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:25.572912 11327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:15:25.594105 11327 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:25.594952 11327 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:25.599118 11326 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:15:25.573416 18402 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:25.543479 18526 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.623585 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:25.624567 11329 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:25.630259 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46758.2us
I20260709 12:15:25.635993 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.592775 18527 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:25.636518 11180 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:25.636057 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71295.5us
W20260709 12:15:25.670558 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.670774 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.640782 18570 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:15:25.641392 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.641913 18576 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.647508 11932 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:25.570899 18403 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:25.653784 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.683416 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.654917 18263 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:25.687819 18532 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:25.688304 18532 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:25.688982 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.608676 18591 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.580444 18541 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.662425 11327 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.649400 18526 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.628494 18396 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:15:25.657686 18567 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.546s	user 0.001s	sys 0.003s
I20260709 12:15:25.685269 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.694908 18344 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:25.670773 18320 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:25.713003 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:25.724992 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:25.713620 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 77721.8us
I20260709 12:15:25.712258 11329 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
W20260709 12:15:25.723440 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 54592.1us
W20260709 12:15:25.726819 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55417.5us
I20260709 12:15:25.727653 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:25.729995 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.728890 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:25.689793 18570 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:25.696679 18496 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:25.737923 18399 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.670248 18593 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:15:25.670459 18521 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:15:25.743642 11327 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:25.723977 18396 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.747335 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:25.750162 18396 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:25.751902 11178 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:25.753119 11178 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:25.756526 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:25.757215 11180 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:25.757253 11927 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:25.766641 18421 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:15:25.766960 18601 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:15:25.745277 18526 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:25.769798 18599 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:15:25.770577 18421 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:25.771628 18421 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.772429 18541 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.771976 18570 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.774711 11257 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:15:25.775048 18595 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:25.776268 18586 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:15:25.777824 18586 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.778314 18586 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.778687 18602 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.779393 18602 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:25.780164 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 64987.7us
I20260709 12:15:25.780956 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.711586 18591 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.689227 18532 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.783764 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.790454 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1398) took 1072 ms. Trace:
W20260709 12:15:25.800259 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55786.2us
I20260709 12:15:25.783641 18496 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:15:25.788379 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:25.801527 11331 rpcz_store.cc:276] 0709 12:15:24.718065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.821173 (+103108us) service_pool.cc:224] Handling call
0709 12:15:24.839991 (+ 18818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:25.413779 (+573788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:25.413818 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:25.414537 (+   719us) log.cc:844] Serialized 26 byte log entry
0709 12:15:25.539994 (+125457us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:25.540085 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:25.540240 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:25.790227 (+249987us) raft_consensus.cc:1685] finished
0709 12:15:25.790248 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:25.790391 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":573300,"raft.queue_time_us":578782,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":8923,"thread_start_us":124893,"threads_started":1,"wal-append.queue_time_us":191935}
I20260709 12:15:25.809249 18581 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.813045 18522 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:25.780975 18402 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.749611 18399 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.815984 18402 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:25.816915 18399 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:25.814459 18532 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:25.819813 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1198) took 1009 ms. Trace:
I20260709 12:15:25.781837 18526 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:25.820170 11181 rpcz_store.cc:276] 0709 12:15:24.810960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.886798 (+ 75838us) service_pool.cc:224] Handling call
0709 12:15:24.946551 (+ 59753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:25.485067 (+538516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:25.819546 (+334479us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54001f40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:25.819737 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236184448}
I20260709 12:15:25.781576 18602 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.777264 18328 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.826786 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:25.828233 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:25.829434 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:25.797570 11927 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:25.834060 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.835366 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.835873 11248 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:25.835958 11181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:25.836557 11181 raft_consensus.cc:606] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.824920 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.814567 18591 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.839181 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.839486 18591 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:25.840399 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.840935 11250 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:25.841199 18328 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.839638 18586 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:25.839214 18514 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 12:15:25.843009 18586 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:25.842921 18514 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.844338 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3228) took 1169 ms. Trace:
I20260709 12:15:25.844779 11176 rpcz_store.cc:276] 0709 12:15:24.675266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.675390 (+   124us) service_pool.cc:224] Handling call
0709 12:15:25.844144 (+1168754us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400001ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:25.844250 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635249920}
I20260709 12:15:25.843015 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.854400 11934 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:25.801963 11178 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:25.876426 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.877823 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.878997 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:25.879591 11255 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:25.880420 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:25.880923 11255 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:25.884714 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 101032us
I20260709 12:15:25.885743 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.887143 11250 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:25.887980 11248 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:25.875180 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:25.887212 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.890180 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.854804 18402 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:25.863969 18526 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:25.893956 11250 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
W20260709 12:15:25.905769 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.917421 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:25.918663 11178 raft_consensus.cc:606] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.918964 11178 raft_consensus.cc:614] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:25.891507 18399 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:25.886272 18587 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:25.892541 18496 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:25.893172 18527 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:25.931869 18587 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.896901 18501 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:25.932806 18501 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.917405 18402 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:25.939893 18389 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:25.939687 18402 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.946848 18514 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:25.947731 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 72653.8us
I20260709 12:15:25.948562 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:25.950156 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab915731e0347ae8353a7f300c44401"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:25.950870 11178 raft_consensus.cc:606] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.951301 11178 raft_consensus.cc:614] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:25.933319 18501 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:25.959528 18496 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:25.960999 18496 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.962591 11250 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:25.964085 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69948.4us
I20260709 12:15:25.965891 11255 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:25.966673 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:25.980124 18389 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:25.982879 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:25.971143 18399 raft_consensus.cc:2764] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:15:25.925745 18570 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 12:15:25.987731 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:25.995651 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3233) took 1318 ms. Trace:
I20260709 12:15:25.996119 11175 rpcz_store.cc:276] 0709 12:15:24.676764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.797765 (+121001us) service_pool.cc:224] Handling call
0709 12:15:24.803273 (+  5508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:25.142933 (+339660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:25.995410 (+852477us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54000d7854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:25.995591 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743838464}
I20260709 12:15:25.999122 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:25.989145 18587 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:25.988468 18389 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:25.989490 18514 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.002492 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:26.042620 18590 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.002s
I20260709 12:15:26.046687 18590 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.986918 18396 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:25.990235 18617 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:26.029882 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:26.049728 18587 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.049954 18617 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.051476 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:26.052057 18587 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:26.052605 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:26.052925 18617 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:26.053545 18590 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:25.995600 18527 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
W20260709 12:15:26.054606 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 52202.2us
I20260709 12:15:26.055555 18527 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:15:26.045921 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58294.5us
I20260709 12:15:26.056984 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:26.036445 18399 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:26.063768 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:26.091929 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92883.4us
I20260709 12:15:26.093809 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:26.095391 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:26.099849 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:26.088753 18590 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:26.104642 18582 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:26.105612 18389 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.105892 18623 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:26.000408 18402 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:26.114639 18344 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:15:26.119134 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 62267.6us
I20260709 12:15:26.101264 18421 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:26.130199 18617 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.096872 18610 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:26.090158 18570 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:15:26.132112 18463 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:15:26.119436 18399 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:26.119119 18590 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:26.162956 18501 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.167546 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2666) took 1563 ms. Trace:
I20260709 12:15:26.179559 11177 rpcz_store.cc:276] 0709 12:15:24.604329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.604494 (+   165us) service_pool.cc:224] Handling call
0709 12:15:26.167301 (+1562807us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54002e54d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:26.167466 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068560128}
I20260709 12:15:26.172600 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:26.176309 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74473.5us
I20260709 12:15:26.182485 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:26.183133 11255 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:26.133168 18396 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:26.133525 18421 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:26.191236 11248 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:15:26.170327 18587 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.193249 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:26.194068 11248 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:26.191576 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 66779.9us
I20260709 12:15:26.196084 18610 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:26.197134 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:26.175704 18527 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:26.199934 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:26.201134 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:26.201519 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:26.201977 11248 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:26.194101 18421 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:26.202616 18463 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:26.213919 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:26.216879 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:26.217706 18513 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:26.219664 18513 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:26.220266 18517 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:26.220733 18517 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.220556 18513 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:26.221194 18517 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:26.196743 18521 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:15:26.222568 18521 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:26.218441 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:26.224386 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:26.224238 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51758.5us
I20260709 12:15:26.226029 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:26.174567 18617 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.226831 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:26.228222 11248 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:26.216595 18582 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:26.212646 18501 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:26.231019 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:26.222925 18521 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:26.230604 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:26.231962 18532 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:26.232362 18545 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:26.230239 18463 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:15:26.227115 18595 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:15:26.250461 18595 consensus_queue.cc:1243] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:15:26.218753 18590 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.253793 11933 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:26.247278 18421 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.242156 18521 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:26.244822 18302 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:26.199075 18587 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.253705 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:26.244230 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:26.257918 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:26.248871 18614 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:26.254842 18545 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:15:26.259392 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:26.263865 18302 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:26.264122 18532 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:26.265261 18463 raft_consensus.cc:2764] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:15:26.265789 18263 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:26.266659 18402 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:26.292212 18263 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:26.268595 18514 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:26.293378 11931 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:26.276130 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:26.294425 11330 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 24
W20260709 12:15:26.276995 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.279886 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:26.283963 11930 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:26.284322 18302 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:26.299788 18532 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.287721 18614 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:26.302838 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.302642 18614 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:26.303418 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.304309 18617 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:26.290146 11331 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:26.290977 18521 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:26.307156 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:26.291594 18616 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:26.291888 18470 raft_consensus.cc:993] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:26.286391 11323 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:26.287060 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:26.297992 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:26.287212 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:26.285934 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:26.320134 11929 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.321774 11248 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:26.305238 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:26.307804 11175 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.329958 11933 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:26.316396 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:26.339509 11927 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:26.379469 11933 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:26.382899 11929 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
W20260709 12:15:26.386673 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 106902us
I20260709 12:15:26.387244 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.388383 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:26.389281 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 158334us
W20260709 12:15:26.407326 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.383291 11330 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:26.418057 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.419260 11181 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:26.419608 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.420341 11175 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:26.421180 11331 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:26.424439 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.283529 18610 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:26.425076 11323 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:26.428718 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.429793 11327 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:26.429857 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:26.453166 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 155293us
I20260709 12:15:26.453418 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:26.283412 18517 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:26.422091 18402 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:26.465783 11331 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:26.469192 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:26.470249 11926 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:26.428824 18610 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:26.382457 18302 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:26.475660 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:26.460675 18602 raft_consensus.cc:1081] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:26.508211 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:26.510442 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:26.511159 11323 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:15:26.424986 18389 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:15:26.308565 18263 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.510484 11181 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:26.287631 18527 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.461388 18525 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:26.467989 18616 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:26.466233 18587 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:26.475006 18545 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:26.507215 18402 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:26.531812 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.534900 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.535622 11331 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:26.535895 11322 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:26.529516 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:26.536216 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 82150.3us
I20260709 12:15:26.536782 11324 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:26.478978 18610 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.564401 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.555035 18421 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:26.537003 18545 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.552637 18263 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:26.601938 11174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3226) took 1989 ms (client timeout 1949 ms). Trace:
I20260709 12:15:26.532307 18593 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:15:26.578641 18616 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:26.597946 18602 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:15:26.601251 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 213962us
I20260709 12:15:26.620049 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:26.620965 11322 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:26.604465 11324 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:26.610742 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:26.611526 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:26.622083 11329 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.607982 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:26.602967 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 45565.5us
I20260709 12:15:26.562827 18609 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:26.600976 18526 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:15:26.602409 11174 rpcz_store.cc:269] 0709 12:15:24.612735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.673799 (+ 61064us) service_pool.cc:224] Handling call
0709 12:15:26.601700 (+1927901us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:26.601867 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427001088}
I20260709 12:15:26.535405 18402 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:26.616014 11323 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:26.640533 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:26.615947 18399 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:26.650734 11331 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
W20260709 12:15:26.635511 18619 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.671s	user 0.005s	sys 0.000s
I20260709 12:15:26.616595 18470 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:26.635876 18593 consensus_queue.cc:1243] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:15:26.661171 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 53318.6us
I20260709 12:15:26.629693 18609 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:26.666146 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:26.667102 11933 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:26.667958 11248 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:26.668051 11178 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:26.660600 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:26.670123 11253 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:26.651882 18402 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:26.678941 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 1043) took 1968 ms. Trace:
I20260709 12:15:26.688114 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:26.617997 18525 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:15:26.691515 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:26.693109 11322 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:26.695689 11329 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
W20260709 12:15:26.688509 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 218392us
I20260709 12:15:26.688870 11248 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:26.688505 11935 rpcz_store.cc:276] 0709 12:15:24.710993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.711209 (+   216us) service_pool.cc:224] Handling call
0709 12:15:25.416625 (+705416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:25.458137 (+ 41512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.671702 (+1213565us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b540041b750. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:26.678832 (+  7130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1621363456}
I20260709 12:15:26.691514 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:26.699960 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.637754 18526 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:26.699153 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:26.705170 11178 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:26.713359 11248 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:26.653981 18399 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:26.669268 18587 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.670137 18522 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:26.707406 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:26.723443 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:26.723578 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:26.719983 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:26.725474 18526 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.726649 18526 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:26.726843 18649 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:26.727437 18649 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.727831 18649 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:26.728997 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.730214 11322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:15:26.698240 18302 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:26.719959 18631 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
W20260709 12:15:26.785840 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 61050.3us
I20260709 12:15:26.787942 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:26.791013 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:26.791975 11181 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:26.768896 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 137057us
I20260709 12:15:26.706966 18517 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:26.789932 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66546.8us
I20260709 12:15:26.669621 18470 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:26.770526 11322 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:26.811415 11322 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:26.794656 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3269) took 1102 ms. Trace:
W20260709 12:15:26.764259 18625 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:15:26.762221 18582 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:26.799139 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:26.837812 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49993us
I20260709 12:15:26.813956 11326 rpcz_store.cc:276] 0709 12:15:25.692154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.692357 (+   203us) service_pool.cc:224] Handling call
0709 12:15:25.692738 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:25.730709 (+ 37971us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:25.730747 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:25.745963 (+ 15216us) log.cc:844] Serialized 26 byte log entry
0709 12:15:25.892143 (+146180us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:25.892186 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:25.892302 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:26.585150 (+692848us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:26.772073 (+186923us) raft_consensus.cc:1685] finished
0709 12:15:26.772097 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:26.794573 (+ 22476us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24331,"raft.queue_time_us":32239,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":11811,"spinlock_wait_cycles":23489920,"thread_start_us":106126,"threads_started":1,"wal-append.queue_time_us":139528}
W20260709 12:15:26.845813 18651 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:15:26.809028 18634 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:15:26.730062 18587 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:26.846349 11927 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:26.722761 18641 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.127s
W20260709 12:15:26.857353 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.857563 11322 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:26.865077 11255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.000s	sys 0.017s
W20260709 12:15:26.865464 11255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.000s	sys 0.017s
W20260709 12:15:26.866348 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:26.985177 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 186104us
I20260709 12:15:26.850373 18516 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:15:27.013571 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.021s	sys 0.059s
W20260709 12:15:27.013419 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 162697us
W20260709 12:15:26.854014 18648 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:15:27.017414 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.022s	sys 0.059s
W20260709 12:15:27.018887 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.019714 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 359218us
I20260709 12:15:27.021960 11322 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:27.027055 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3239) took 2305 ms. Trace:
I20260709 12:15:27.030578 18513 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:27.031062 18513 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:27.031484 18587 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:27.020852 18651 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:15:27.032030 18513 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:27.033154 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.035112 18470 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:27.036347 18645 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:27.036860 18645 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:27.019210 18521 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.701s	user 0.003s	sys 0.009s
I20260709 12:15:27.037242 18645 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:27.039721 18525 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.040297 18525 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.040938 18521 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:27.041548 18521 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.042680 18525 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:27.027580 11173 rpcz_store.cc:276] 0709 12:15:24.732141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.828530 (+ 96389us) service_pool.cc:224] Handling call
0709 12:15:24.861806 (+ 33276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:24.862087 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:25.110091 (+248004us) log.cc:844] Serialized 14 byte log entry
0709 12:15:25.248299 (+138208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:25.248357 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:25.248405 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:25.248471 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:25.554442 (+305971us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b18000911e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:27.020801 (+1466359us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b5400364a50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:27.020941 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126015,"prepare.queue_time_us":131640,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":90184,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":5028,"spinlock_wait_cycles":61240448,"thread_start_us":186836,"threads_started":3,"wal-append.queue_time_us":137915,"wal-append.run_cpu_time_us":1363,"wal-append.run_wall_time_us":1002646}
I20260709 12:15:27.054106 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:27.071532 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:27.049492 18516 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:27.074733 18516 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.077383 11177 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:27.077939 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:27.096451 11175 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:27.049957 18513 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:26.851620 18399 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.102226 11325 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:15:27.050181 18656 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:26.792733 18609 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.052426 18545 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:27.111007 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57028.9us
I20260709 12:15:27.116585 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:27.112448 11181 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
W20260709 12:15:27.113816 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.120908 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:27.069465 18532 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:15:27.122341 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.139284 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:27.116194 18540 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:27.150185 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.186537 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.213369 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.215538 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:27.188035 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.118366 18617 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:15:27.275537 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 87614.9us
I20260709 12:15:27.297616 11325 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 18: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:27.125317 18545 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:27.124387 18522 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:27.120133 18653 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:27.129168 18609 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:27.312405 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.274520 18517 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:15:27.323652 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47371.5us
I20260709 12:15:27.329479 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:27.296322 18616 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.330823 11181 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:27.299456 18586 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.296023 18617 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:27.314900 18522 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.317248 18545 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:27.127873 18402 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.342094 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:27.353518 11934 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:27.354480 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:27.348120 18617 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:27.355682 18328 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:27.355754 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:27.356272 18328 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:27.357103 18328 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:27.358780 18658 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:27.360714 18665 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:27.361573 18665 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.364071 18666 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:15:27.366614 18657 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:15:27.368887 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:27.369136 18661 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 12:15:27.369601 11330 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:27.370553 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:27.371049 11330 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:27.366636 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1166) took 1092 ms. Trace:
I20260709 12:15:27.341436 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:27.373330 11931 rpcz_store.cc:276] 0709 12:15:26.273729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.273853 (+   124us) service_pool.cc:224] Handling call
0709 12:15:26.287343 (+ 13490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.293946 (+  6603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.366357 (+1072411us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b54005afd54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:27.366563 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13435136}
I20260709 12:15:27.342939 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:27.348438 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:27.375478 11252 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:27.348722 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:27.349222 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:27.349264 11927 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:27.376850 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1129) took 1176 ms. Trace:
I20260709 12:15:27.324540 18603 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.116434 18656 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.331470 18587 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.373111 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.329089 18527 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:27.377578 11255 rpcz_store.cc:276] 0709 12:15:26.189330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.189657 (+   327us) service_pool.cc:224] Handling call
0709 12:15:26.199691 (+ 10034us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.201865 (+  2174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.201894 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.225339 (+ 23445us) log.cc:844] Serialized 26 byte log entry
0709 12:15:26.867655 (+642316us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:27.016493 (+148838us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.034136 (+ 17643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.034220 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.037069 (+  2849us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b1800312b80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff557 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:27.365146 (+328077us) raft_consensus.cc:1685] finished
0709 12:15:27.365298 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.365543 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18974,"prepare.queue_time_us":148825,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":25871,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":12859,"spinlock_wait_cycles":2646016,"thread_start_us":634305,"threads_started":1,"wal-append.queue_time_us":854826}
W20260709 12:15:27.404698 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4307956850002483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 12:15:27.402474 11935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.002s	sys 0.005s
I20260709 12:15:27.407495 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1350) took 1938 ms. Trace:
W20260709 12:15:27.414346 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4244956850002382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 12:15:27.406438 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3041) took 1935 ms. Trace:
W20260709 12:15:27.414928 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4109956850002163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
W20260709 12:15:27.415568 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3794956850001654s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
I20260709 12:15:27.416055 11330 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:15:27.416158 11296 connection.cc:256] RPC call timeout handler was delayed by -1.317395685000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 12:15:27.416260 11935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.002s	sys 0.005s
W20260709 12:15:27.416935 11296 connection.cc:256] RPC call timeout handler was delayed by -1.317395685000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 12:15:27.418072 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2975956850002603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 12:15:27.418642 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2714956850002181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 12:15:27.419050 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2183956850001323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 12:15:27.419483 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1904956850000872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 12:15:27.419888 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1715956850000566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 12:15:27.420307 11296 connection.cc:256] RPC call timeout handler was delayed by -1.161695685000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 12:15:27.420718 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1472956850002447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 12:15:27.421150 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1454956850002418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 12:15:27.421607 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1454956850002418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 12:15:27.422071 11296 connection.cc:256] RPC call timeout handler was delayed by -1.110395685000185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:15:27.422535 11296 connection.cc:256] RPC call timeout handler was delayed by -1.110395685000185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:15:27.422950 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1049956850001763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
W20260709 12:15:27.423357 11296 connection.cc:256] RPC call timeout handler was delayed by -1.0113956850002523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 12:15:27.424299 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:27.425825 11325 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:27.414543 11932 rpcz_store.cc:276] 0709 12:15:25.469569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.469909 (+   340us) service_pool.cc:224] Handling call
0709 12:15:25.505103 (+ 35194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:25.679036 (+173933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.407156 (+1728120us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54004c2554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:27.407412 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327014144}
I20260709 12:15:27.415462 11257 rpcz_store.cc:276] 0709 12:15:25.471244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.480206 (+  8962us) service_pool.cc:224] Handling call
0709 12:15:25.480552 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:25.775183 (+294631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.406003 (+1630820us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400088ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:27.406303 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675630976}
I20260709 12:15:27.417886 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.352250 18545 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:27.430092 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: true dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:27.432484 11927 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:27.431028 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:27.346235 18517 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:27.352998 18522 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:27.434026 11326 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:27.434188 11330 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:27.439319 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.448374 18517 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.449791 18659 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.450408 18659 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.452865 18659 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:27.453809 18644 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.549s	user 0.000s	sys 0.002s
I20260709 12:15:27.426395 18302 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:27.455257 18302 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.410725 18603 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.460335 18603 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:27.426735 18532 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:27.461158 18532 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.461536 18532 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:27.462666 18610 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.912s	user 0.009s	sys 0.007s
I20260709 12:15:27.463156 18610 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.347398 18586 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.343122 18616 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.412789 18666 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:27.474646 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.474992 11327 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:27.488514 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1132) took 1298 ms. Trace:
I20260709 12:15:27.488976 18660 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:27.491971 18263 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.883s	user 0.005s	sys 0.005s
I20260709 12:15:27.492051 18522 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.492998 11254 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 12:15:27.453434 18659 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.489020 11250 rpcz_store.cc:276] 0709 12:15:26.190300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.239040 (+ 48740us) service_pool.cc:224] Handling call
0709 12:15:26.270023 (+ 30983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.285030 (+ 15007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.285062 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.314154 (+ 29092us) log.cc:844] Serialized 26 byte log entry
0709 12:15:26.617002 (+302848us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:26.617044 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.617112 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.488236 (+871124us) raft_consensus.cc:1685] finished
0709 12:15:27.488263 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.488436 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6965,"raft.queue_time_us":9579,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":20126,"thread_start_us":169522,"threads_started":1,"wal-append.queue_time_us":300847}
I20260709 12:15:27.502002 11180 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:27.434309 18521 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:27.505411 11927 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:27.426170 18587 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.506172 18615 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:27.464592 18650 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:27.516736 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:27.519785 11322 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:27.522348 11322 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:27.524098 11327 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:27.524857 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.486752 18302 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:27.432673 18527 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:27.513885 18649 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.786s	user 0.008s	sys 0.005s
I20260709 12:15:27.534135 18587 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:27.537523 18666 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.539032 18586 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:27.542174 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 118019us
I20260709 12:15:27.531828 18653 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:27.550231 18522 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:27.552470 18526 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.825s	user 0.008s	sys 0.004s
I20260709 12:15:27.561009 18660 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.504050 18659 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:27.558329 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.507896 18616 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:27.563951 11324 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:27.519295 18615 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.529953 18650 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:27.571920 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:27.552984 18545 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.515273 18645 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.572300 18653 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.588702 18660 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.614224 18610 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:27.615600 18623 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:27.616077 18623 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:27.618661 18650 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.623349 18645 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:27.607229 18653 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:27.624485 18555 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.625082 18555 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.630890 18527 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:27.633190 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:27.601399 18615 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:27.567051 18649 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:27.637682 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.641219 11326 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
W20260709 12:15:27.634975 18667 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
W20260709 12:15:27.623390 18647 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.776s	user 0.000s	sys 0.010s
W20260709 12:15:27.659914 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 88107.4us
I20260709 12:15:27.660917 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:27.617038 18623 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.632036 18302 raft_consensus.cc:2764] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 12:15:27.667182 11174 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 12:15:27.674345 18520 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:27.677914 18643 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.678565 18643 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.679178 18532 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.680470 18526 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.681016 18643 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:27.681208 18681 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:27.681198 18527 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:27.681818 18681 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.682607 18527 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.684491 18681 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:27.684967 18527 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:27.681682 11901 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.686614 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1136) took 1032 ms. Trace:
I20260709 12:15:27.686748 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.687644 18344 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:27.688644 11901 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:27.687144 11254 rpcz_store.cc:276] 0709 12:15:26.654067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.654246 (+   179us) service_pool.cc:224] Handling call
0709 12:15:26.663076 (+  8830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:27.493351 (+830275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.686319 (+192968us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b5400278154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:27.686551 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907712256}
W20260709 12:15:27.670609 18513 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.007s
I20260709 12:15:27.691130 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1203) took 1382 ms. Trace:
I20260709 12:15:27.698693 11176 rpcz_store.cc:276] 0709 12:15:26.308859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.315909 (+  7050us) service_pool.cc:224] Handling call
0709 12:15:26.316281 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.622144 (+305863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.622177 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:27.549244 (+927067us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b2c00013008. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:27.549811 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:15:27.606298 (+ 56487us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:27.606338 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.606413 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.688350 (+ 81937us) raft_consensus.cc:1685] finished
0709 12:15:27.688370 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.691056 (+  2686us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162384,"raft.queue_time_us":218241,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":11244,"spinlock_wait_cycles":4226304,"thread_start_us":209161,"threads_started":2,"wal-append.queue_time_us":117783}
I20260709 12:15:27.690074 18610 raft_consensus.cc:2764] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:15:27.699472 18527 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:27.691233 18516 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:27.722820 11182 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
W20260709 12:15:27.734573 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.668818 18521 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:15:27.681066 18526 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.744015 18630 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:27.745944 18513 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.744812 18527 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:27.748152 18660 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:27.792709 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.805029 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 144220us
I20260709 12:15:27.806314 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:27.807307 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 142764us
I20260709 12:15:27.809867 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:27.810940 18689 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:27.819769 11178 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:27.777390 18513 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:27.829591 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:27.836619 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:27.844563 18664 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:15:27.852074 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.870451 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2873) took 1417 ms. Trace:
I20260709 12:15:27.871059 11327 rpcz_store.cc:276] 0709 12:15:26.452813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.453114 (+   301us) service_pool.cc:224] Handling call
0709 12:15:27.870188 (+1417074us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000e6854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:27.870362 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347896704}
W20260709 12:15:27.877045 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 188463us
I20260709 12:15:27.869454 18344 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:27.834196 18649 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:27.882689 11327 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:27.825959 18691 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:27.878723 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:27.834899 18610 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:27.899910 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.884006 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54515.6us
I20260709 12:15:27.895123 18690 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:15:27.929780 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 101212us
W20260709 12:15:27.883154 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58236.6us
I20260709 12:15:27.930469 11331 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:27.930902 11331 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:27.933528 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:27.931469 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52837.5us
I20260709 12:15:27.936169 11323 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:27.939956 11929 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:15:27.941586 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:27.906711 18609 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:27.942358 11929 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:27.908329 18347 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.907088 18690 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:27.951860 11323 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:15:27.964547 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 58383.2us
I20260709 12:15:27.933043 18650 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.937281 18501 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:27.959964 11331 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.968581 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:27.972448 11331 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:27.934944 18666 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:27.960623 18526 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:27.995280 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:27.935230 18517 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
W20260709 12:15:28.027602 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54641.8us
I20260709 12:15:28.030316 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:28.031414 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.032087 11323 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:28.032927 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.033361 11323 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:28.050706 11323 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:27.972138 18501 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.963366 18609 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.060213 18609 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:27.965659 18347 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.062397 18347 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:28.062783 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.068981 18703 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:28.070711 18703 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:28.071287 18703 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.072853 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3275) took 1002 ms. Trace:
I20260709 12:15:28.079526 11329 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:28.095075 18513 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.073546 18703 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.097304 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.098183 11328 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:15:28.048696 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 110440us
W20260709 12:15:28.053404 18694 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:15:28.042469 18526 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:28.110863 18526 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.112223 11251 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:28.112869 18593 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.884992 18527 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:28.054203 18685 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:15:28.054934 18684 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:15:28.113354 18593 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.054253 18690 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:27.974479 18650 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.073299 11326 rpcz_store.cc:276] 0709 12:15:27.070875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:27.071099 (+   224us) service_pool.cc:224] Handling call
0709 12:15:28.072662 (+1001563us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54000e5954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:28.072792 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831232256}
I20260709 12:15:28.083393 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:28.084936 11180 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:15:28.105432 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 75044.7us
I20260709 12:15:28.105908 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:28.085798 18639 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.472s	user 0.004s	sys 0.006s
W20260709 12:15:28.087323 18617 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.001s
I20260709 12:15:28.043706 18610 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:28.117681 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.124346 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.025745 18666 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.113229 18526 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.120561 11176 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:28.126344 11176 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:28.135563 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:28.138556 11173 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:28.146733 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:28.148205 11175 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:28.149493 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:28.150290 11175 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:28.146318 11176 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.057698 18698 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 12:15:28.157727 18666 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:27.969523 18344 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.158785 11331 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:28.160717 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:28.139952 18520 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:28.161942 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:28.162204 18520 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:28.162312 18516 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.162607 11180 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:28.163015 18516 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.163254 18520 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.164047 11180 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:15:28.123327 18617 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.165035 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:28.102313 18501 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:28.169250 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:28.165411 18516 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.172086 18516 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.173065 18703 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.173803 18627 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:28.173879 18265 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.174489 18627 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.174621 18265 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.175046 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:28.176014 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:28.176635 18627 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.176740 18660 raft_consensus.cc:2764] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:15:28.177297 18666 raft_consensus.cc:2764] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:15:28.120026 18705 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:28.082820 18653 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.195187 18650 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.197854 18705 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.177134 18265 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.199421 18653 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.201166 18705 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:28.207621 18615 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.001s
I20260709 12:15:28.208103 18615 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.210889 18532 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:28.182545 18688 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 12:15:28.173610 18703 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.177155 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:28.189240 18501 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.190657 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:28.191349 18704 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:28.224231 18704 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.224640 18704 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:28.195149 18697 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.225s	user 0.000s	sys 0.000s
W20260709 12:15:28.196008 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2688) took 2832 ms (client timeout 2175 ms). Trace:
I20260709 12:15:28.138556 18347 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.179179 18263 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:28.172924 18516 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.205222 18545 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.221998 18610 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:28.223579 18703 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:28.197580 18687 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 12:15:28.223762 18501 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.177561 18691 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:28.193137 18665 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:28.225687 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44589.3us
I20260709 12:15:28.251399 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:28.253085 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:28.253244 18705 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:28.229755 11182 rpcz_store.cc:269] 0709 12:15:25.363968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.364138 (+   170us) service_pool.cc:224] Handling call
0709 12:15:28.195721 (+2831583us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b54000d8754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:28.195906 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5214319872}
I20260709 12:15:28.231459 18265 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:15:28.238031 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 150433us
I20260709 12:15:28.242290 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:28.298863 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.299010 11323 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:28.227488 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:28.254745 18627 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.255509 18389 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:28.256776 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.264387 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.257531 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:28.300472 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47491us
I20260709 12:15:28.303963 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:28.311909 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.265473 18666 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:28.306418 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:28.371627 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65295.9us
I20260709 12:15:28.372826 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:28.250500 18691 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:28.264490 18702 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:28.234539 18347 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.303769 18501 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:28.376375 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 148999us
I20260709 12:15:28.302162 18660 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:28.297017 18265 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:28.378816 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:28.365965 18617 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.377449 18665 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:28.377621 18691 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:28.399094 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95206us
I20260709 12:15:28.380985 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:28.370819 18630 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:28.296496 18705 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.378085 18653 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:28.379896 18545 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:28.436921 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:28.439211 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:28.364180 18389 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:28.441874 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 178380us
I20260709 12:15:28.443239 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.401515 18265 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:28.307365 18627 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.445410 18653 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:28.501652 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:28.377702 18703 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:15:28.472033 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.406347 18707 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:28.406678 18517 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:28.447281 18615 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.435088 18545 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:28.476529 18706 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:15:28.390796 18702 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.400615 18501 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.506047 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.518775 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:28.524585 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 85487.3us
I20260709 12:15:28.444458 18263 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:15:28.477541 18527 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:15:28.535595 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.567826 11331 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:28.436308 18630 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.504040 18665 raft_consensus.cc:2764] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:15:28.508018 18703 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:28.573452 11929 raft_consensus.cc:2368] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:28.575706 11932 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:15:28.537027 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.570894 11326 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:28.576871 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:28.576579 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 188181us
I20260709 12:15:28.478302 18711 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.120s
I20260709 12:15:28.523474 18521 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:28.523488 18664 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.520082 18517 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.578352 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:28.580476 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:28.581825 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.585224 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:28.588325 11331 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:28.589802 11326 raft_consensus.cc:2376] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19. Re-sending same reply.
I20260709 12:15:28.591177 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:28.591912 11326 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:28.532064 18545 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:28.601068 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.576995 18703 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:28.579769 18695 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:28.593250 18517 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:28.626292 18713 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:28.626960 18713 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:28.628248 18713 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.628300 18516 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.620930 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:28.631064 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.632827 18723 raft_consensus.cc:2764] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 12:15:28.635797 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:28.598905 18520 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:28.603696 18595 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:28.519722 18707 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.603401 18521 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.594153 18664 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.802670 11249 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:28.813607 11176 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:28.728394 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:28.802413 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2655) took 3402 ms (client timeout 1789 ms). Trace:
W20260709 12:15:28.698930 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.823268 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.820415 11251 rpcz_store.cc:269] 0709 12:15:25.400142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.401103 (+   961us) service_pool.cc:224] Handling call
0709 12:15:28.802127 (+3401024us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b5400108e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:28.802303 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6153308928}
I20260709 12:15:28.607728 18665 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:28.606778 18726 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.773267 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:28.681098 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46046.3us
I20260709 12:15:28.730602 11254 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:15:28.830366 11327 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:28.729493 11250 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:28.818869 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3256) took 1987 ms. Trace:
I20260709 12:15:28.831408 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.831657 11174 rpcz_store.cc:276] 0709 12:15:26.831502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.837229 (+  5727us) service_pool.cc:224] Handling call
0709 12:15:26.847184 (+  9955us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:27.688775 (+841591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:28.731143 (+1042368us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54000da054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:28.818806 (+ 87663us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1885052672}
I20260709 12:15:28.832973 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:28.833597 11327 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:28.833909 11331 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:15:28.640293 11326 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:28.736538 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:28.830479 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:28.834517 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:28.835752 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:28.836129 11327 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
W20260709 12:15:28.729097 18716 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:15:28.837483 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:28.607089 18653 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:28.866113 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.630307 18712 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:15:28.838428 11257 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:28.841336 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:28.687906 18719 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:15:28.879689 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:28.880669 11927 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:28.868826 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:28.839524 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.811565 18582 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.803303 18713 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:28.820695 18556 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:28.837765 18643 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.893193 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:28.687407 18723 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:28.689296 18347 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.868086 18662 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:28.933519 18403 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:28.933854 18662 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:28.935271 18662 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:28.875443 18593 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.941241 11328 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:15:28.946079 18470 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.946746 18470 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:28.948735 18704 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.000s
I20260709 12:15:28.950035 18402 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:28.956315 18399 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:28.961066 18590 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:28.884359 18532 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:28.963372 18532 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.966182 11328 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:15:28.843612 18660 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:28.883100 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:28.722524 18702 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.680333 18545 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.835590 18516 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.619202 18463 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.897209 18723 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:28.897818 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:28.898346 18722 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:15:28.899554 18720 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 12:15:28.900277 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:28.900586 11931 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:28.901186 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:28.902472 11930 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 12:15:28.903117 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:28.903894 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:28.848541 18726 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.915318 18664 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.917486 18521 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.917995 11327 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:15:28.968645 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 85663.2us
I20260709 12:15:28.869306 18609 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:28.835337 18595 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.621721 18695 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.103538 11328 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
W20260709 12:15:29.124577 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:29.073141 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.136053 11930 pending_rounds.cc:85] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 17
I20260709 12:15:29.023703 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:28.949393 18704 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.150179 11930 pending_rounds.cc:107] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 24.18
W20260709 12:15:29.042752 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.151245 11930 raft_consensus.cc:2889] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:29.152410 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 128811us
I20260709 12:15:28.975665 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:28.988734 18718 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.332s	user 0.003s	sys 0.000s
I20260709 12:15:28.975672 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1205) took 2665 ms. Trace:
I20260709 12:15:28.977330 18545 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.990821 18702 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.166747 18344 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:29.175879 18532 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.175985 18344 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:29.176370 18556 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:29.176923 18556 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:29.177066 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.178416 11932 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:29.177059 18344 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.182400 18702 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.183210 18595 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.991536 18699 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:29.184063 18699 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.184939 18702 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.976094 18463 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.186478 18699 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.187716 18463 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.999747 18656 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.188019 18707 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:28.997040 18659 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.190651 18695 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:28.984854 18516 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.198868 11176 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:29.199362 18704 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.199494 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2692) took 1116 ms. Trace:
I20260709 12:15:29.199944 11176 rpcz_store.cc:276] 0709 12:15:28.082943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.083130 (+   187us) service_pool.cc:224] Handling call
0709 12:15:28.813325 (+730195us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b58007625d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66480aa 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66f027c 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c
0709 12:15:29.199430 (+386105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108974464}
I20260709 12:15:29.200536 18516 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.173491 18628 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.201681 18628 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:29.205207 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.978524 18555 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.163954 18501 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:29.209877 18540 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 12:15:29.211611 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2882) took 2069 ms. Trace:
W20260709 12:15:29.212249 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.213496 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:29.217953 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.218729 18650 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:29.219467 18650 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:29.221243 18265 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.002s
I20260709 12:15:29.221812 18265 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.212015 11329 rpcz_store.cc:276] 0709 12:15:27.142589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:27.540945 (+398356us) service_pool.cc:224] Handling call
0709 12:15:28.079268 (+538323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:28.094479 (+ 15211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.211415 (+1116936us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54000e5950. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:29.211556 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237085440}
W20260709 12:15:29.223897 18666 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.846s	user 0.011s	sys 0.002s
W20260709 12:15:29.222993 18691 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.004s
I20260709 12:15:29.234794 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.234990 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.235797 11327 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:15:29.233050 18399 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.242419 18399 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.994004 18725 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:28.987630 18610 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.750s	user 0.011s	sys 0.001s
I20260709 12:15:29.071107 18660 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:29.274446 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.277051 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.278640 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.245283 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.236466 18691 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.250156 18730 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:29.259172 18520 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.001s
I20260709 12:15:29.233417 18628 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.260077 18650 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:29.261159 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.225077 18666 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:29.181332 18517 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.262881 18399 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.267441 18585 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:29.244171 18703 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.000s
I20260709 12:15:29.270264 18265 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.273358 11927 raft_consensus.cc:2410] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:15:28.893827 18582 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.319684 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.285046 18610 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:29.324877 11326 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:28.975409 18642 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:29.325481 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.329430 11254 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:29.171726 11179 rpcz_store.cc:276] 0709 12:15:26.310383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.310502 (+   119us) service_pool.cc:224] Handling call
0709 12:15:26.310937 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.428519 (+117582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.428552 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.429089 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:15:28.567106 (+2138017us) spinlock_profiling.cc:243] Waited 971 us on lock 0x7b18002dc940. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:28.819035 (+251929us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:28.819075 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:28.819771 (+   696us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:28.973868 (+154097us) raft_consensus.cc:1685] finished
0709 12:15:28.973906 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:28.975602 (+  1696us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172868,"raft.queue_time_us":136111,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":11055,"spinlock_wait_cycles":2233600,"thread_start_us":251727,"threads_started":1,"wal-append.queue_time_us":2467432}
I20260709 12:15:29.241655 11328 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:29.281348 11327 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:29.281852 11329 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:29.332120 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:29.287998 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.302436 18520 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:28.973143 18643 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.295794 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.308468 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.308940 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.316762 11250 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:29.320701 11252 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:29.234216 18609 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:28.974117 18347 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:15:29.323311 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:29.388078 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.323947 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.327021 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.389276 11255 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
W20260709 12:15:29.333395 11325 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2888) took 2026 ms (client timeout 1622 ms). Trace:
W20260709 12:15:29.389854 11325 rpcz_store.cc:269] 0709 12:15:27.307578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:27.324822 (+ 17244us) service_pool.cc:224] Handling call
0709 12:15:29.333162 (+2008340us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54000c67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:29.333330 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87632640}
I20260709 12:15:29.333992 11330 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:29.335755 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.344393 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.389482 11248 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
W20260709 12:15:29.391840 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47602.9us
I20260709 12:15:29.386364 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.391256 11323 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:29.393126 11327 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
W20260709 12:15:29.406028 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 70429.2us
I20260709 12:15:29.408949 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:29.414331 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.414741 11252 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:29.415172 11251 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:29.417047 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.417835 11325 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:29.419901 18347 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:29.421057 18347 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.421693 18553 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.422219 18553 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.424508 18553 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.425485 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.426332 11257 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:29.427158 18590 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:29.427733 18590 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:29.428159 18590 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:29.428889 18517 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.388264 18666 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.430753 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.431056 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:29.431488 11251 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:29.431617 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:29.323212 18703 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.433404 11251 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:29.434170 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:29.434763 18703 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.435322 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.280818 18660 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.414393 18653 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:29.436686 18653 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.437062 18653 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:29.438421 18660 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.439353 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.440812 18527 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:29.441195 18527 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.431618 11252 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:29.441543 18527 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:29.447091 18691 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.455529 18582 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.457849 18609 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.457975 11253 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:29.301276 18730 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.430569 18666 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.276036 18725 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.460386 18666 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.460386 18730 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.360611 18642 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.462199 18725 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.436017 11250 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:29.427259 18623 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:29.434845 11255 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:29.466764 11252 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:15:29.467609 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.022s	sys 0.027s
W20260709 12:15:29.468822 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.022s	sys 0.027s
I20260709 12:15:29.477367 11323 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:29.478847 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:29.487967 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 316456us
I20260709 12:15:29.489846 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.494608 11255 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:29.490588 11254 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:29.503057 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:29.482985 18665 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.010s	sys 0.038s
I20260709 12:15:29.516997 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.536732 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:29.542743 11251 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:29.543491 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.537971 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.499027 18553 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.322986 18585 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.498186 18725 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.515497 18662 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:29.554720 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3285) took 1294 ms. Trace:
W20260709 12:15:29.544602 18665 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.010s	sys 0.039s
W20260709 12:15:29.559880 11178 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3252) took 2836 ms (client timeout 1971 ms). Trace:
W20260709 12:15:29.564606 11178 rpcz_store.cc:269] 0709 12:15:26.723078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.723322 (+   244us) service_pool.cc:224] Handling call
0709 12:15:29.559593 (+2836271us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54002e3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:29.559779 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220616448}
W20260709 12:15:29.563807 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.568463 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:29.568549 11935 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:29.564260 18725 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.578051 18590 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.567633 18662 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.582113 18555 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:15:29.582598 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.583566 18735 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:29.584753 18642 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.585484 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:29.586447 18555 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:29.586896 18555 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.587286 18555 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:29.588819 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:29.588450 18556 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.589792 18532 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:29.592053 18532 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:29.593909 18659 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:29.594913 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:29.556973 18553 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.597182 18553 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.597357 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3277) took 2525 ms. Trace:
I20260709 12:15:29.598127 18553 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:29.598662 18553 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.599017 18553 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:29.600340 18532 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:29.597999 11330 rpcz_store.cc:276] 0709 12:15:27.071913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:27.438152 (+366239us) service_pool.cc:224] Handling call
0709 12:15:29.333801 (+1895649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.391347 (+ 57546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.597084 (+205737us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54000c67d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:29.597255 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4358494080}
I20260709 12:15:29.602016 18642 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.602805 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:29.602612 18642 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.603456 11330 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:29.603706 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:29.604549 18532 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:29.605103 18532 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.605588 18532 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:29.601557 18527 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.607788 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:29.580238 18590 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.611505 11251 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:29.611472 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:29.566215 18665 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:29.610289 18609 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:29.617456 11330 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:29.608466 18726 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.582141 18662 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.618642 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:29.620689 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:29.621352 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.617620 18698 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:29.628019 18698 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.629758 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.630518 11330 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:29.633030 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.632642 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3289) took 1327 ms. Trace:
I20260709 12:15:29.621172 18652 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:29.633919 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.633914 11222 leader_election.cc:400] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:29.611851 18659 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:15:29.622915 18735 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:29.647936 11222 leader_election.cc:403] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:15:29.650115 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.648442 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:29.611820 18653 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.649915 18662 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:29.653525 18609 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:29.666567 18662 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:29.666818 18609 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.667301 18662 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:29.668394 18609 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.647220 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:29.659765 18527 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.662581 18555 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.674234 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:29.647221 18652 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.665781 11330 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:29.670913 18399 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:29.672272 18556 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.630935 18347 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:29.677618 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.680583 11933 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:15:29.661175 18532 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.683387 18347 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:29.683753 18556 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:29.686220 18735 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:29.681686 18520 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:29.675624 18265 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:15:29.686554 18665 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.687198 18470 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.696766 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:29.634835 18545 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:15:29.688964 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.679227 18652 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:29.703049 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:29.703637 11322 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:29.685456 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:29.701059 11330 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:29.701056 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.692126 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2901) took 1328 ms. Trace:
I20260709 12:15:29.697398 11327 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:29.704558 11323 rpcz_store.cc:276] 0709 12:15:28.363987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.736130 (+372143us) service_pool.cc:224] Handling call
0709 12:15:29.691950 (+955820us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54000c08d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:29.692065 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269636352}
I20260709 12:15:29.706282 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.706809 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.707639 11926 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:29.707736 11323 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:29.708293 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2905) took 1263 ms. Trace:
W20260709 12:15:29.708354 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 1.971s (SENT) [suppressed 13 similar messages]
I20260709 12:15:29.708635 11323 rpcz_store.cc:276] 0709 12:15:28.444589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.705559 (+1260970us) service_pool.cc:224] Handling call
0709 12:15:29.707437 (+  1878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.708068 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.708235 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:29.706801 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.710146 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3292) took 1221 ms. Trace:
I20260709 12:15:29.710502 11323 rpcz_store.cc:276] 0709 12:15:28.488792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.709384 (+1220592us) service_pool.cc:224] Handling call
0709 12:15:29.709688 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.709864 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:29.709886 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:29.709924 (+    38us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:29.709951 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.709996 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:29.710101 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:29.714684 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.715329 11330 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:29.715942 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.716341 11330 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:15:29.716496 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.719800 11327 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:29.720427 11323 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:29.721309 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3294) took 1146 ms. Trace:
I20260709 12:15:29.721693 11327 rpcz_store.cc:276] 0709 12:15:28.575342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.720535 (+1145193us) service_pool.cc:224] Handling call
0709 12:15:29.720811 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.720979 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:29.721004 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:29.721046 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:29.721085 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.721133 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:29.721257 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:29.723793 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:29.687711 18627 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:29.726406 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.687772 18739 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.688208 18735 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:29.692358 18713 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.727541 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.688472 18520 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.729996 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:29.730293 11330 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:29.735186 11322 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:29.714612 18265 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.739856 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3293) took 1215 ms. Trace:
I20260709 12:15:29.714125 18517 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:29.714198 18726 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.725484 18623 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:29.737313 18555 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.750375 11330 rpcz_store.cc:276] 0709 12:15:28.524396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.715957 (+1191561us) service_pool.cc:224] Handling call
0709 12:15:29.716230 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.739612 (+ 23382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.739803 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:29.723688 18614 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:29.759193 11330 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:29.752553 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.687840 18556 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.735059 18659 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:29.763659 18659 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.688165 18347 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.764115 18659 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:29.755121 18726 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.771031 18623 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:29.755713 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.758839 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:29.738391 18735 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:29.775882 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:29.723908 18591 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:15:29.740602 18520 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:29.725963 18521 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:15:29.772595 18623 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.788620 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.781072 18556 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.772625 18726 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.778223 18545 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:29.737880 18739 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.749096 18650 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:15:29.779220 18614 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:29.756664 18265 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.742455 18713 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:29.794094 18627 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:29.792723 18516 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:29.819164 18545 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:29.824858 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 101007us
I20260709 12:15:29.829015 11322 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:15:29.829304 11251 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:15:29.802592 18344 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:15:29.834353 11929 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:15:29.837728 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 111383us
I20260709 12:15:29.837843 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.847640 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.848570 11928 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:29.846033 11254 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:29.826211 18516 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:29.855654 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.832880 18627 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:29.868443 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.875379 11934 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:29.869417 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:29.855859 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.831276 18653 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.876608 11251 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:29.877915 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.883162 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.883870 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:29.885289 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:29.886794 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2906) took 1286 ms. Trace:
I20260709 12:15:29.889541 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:29.871208 18532 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:29.876053 18516 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:29.901278 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:29.886415 18526 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.917842 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:29.908191 18328 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:29.970890 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:29.979588 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:30.023380 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:30.034543 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.880733 18545 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.047451 18265 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.048777 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.075605 11251 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
W20260709 12:15:30.050310 18757 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:15:29.891702 18614 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:29.926097 18526 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.092056 18662 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.099998 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.107467 18739 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:29.982223 18627 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.111819 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.112664 11255 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:29.558768 11331 rpcz_store.cc:276] 0709 12:15:28.260590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.878755 (+618165us) service_pool.cc:224] Handling call
0709 12:15:28.895533 (+ 16778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:28.895773 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:29.299692 (+403919us) log.cc:844] Serialized 14 byte log entry
0709 12:15:29.553773 (+254081us) log.cc:844] Serialized 14 byte log entry
0709 12:15:29.554098 (+   325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:29.554174 (+    76us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:29.554399 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.554502 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:29.554644 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":347729,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":87,"raft.queue_time_us":228,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":12930,"spinlock_wait_cycles":18987776,"thread_start_us":566845,"threads_started":2}
I20260709 12:15:30.123152 18642 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.124521 18735 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:30.125801 18749 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.383s	user 0.005s	sys 0.000s
I20260709 12:15:30.117516 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.133751 11331 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.135936 11255 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:30.141319 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.142264 11255 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.148828 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.152309 18526 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.158043 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.159013 11251 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:30.161715 11930 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:15:30.162235 11930 pending_rounds.cc:85] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 15
I20260709 12:15:30.162499 11930 pending_rounds.cc:107] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:15:30.162770 11930 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:30.164995 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.095717 18553 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.167315 18735 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.167145 18520 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.163837 18517 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:30.169309 18659 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.175696 18739 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.169590 18652 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.171327 18662 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.174532 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1374) took 1264 ms. Trace:
I20260709 12:15:30.185803 11254 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:15:30.187058 18739 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.187796 18659 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.188273 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:30.191620 11927 rpcz_store.cc:276] 0709 12:15:28.896973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.903653 (+  6680us) service_pool.cc:224] Handling call
0709 12:15:30.160890 (+1257237us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400650254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:30.161068 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317600640}
I20260709 12:15:30.199371 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:30.200515 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.176803 11331 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:30.204419 18652 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.205392 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.182641 18553 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.207929 18553 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:30.208508 18553 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:30.209909 18553 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.211351 18389 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.185446 18520 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.214030 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.214756 11251 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:30.202507 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.218526 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:30.219003 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.219308 11255 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:30.219215 11331 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:30.220942 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.221920 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.222113 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.222807 18389 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:30.223225 18389 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.223778 18389 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:30.223196 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.225217 11331 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:30.233289 11251 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:30.255640 11179 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:15:30.256922 11175 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:30.257579 11176 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:30.258622 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.257747 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:30.260643 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.261191 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.261441 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.261564 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.261394 11175 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:30.262944 11331 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:30.263470 11174 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:30.280085 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.252393 18735 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.294646 18698 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.294543 18735 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.296326 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2910) took 1154 ms. Trace:
W20260709 12:15:30.296891 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.418s	user 0.006s	sys 0.021s
I20260709 12:15:30.296815 11331 rpcz_store.cc:276] 0709 12:15:29.141951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.295542 (+1153591us) service_pool.cc:224] Handling call
0709 12:15:30.295856 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.296027 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.296051 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:30.296092 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:30.296125 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.296180 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.296271 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:30.300154 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.421s	user 0.006s	sys 0.022s
I20260709 12:15:30.245666 18659 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:30.265883 18653 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:30.288874 18516 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.263360 18328 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.291723 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1426) took 1307 ms. Trace:
I20260709 12:15:30.320559 18328 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.320987 11322 rpcz_store.cc:276] 0709 12:15:28.984567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.743832 (+759265us) service_pool.cc:224] Handling call
0709 12:15:29.744130 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.829552 (+ 85422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.291432 (+461880us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b54002f76d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:30.291654 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194046848}
I20260709 12:15:30.322463 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.323231 11322 raft_consensus.cc:2393] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 27: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.250453 18399 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:15:30.324488 11252 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:15:30.324611 18516 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:30.282339 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.330499 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.331635 11181 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.331933 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1423) took 1348 ms. Trace:
I20260709 12:15:30.332340 11324 rpcz_store.cc:276] 0709 12:15:28.983546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.733514 (+749968us) service_pool.cc:224] Handling call
0709 12:15:29.733889 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:29.734253 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:29.734284 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:29.741408 (+  7124us) log.cc:844] Serialized 26 byte log entry
0709 12:15:30.205519 (+464111us) spinlock_profiling.cc:243] Waited 6.54 ms on lock 0x7b18002a17e0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:30.249517 (+ 43998us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:30.249576 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.249709 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:30.331663 (+ 81954us) raft_consensus.cc:1685] finished
0709 12:15:30.331682 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.331874 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5823,"spinlock_wait_cycles":15049472,"thread_start_us":43765,"threads_started":1,"wal-append.queue_time_us":576300}
W20260709 12:15:30.302888 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 1.46536e+06us
I20260709 12:15:30.304239 18705 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.304169 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.304553 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.305097 18527 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:30.305944 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.307001 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.307160 18713 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:30.308243 11182 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:30.309198 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.302198 18735 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.387037 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.314869 18653 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:30.315709 18739 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:30.324927 18399 consensus_queue.cc:1243] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:15:30.292498 18389 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.324617 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3299) took 1153 ms. Trace:
I20260709 12:15:30.334231 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.334904 11174 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:30.335907 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:30.393932 11322 rpcz_store.cc:276] 0709 12:15:29.170980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.323835 (+1152855us) service_pool.cc:224] Handling call
0709 12:15:30.324154 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.324325 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.324347 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:30.324386 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:30.324414 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.324468 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.324568 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:30.394513 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 60374.1us
I20260709 12:15:30.395622 11322 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:15:30.396188 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2915) took 1140 ms. Trace:
I20260709 12:15:30.310984 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.387519 11324 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:30.388631 18660 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.309232 18585 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:30.392367 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55794.5us
I20260709 12:15:30.394647 11930 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:30.397277 11331 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:30.396498 11322 rpcz_store.cc:276] 0709 12:15:29.255584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.395011 (+1139427us) service_pool.cc:224] Handling call
0709 12:15:30.395318 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.395953 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.396136 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:30.387547 11175 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.400905 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1429) took 1221 ms. Trace:
I20260709 12:15:30.401563 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.401247 11324 rpcz_store.cc:276] 0709 12:15:29.179721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.334899 (+1155178us) service_pool.cc:224] Handling call
0709 12:15:30.385834 (+ 50935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.400718 (+ 14884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.400863 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:30.402997 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2911) took 1236 ms. Trace:
I20260709 12:15:30.338933 18527 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:30.403393 11329 rpcz_store.cc:276] 0709 12:15:29.166587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.331841 (+165254us) service_pool.cc:224] Handling call
0709 12:15:30.402829 (+1070988us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54003318d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:30.402942 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358698368}
I20260709 12:15:30.328691 18659 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.334805 18715 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:15:30.408572 18725 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.408852 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.410442 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.410584 18389 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.411228 18691 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:30.412753 18691 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:30.412869 18653 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.406643 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.415107 18545 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:30.415509 18545 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.415870 18545 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.398378 18660 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.405674 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.413470 18691 raft_consensus.cc:2764] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 12:15:30.417827 18735 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:30.419757 18660 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.422032 11930 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:30.426633 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1432) took 1150 ms. Trace:
I20260709 12:15:30.427393 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3039) took 4957 ms. Trace:
I20260709 12:15:30.398838 18739 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:30.423815 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.427830 11249 rpcz_store.cc:276] 0709 12:15:25.469739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.470122 (+   383us) service_pool.cc:224] Handling call
0709 12:15:28.802293 (+3332171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:28.819513 (+ 17220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.427169 (+1607656us) spinlock_profiling.cc:243] Waited 3.31 s on lock 0x7b5400108e50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:30.427303 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7604819200}
I20260709 12:15:30.427080 11331 rpcz_store.cc:276] 0709 12:15:29.275797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.406201 (+1130404us) service_pool.cc:224] Handling call
0709 12:15:30.426148 (+ 19947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.426337 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.426362 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:30.426400 (+    38us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:30.426429 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.426483 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.426581 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:30.426191 18527 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.433053 11179 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:15:30.432984 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.435688 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.436170 18659 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.437175 18527 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.431773 18725 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:30.427083 18630 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:30.436367 11331 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:30.443011 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.443295 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.445667 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.448925 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:30.463507 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:30.464005 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.470815 11331 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:30.472196 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.473235 11331 raft_consensus.cc:2393] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:15:30.496847 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.499367 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.501808 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.503906 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.509459 11177 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:30.510017 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.512171 11176 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.517498 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.520431 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.530093 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:30.531459 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55945.5us
I20260709 12:15:30.530635 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.529266 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.529407 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.535599 11181 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.535971 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.532407 11179 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:30.536386 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.539664 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.540122 11181 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.530257 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.540711 11934 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:30.542294 11254 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.470227 18725 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:30.545243 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.546083 11179 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.546823 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.554775 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.556071 11173 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:15:30.555712 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.548635 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.559674 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.561429 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.561769 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.563498 11174 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.564203 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:30.565421 18590 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:30.565888 18590 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:30.566234 18590 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:30.568794 18725 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:30.570423 18773 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:15:30.570998 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.559329 18627 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:15:30.559139 18585 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 12:15:30.561368 18526 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:15:30.574810 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.575084 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.576212 18582 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:30.576663 18582 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:30.577056 18582 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:30.578428 18771 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:15:30.585549 11180 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:30.544678 18775 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:30.599659 11181 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:30.586798 18739 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.612933 18739 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.615429 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.616209 18739 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:30.623279 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 83102.4us
W20260709 12:15:30.623530 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3073) took 1298 ms (client timeout 1570 ms). Trace:
W20260709 12:15:30.624028 11253 rpcz_store.cc:269] 0709 12:15:29.306983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.308205 (+  1222us) service_pool.cc:224] Handling call
0709 12:15:30.605449 (+1297244us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001dd3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:30.605602 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277570432}
I20260709 12:15:30.628930 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.657316 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:30.630851 11296 connection.cc:256] RPC call timeout handler was delayed by -1.7513882220000596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 12:15:30.658371 11296 connection.cc:256] RPC call timeout handler was delayed by -1.5677882219999901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
I20260709 12:15:30.635526 18582 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:30.668637 18752 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.928s	user 0.002s	sys 0.000s
I20260709 12:15:30.636111 18545 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.630616 18739 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:30.676826 11175 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:29.887315 11327 rpcz_store.cc:276] 0709 12:15:28.600523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.722718 (+1122195us) service_pool.cc:224] Handling call
0709 12:15:29.722988 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.723183 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:29.802007 (+ 78824us) log.cc:844] Serialized 14 byte log entry
0709 12:15:29.886457 (+ 84450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:29.886514 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:29.886564 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.886631 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:29.886742 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":170912,"raft.queue_time_us":156,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":2687,"thread_start_us":135255,"threads_started":2}
I20260709 12:15:30.689239 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:30.631242 18470 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.032s	sys 0.022s
I20260709 12:15:30.630467 18726 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:15:30.597967 18775 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.641901 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.694636 11180 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:30.694784 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.641969 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:29.643911 11325 rpcz_store.cc:276] 0709 12:15:28.305286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.418542 (+1113256us) service_pool.cc:224] Handling call
0709 12:15:29.418887 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.419109 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:29.419456 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 12:15:29.624455 (+204999us) log.cc:844] Serialized 14 byte log entry
0709 12:15:29.632282 (+  7827us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:29.632349 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:29.632398 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.632467 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:29.632588 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4708,"prepare.queue_time_us":6578,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":8575,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":6400,"spinlock_wait_cycles":5538688,"thread_start_us":203782,"threads_started":1}
I20260709 12:15:30.643776 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:30.644675 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 786851d4227e4dd89b75ccdea0b1f03b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:30.645972 18526 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.646631 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.648737 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3042) took 5177 ms. Trace:
I20260709 12:15:30.645406 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.652220 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.697197 11178 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
W20260709 12:15:30.697122 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51804.2us
I20260709 12:15:30.650523 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:30.658929 11296 connection.cc:256] RPC call timeout handler was delayed by -1.5677882219999901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
I20260709 12:15:30.698155 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:30.698596 11296 connection.cc:256] RPC call timeout handler was delayed by -1.3913882219999323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
W20260709 12:15:30.698967 11296 connection.cc:256] RPC call timeout handler was delayed by -1.2428882219999196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
I20260709 12:15:30.699515 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.700565 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.698259 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.696411 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.702304 11180 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.702696 18545 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.703095 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.703318 18690 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:30.703253 18526 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.704156 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.673791 18739 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.696838 11256 rpcz_store.cc:276] 0709 12:15:25.471524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.845772 (+374248us) service_pool.cc:224] Handling call
0709 12:15:25.846101 (+   329us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:15:25.846513 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:27.367754 (+1521241us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b1800265f00. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf 00007f3ec6706038
0709 12:15:27.411769 (+ 44015us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.794968 (+383199us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:15:29.300213 (+1505245us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b2c00000e98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:29.319846 (+ 19633us) log.cc:844] Serialized 78 byte log entry
0709 12:15:29.612727 (+292881us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:29.613019 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 12:15:29.616706 (+  3687us) log.cc:844] Serialized 14 byte log entry
0709 12:15:29.616920 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.617016 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:29.789728 (+172712us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b1800265f00. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:30.167396 (+377668us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b1800265f00. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f
0709 12:15:30.438347 (+270951us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b4800042b58. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:15:30.635886 (+197539us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b1800265f00. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:30.648501 (+ 12615us) raft_consensus.cc:1685] finished
0709 12:15:30.648529 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.648677 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9175,"prepare.queue_time_us":172924,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":130,"raft.queue_time_us":3853,"raft.run_cpu_time_us":32092,"raft.run_wall_time_us":834950,"spinlock_wait_cycles":64664576,"thread_start_us":643277,"threads_started":3,"wal-append.queue_time_us":649766,"wal-append.run_cpu_time_us":2014,"wal-append.run_wall_time_us":1294848}
I20260709 12:15:30.706090 18526 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:30.707170 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:30.692114 18470 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.032s	sys 0.022s
I20260709 12:15:30.707798 18739 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.609282 18590 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.660284 11173 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:30.708925 18660 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:30.709616 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.710202 18590 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:30.710757 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.711755 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.712891 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.687671 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.713901 18590 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:30.714010 11327 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:30.714358 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.715056 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.715166 18590 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:30.715327 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64847.1us
I20260709 12:15:30.716079 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.716383 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.716907 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.716998 11325 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
W20260709 12:15:30.695833 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51225.6us
I20260709 12:15:30.697983 11329 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.717798 18726 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:30.717938 18399 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:30.718231 18726 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.718468 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.718678 18726 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:30.695811 11179 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:30.719587 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.720202 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.720700 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.718493 18399 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.722081 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.722612 11327 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.721304 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.708549 18739 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:30.711790 18775 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:30.724025 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.724148 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.707767 11178 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:30.729761 11900 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:30.721761 18399 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:30.734931 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.735639 11930 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:30.726229 18520 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:30.726763 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.726976 18642 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:30.741567 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.727229 18590 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:30.723577 11179 raft_consensus.cc:2376] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20. Re-sending same reply.
I20260709 12:15:30.729872 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.725028 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.734148 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.658687 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.749156 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.750056 18590 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:30.750288 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.750500 18590 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:30.751441 18590 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.752665 18590 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:30.753314 11178 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:15:30.746284 18527 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:15:30.739331 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:30.686131 18582 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.725812 18664 raft_consensus.cc:2764] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:15:30.727897 18660 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.742379 11180 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:30.742645 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.746616 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.754671 11178 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:30.751463 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.765532 11181 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:30.766099 11173 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:30.766355 11178 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:30.766719 11181 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:30.766839 11178 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:30.762879 11327 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:30.738445 18709 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:30.741560 18526 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:30.771046 18526 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:30.771457 18526 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.768710 11325 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:30.774516 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.775669 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.776333 11325 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 19: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.777081 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.763521 18698 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:30.781738 18709 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.782339 18709 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.784260 11325 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:30.728405 18627 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:30.767817 11178 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.769340 18691 raft_consensus.cc:2764] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:15:30.786552 18627 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:30.788367 18627 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:30.793278 18730 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.793711 18660 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.794005 18730 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.795717 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.796357 18660 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:30.797053 18660 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:30.797551 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:30.754709 18527 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:30.799513 18527 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:30.742306 18665 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:15:30.785296 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.746419 18628 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:30.769682 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:30.801497 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.803849 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1421) took 2303 ms. Trace:
I20260709 12:15:30.805160 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.805455 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.827893 11324 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:30.812774 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:30.829101 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.829910 11324 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.817381 11180 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:30.830804 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.814994 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.831915 11255 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:15:30.832016 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.832726 11324 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:30.814352 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.804352 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.833952 11249 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:30.802327 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.832401 11323 rpcz_store.cc:276] 0709 12:15:28.500907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.711460 (+1210553us) service_pool.cc:224] Handling call
0709 12:15:29.720268 (+  8808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.720779 (+   511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.797623 (+1076844us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b54001c52d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:30.803781 (+  6158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19605760}
I20260709 12:15:30.835214 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.789047 18726 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.836951 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:30.802943 11325 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:30.833854 11256 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:30.843679 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.844664 11253 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:30.829670 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:30.846910 11323 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:30.846504 11327 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:30.845052 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:30.848832 11323 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:15:30.755053 18643 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.854328 11323 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:30.854707 11255 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:30.855813 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.856271 11323 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:30.860100 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.862062 11324 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:15:30.863863 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.864300 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.865309 11251 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:30.801681 18775 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:30.866250 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.869401 11255 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:30.867293 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:30.802577 18715 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:30.868779 11257 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:30.876314 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.880914 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:30.800849 18328 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:30.864243 18389 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:15:30.867139 18614 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:15:30.883882 11933 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:15:30.884140 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.884650 18735 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:30.886566 18630 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:30.891041 11253 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:30.918838 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.920009 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.920603 11253 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:30.864939 18540 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:30.891325 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1367) took 2063 ms. Trace:
I20260709 12:15:30.868368 18643 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:30.741446 18725 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.123s
I20260709 12:15:30.898980 11249 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:30.900693 11932 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:30.853876 18585 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:30.903493 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:30.903703 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:30.927263 11255 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:30.944729 11322 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:15:30.945853 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.946125 11322 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:30.946655 11255 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:30.947425 11322 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:30.948181 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.948648 11322 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:15:30.850492 18690 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:30.913070 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.955713 11248 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:30.852818 18665 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:30.921923 11935 rpcz_store.cc:276] 0709 12:15:28.828365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.828968 (+   603us) service_pool.cc:224] Handling call
0709 12:15:30.891111 (+2062143us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b5400650ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:30.891258 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579258752}
W20260709 12:15:30.956341 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 52946.5us
I20260709 12:15:30.802503 18705 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.095s
I20260709 12:15:30.922998 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 2907) took 2186 ms. Trace:
I20260709 12:15:30.921583 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.957691 11326 rpcz_store.cc:276] 0709 12:15:28.737049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.840996 (+103947us) service_pool.cc:224] Handling call
0709 12:15:30.922799 (+2081803us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b540026c5d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:30.922932 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026113920}
I20260709 12:15:30.958105 11253 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:30.923854 11330 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:30.930173 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.960287 11252 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
W20260709 12:15:30.932687 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.934373 11180 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 12:15:30.800658 18527 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.934648 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.957141 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:15:30.936017 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.965461 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.853403 18623 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 12:15:30.879180 18715 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:30.951004 11175 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:30.951928 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:30.975719 11328 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:15:30.976097 11179 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.952617 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.977046 11249 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 17.
I20260709 12:15:30.977957 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.979010 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.979698 11249 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:30.954267 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.893951 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:30.966130 11257 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:30.982468 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:30.968124 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.983108 11255 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:30.976711 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 124941us
I20260709 12:15:30.982491 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:30.971905 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.972152 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.989490 11176 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:30.972944 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:30.800375 18627 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:30.991650 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:30.992856 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:30.942881 18540 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:30.998649 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:31.008945 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.985728 18715 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:31.026486 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:31.027920 11175 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:30.994422 18527 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:31.001685 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:30.955375 18690 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:30.995440 11176 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:30.994549 11322 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:15:31.003381 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46325.2us
I20260709 12:15:31.006294 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:31.037218 11248 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 18: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:31.037775 18690 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:31.038205 11248 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:31.038600 11248 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:31.039328 11248 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.041112 11248 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:30.965684 18665 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:30.994181 18516 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:31.035702 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:31.000732 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.006754 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:31.051949 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:31.054674 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:31.007431 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
W20260709 12:15:31.068629 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.085140 11175 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:31.087504 11931 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:15:31.059147 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92521.6us
I20260709 12:15:31.031939 18785 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:15:31.031994 18705 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:15:31.094602 11255 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
W20260709 12:15:31.109372 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 108678us
I20260709 12:15:31.032997 18689 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:31.116106 11257 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:31.116372 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:31.120764 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.031038 18389 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.077s
I20260709 12:15:31.121765 11255 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:31.056442 18516 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:31.132521 11248 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:15:31.140311 11248 pending_rounds.cc:85] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 15
I20260709 12:15:31.141685 11248 pending_rounds.cc:107] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:15:31.142095 11248 raft_consensus.cc:2889] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:31.089316 18775 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:31.042532 18470 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 12:15:31.133411 11257 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:15:31.143776 11249 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:15:31.147202 11249 pending_rounds.cc:85] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 13
I20260709 12:15:31.147545 11249 pending_rounds.cc:107] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:15:31.147897 11249 raft_consensus.cc:2889] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:31.149461 11248 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 12:15:31.115322 18585 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:31.119545 18623 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:31.117509 18791 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:31.137476 18643 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:31.113845 18628 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:15:31.151575 18689 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:31.165753 11254 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:15:31.144301 18788 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.180502 11254 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:31.142645 18792 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:15:31.182411 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 130522us
I20260709 12:15:31.187294 18627 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:31.138794 18540 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.143638 18630 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
W20260709 12:15:31.194789 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.148237 18775 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.159875 18643 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:31.212913 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82105.9us
I20260709 12:15:31.184326 18792 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.163053 18399 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.217372 11181 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:31.215193 18643 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:31.181648 18791 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:15:31.275080 18775 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
W20260709 12:15:31.277961 18799 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:15:31.279760 18709 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.281267 11250 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:31.208288 18389 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:15:31.187688 18788 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:31.261621 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 251800us
I20260709 12:15:31.218852 18792 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:31.219211 18585 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.126839 18793 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 12:15:31.273133 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:31.285276 18526 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.010s
I20260709 12:15:31.268643 11932 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:15:31.283980 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3296) took 2158 ms. Trace:
I20260709 12:15:31.285913 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2926) took 1437 ms. Trace:
I20260709 12:15:31.327123 18570 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:15:31.326707 11330 rpcz_store.cc:276] 0709 12:15:29.125795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.760417 (+634622us) service_pool.cc:224] Handling call
0709 12:15:30.923641 (+1163224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.959406 (+ 35765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.283676 (+324270us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540026c5d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:31.283878 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2674439040}
I20260709 12:15:31.327328 11325 rpcz_store.cc:276] 0709 12:15:29.848525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.838915 (+990390us) service_pool.cc:224] Handling call
0709 12:15:30.864562 (+ 25647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:30.864869 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.864913 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:30.905552 (+ 40639us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.139457 (+233905us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:31.139509 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.139597 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:31.285565 (+145968us) raft_consensus.cc:1685] finished
0709 12:15:31.285596 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.285837 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57360,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":2947,"thread_start_us":171932,"threads_started":1,"wal-append.queue_time_us":274319}
I20260709 12:15:31.335943 11325 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:31.299865 11256 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:31.303304 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.306648 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:31.342281 11254 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:31.342486 18715 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.282521 18665 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:31.325264 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:31.345664 11182 raft_consensus.cc:606] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.346026 11182 raft_consensus.cc:614] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:31.346441 18627 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:31.349014 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2936) took 1025 ms. Trace:
I20260709 12:15:31.349741 18805 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:31.347414 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.356068 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.349383 11327 rpcz_store.cc:276] 0709 12:15:30.323385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.852612 (+529227us) service_pool.cc:224] Handling call
0709 12:15:30.852878 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:31.118571 (+265693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.118605 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:31.118651 (+    46us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:31.118689 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.348796 (+230107us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540042b8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:31.348843 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.348961 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610461440}
I20260709 12:15:31.266973 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.335987 18540 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:15:31.288029 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:31.324482 18389 consensus_queue.cc:1243] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:15:31.344729 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:31.355465 18590 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.000s
I20260709 12:15:31.362732 11322 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:31.364986 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 98117.5us
I20260709 12:15:31.378336 11927 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 19 because replica has last-logged OpId of term: 18 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 16.
I20260709 12:15:31.378706 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:31.366746 18715 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:31.367120 18389 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:31.380308 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:31.380640 18389 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:31.380945 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:31.381078 18389 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:31.383435 18690 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.384688 18516 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.366776 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.386199 18709 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:31.363281 18653 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:31.386528 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.386755 11248 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:31.357977 18805 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:31.388283 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:31.373209 18623 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.389926 18623 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:31.389919 18715 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:31.391134 18399 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:31.391503 18762 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:31.391935 18762 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:31.390074 18590 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:31.332160 18526 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.393106 18521 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:31.392810 18762 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.395568 18520 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:31.396036 18520 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:31.396363 18762 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:31.392920 18664 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:31.386318 11328 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:31.400254 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:31.389477 18527 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:31.401859 18516 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:31.393503 18521 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:31.373440 18788 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:31.402815 11929 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:31.403232 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:31.404119 11328 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:31.405236 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:31.392824 18590 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:31.394874 18526 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:31.406152 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:31.407279 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:31.396939 18520 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.371614 18730 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:31.407716 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:31.417972 11322 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:31.408751 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:31.419255 11327 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:31.411618 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.412608 11254 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:31.361456 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:31.420645 11250 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:31.420608 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59238.2us
I20260709 12:15:31.422062 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.428731 11181 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:15:31.432047 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:31.432906 11181 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:31.389847 18690 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:31.435137 11327 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:31.438628 11327 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:15:31.437006 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:31.442649 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.451009 11254 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:31.452404 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43730.5us
I20260709 12:15:31.453088 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:31.457530 11177 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:31.450201 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:31.451881 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:31.460601 11181 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:31.465245 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:31.468132 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:31.469590 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:31.433847 18650 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:31.402824 18643 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.457892 18659 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:31.425173 18705 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:15:31.424643 18630 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:15:31.425271 18527 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:15:31.424609 18709 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:15:31.487674 18659 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:31.488665 18659 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.452030 18793 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:31.469861 18555 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:31.465906 18399 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:31.470912 18730 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:31.455175 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.500577 18585 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:31.472116 18800 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:31.500700 18627 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:31.508560 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53502us
I20260709 12:15:31.500387 11177 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:31.472529 18650 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.458288 18626 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
W20260709 12:15:31.511938 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46781.8us
I20260709 12:15:31.486624 18698 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:31.461583 18778 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:15:31.472029 18785 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:15:31.514074 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.527336 18642 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:31.527875 18642 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:31.528836 18642 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.493149 18792 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.472856 18789 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:15:31.447602 18795 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 12:15:31.537616 18609 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:15:31.541710 18792 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:31.505864 18791 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:31.506145 18730 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.512599 18650 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:31.516532 18809 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:15:31.562258 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.585217 18809 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:31.586382 18809 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.560817 18730 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:31.520922 18800 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:31.517539 18709 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:15:31.592656 18609 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:15:31.593099 18808 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:15:31.508718 18623 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.553293 18698 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:31.599360 18643 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:31.533995 18590 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.609850 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1441) took 1209 ms. Trace:
I20260709 12:15:31.612511 11323 rpcz_store.cc:276] 0709 12:15:30.399970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.857491 (+457521us) service_pool.cc:224] Handling call
0709 12:15:30.857772 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.857999 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:15:30.858327 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 12:15:31.609207 (+750880us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b18003c2b60. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287
0709 12:15:31.609344 (+   137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:31.609412 (+    68us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:31.609475 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.609570 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.609770 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":10799,"raft.queue_time_us":79223,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6947,"spinlock_wait_cycles":43190528,"thread_start_us":102352,"threads_started":1}
I20260709 12:15:31.615150 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:31.618022 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2937) took 1198 ms. Trace:
I20260709 12:15:31.618566 11324 rpcz_store.cc:276] 0709 12:15:30.419953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.868534 (+448581us) service_pool.cc:224] Handling call
0709 12:15:30.868856 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.270437 (+401581us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b18003d3ba0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66e4bfb 00007f3ec66ef64b
0709 12:15:31.270586 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.270616 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.272189 (+  1573us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.325781 (+ 53592us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:31.325836 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.325941 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:31.601413 (+275472us) raft_consensus.cc:1685] finished
0709 12:15:31.601442 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.617933 (+ 16491us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74335,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":12342,"spinlock_wait_cycles":2690432,"thread_start_us":144493,"threads_started":2,"wal-append.queue_time_us":178993}
I20260709 12:15:31.624891 11324 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:31.595485 18609 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:31.645026 11323 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:31.652797 11323 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:31.654798 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 140782us
W20260709 12:15:31.640084 18815 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:15:31.642916 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 105805us
I20260709 12:15:31.659577 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:31.660182 11256 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:31.663735 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:31.666654 11182 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:31.672010 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 192273us
I20260709 12:15:31.701540 11181 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:15:31.701543 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:31.703012 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.704025 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:31.705418 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:31.732457 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:31.732653 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51346.1us
I20260709 12:15:31.736557 18691 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.668685 18705 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:31.747782 11252 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:31.742658 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.750810 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:31.744681 11182 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:31.651484 18755 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
W20260709 12:15:31.753454 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 96805.8us
I20260709 12:15:31.766939 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.664746 18643 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 12:15:31.790352 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47791.9us
I20260709 12:15:31.791267 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.773682 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:31.792281 11248 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:31.795202 18399 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:31.731444 18775 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.820469 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:31.833750 18713 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:31.834401 18713 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:31.835598 18713 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.837208 18713 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:31.844242 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.845911 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2923) took 2151 ms. Trace:
I20260709 12:15:31.846436 11331 rpcz_store.cc:276] 0709 12:15:29.694434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.533604 (+839170us) service_pool.cc:224] Handling call
0709 12:15:30.533895 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.713615 (+179720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.713667 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:30.713709 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:30.713744 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.845555 (+1131811us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400331dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:31.845624 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.845838 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411535744}
I20260709 12:15:31.847959 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:31.848649 11331 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:31.749727 18516 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:15:31.865253 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44896.3us
W20260709 12:15:31.886650 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 119833us
I20260709 12:15:31.727254 18555 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 12:15:31.810935 18705 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:31.894198 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.894750 11331 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:31.899801 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:31.890475 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.900225 11901 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:31.903919 18821 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:15:31.905153 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48465.7us
I20260709 12:15:31.737239 18691 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.915545 18801 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:31.926360 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:31.917919 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:31.679116 18698 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:31.901366 18703 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:31.922632 11180 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:31.949009 11180 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:31.773159 18526 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.136s
I20260709 12:15:31.889099 18399 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:31.931749 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.893971 18775 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:31.935283 18389 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.000s
W20260709 12:15:31.939383 18820 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.205s	user 0.000s	sys 0.002s
I20260709 12:15:31.677522 18800 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.707953 18609 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.951792 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:31.961372 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.945043 18816 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:31.974387 11256 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
W20260709 12:15:31.983161 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:31.994902 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63241us
I20260709 12:15:31.930375 18801 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.930008 18665 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.949816 11180 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:31.930984 18794 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:32.012089 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:32.016716 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65053.3us
I20260709 12:15:32.016969 11180 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:32.021020 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:32.022102 11256 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:31.993937 18628 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.930588 18793 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.079s
I20260709 12:15:32.017455 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:32.029762 18816 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:31.997604 18791 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:32.034191 18726 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:32.035393 18705 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.029258 11331 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:32.017457 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:31.995256 18792 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:32.071347 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53955.2us
I20260709 12:15:32.072675 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:32.046274 11330 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:32.019728 18623 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:32.062027 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44650.7us
I20260709 12:15:32.084184 11256 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:32.010025 18755 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:32.057325 11180 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:15:32.019807 18801 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:32.086102 11180 pending_rounds.cc:85] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 15
I20260709 12:15:32.096101 11180 pending_rounds.cc:107] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:15:32.040551 18399 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:32.093657 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:32.063771 18585 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:32.096972 11180 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:32.005311 18389 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.108144 18399 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:32.092820 18792 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.109615 18399 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.110517 18792 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:32.098475 18709 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:32.113389 18520 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:32.114099 18804 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:32.071144 18614 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:31.998507 18521 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:32.093022 18265 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:32.040252 18665 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:32.100253 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:32.097607 18659 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:32.114547 18804 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:32.220271 18804 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.061076 18628 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.101814 18793 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:32.222146 18793 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:32.114976 18791 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:32.225477 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 125299us
I20260709 12:15:32.226408 11222 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:32.231518 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:32.251968 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:32.225021 18265 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:32.238597 18794 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:32.223402 18793 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 16, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.264271 18775 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:32.229519 18614 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:32.268306 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2700) took 1336 ms. Trace:
I20260709 12:15:32.240247 18709 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:32.299224 18739 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.299940 18788 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.257932 18816 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.302806 18389 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:32.303267 18691 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:32.303565 18715 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.913s	user 0.012s	sys 0.000s
I20260709 12:15:32.440263 18715 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:32.450976 18837 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 12:15:32.453125 18829 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.381s	user 0.000s	sys 0.001s
I20260709 12:15:32.454802 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1442) took 2028 ms. Trace:
W20260709 12:15:32.455354 18834 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
W20260709 12:15:32.359587 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 107712us
I20260709 12:15:32.232446 18555 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:15:32.416062 11174 rpcz_store.cc:276] 0709 12:15:30.931914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.940395 (+  8481us) service_pool.cc:224] Handling call
0709 12:15:32.268002 (+1327607us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54001f45d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:32.268229 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773202176}
W20260709 12:15:32.388458 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.258060 18791 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:15:32.427037 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.435290 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3317) took 1573 ms. Trace:
I20260709 12:15:32.455242 11329 rpcz_store.cc:276] 0709 12:15:30.426127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.954518 (+528391us) service_pool.cc:224] Handling call
0709 12:15:30.954838 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:30.967674 (+ 12836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.967715 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.675314 (+707599us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b2c00126d18. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:31.675752 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.858902 (+183150us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:31.858944 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.859018 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:32.454474 (+595456us) raft_consensus.cc:1685] finished
0709 12:15:32.454495 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:32.454728 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18659,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":10406,"spinlock_wait_cycles":44226816,"thread_start_us":182784,"threads_started":1,"wal-append.queue_time_us":328925}
I20260709 12:15:32.458850 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:32.369679 18703 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:32.465940 11179 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:32.305974 18628 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:32.418283 18626 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.358124 18659 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:32.470042 11179 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:32.375660 18794 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:32.472352 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.023s	sys 0.047s
W20260709 12:15:32.502496 11327 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.047s	sys 0.078s
W20260709 12:15:32.504653 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:32.539305 18730 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.945s	user 0.008s	sys 0.004s
W20260709 12:15:32.534502 18800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.018s	sys 0.020s
I20260709 12:15:32.528378 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:32.432193 18792 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.097s
I20260709 12:15:32.471199 18689 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:32.431916 18739 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:32.471571 18709 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 12:15:32.492400 11179 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.462374 18791 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:32.436206 18788 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:32.490931 18793 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:32.534658 18794 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:32.620532 11327 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.047s	sys 0.078s
W20260709 12:15:32.554514 18826 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:15:32.375306 18585 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.620632 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:32.622236 18800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.024s	sys 0.040s
I20260709 12:15:32.624755 18730 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:32.620474 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.023s	sys 0.055s
W20260709 12:15:32.648353 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 694420us
W20260709 12:15:32.651301 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.503381 18626 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:32.654050 11325 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.001s	sys 0.021s
I20260709 12:15:32.623409 18845 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:32.624778 18399 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.009s	sys 0.014s
W20260709 12:15:32.624703 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.003s	sys 0.014s
I20260709 12:15:32.637581 11179 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:32.658440 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.003s	sys 0.014s
W20260709 12:15:32.653360 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 266152us
I20260709 12:15:32.635263 18791 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:32.654523 11325 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.002s	sys 0.021s
W20260709 12:15:32.625948 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.012s	sys 0.012s
W20260709 12:15:32.665323 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.013s	sys 0.012s
W20260709 12:15:32.665866 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 580407us
I20260709 12:15:32.623819 18689 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:32.667557 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:32.614677 18265 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:32.659982 18399 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.009s	sys 0.014s
I20260709 12:15:32.683463 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:32.668159 18653 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 12:15:32.693846 18399 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:15:32.694223 18835 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:15:32.696470 18831 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:15:32.663782 18665 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:32.698890 18788 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:32.675328 18689 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:32.689108 18847 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:32.703251 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:32.703912 18855 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:32.704593 18789 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:32.772087 18789 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:32.772872 18789 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:32.745436 18775 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.625705 18614 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:32.775573 18626 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:32.775856 18739 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:32.717733 18866 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:32.776868 18866 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.777539 18730 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:32.779790 18866 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:32.717782 18713 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.838s	user 0.010s	sys 0.003s
W20260709 12:15:32.719174 18825 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.049s	user 0.000s	sys 0.013s
I20260709 12:15:32.721462 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:32.705580 18652 raft_consensus.cc:2764] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 12:15:32.754081 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50922.7us
I20260709 12:15:32.663574 18845 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.655282 18659 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.754981 18862 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:32.754630 18856 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:32.759734 18735 raft_consensus.cc:2764] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:15:32.770686 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1166) took 1111 ms. Trace:
I20260709 12:15:32.816530 11253 rpcz_store.cc:276] 0709 12:15:31.659129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.659264 (+   135us) service_pool.cc:224] Handling call
0709 12:15:31.659629 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.659963 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.659989 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.667735 (+  7746us) log.cc:844] Serialized 26 byte log entry
0709 12:15:32.063233 (+395498us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:32.063284 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:32.063655 (+   371us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:32.770411 (+706756us) raft_consensus.cc:1685] finished
0709 12:15:32.770435 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:32.770605 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51539,"raft.queue_time_us":436,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12309,"spinlock_wait_cycles":37888,"thread_start_us":293035,"threads_started":1,"wal-append.queue_time_us":739428}
I20260709 12:15:32.722872 18399 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:32.691320 18858 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:32.773999 18775 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.773932 18789 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.775049 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2956) took 1281 ms. Trace:
I20260709 12:15:32.749670 18630 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:32.795372 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74030.7us
I20260709 12:15:32.844564 11324 rpcz_store.cc:276] 0709 12:15:31.493829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.635081 (+141252us) service_pool.cc:224] Handling call
0709 12:15:31.635521 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.635969 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.636010 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.641147 (+  5137us) log.cc:844] Serialized 26 byte log entry
0709 12:15:32.007499 (+366352us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:32.007542 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:32.007607 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:32.774823 (+767216us) raft_consensus.cc:1685] finished
0709 12:15:32.774844 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:32.774983 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":641,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":4802,"thread_start_us":327991,"threads_started":1,"wal-append.queue_time_us":715007}
I20260709 12:15:32.849764 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2948) took 1438 ms. Trace:
I20260709 12:15:32.792868 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:32.850342 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:32.858232 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 76226.9us
I20260709 12:15:32.859293 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:32.850234 11328 rpcz_store.cc:276] 0709 12:15:31.410974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.422886 (+ 11912us) service_pool.cc:224] Handling call
0709 12:15:31.423170 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.423510 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.423538 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.424010 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.588626 (+164616us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b18003d8e80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:31.671755 (+ 83129us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:31.671804 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.671871 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:32.299413 (+627542us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:15:32.718548 (+419135us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:32.849488 (+130940us) raft_consensus.cc:1685] finished
0709 12:15:32.849515 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:32.849695 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":1836,"raft.run_wall_time_us":874634,"spinlock_wait_cycles":317555712,"thread_start_us":82953,"threads_started":1,"wal-append.queue_time_us":246145}
I20260709 12:15:32.823590 18739 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:32.862429 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:32.866535 11330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.015s	sys 0.000s
W20260709 12:15:32.870425 11330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.015s	sys 0.001s
I20260709 12:15:32.873797 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:32.876928 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:32.877947 18788 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:32.879129 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:32.879586 18788 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:32.880043 18788 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:32.880434 18788 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:32.886310 11927 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:32.887138 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:32.888545 11252 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: [24.12-24.12]
I20260709 12:15:32.889207 11930 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:32.890304 11250 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:32.890709 11927 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:32.891786 11250 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
W20260709 12:15:32.875682 18650 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.295s	user 0.014s	sys 0.000s
I20260709 12:15:32.895148 11330 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:15:32.925702 11252 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:32.925550 11248 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:32.895880 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:32.881851 18850 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:32.928548 11931 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:15:32.928954 11254 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:15:32.928970 11931 pending_rounds.cc:85] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 13
I20260709 12:15:32.930236 11931 pending_rounds.cc:107] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:15:32.930461 11253 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:32.930472 11931 raft_consensus.cc:2889] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:32.932824 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 465373us
I20260709 12:15:32.875252 18793 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:32.883114 18789 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:32.884994 18652 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:32.936005 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:32.883220 18851 raft_consensus.cc:2764] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:15:32.891000 18832 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:32.945355 11178 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:32.860877 18630 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.792312 18847 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.894663 18840 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:32.881901 18665 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:15:32.948467 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:32.932932 18856 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:32.934458 18857 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:32.842126 18862 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.943431 18846 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:32.851867 18858 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:32.946676 18739 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:33.087033 11178 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:32.949119 18850 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:33.080227 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:32.966470 18614 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.933746 18650 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.121428 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1231) took 2457 ms. Trace:
W20260709 12:15:33.154960 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74825.7us
I20260709 12:15:33.185586 11178 rpcz_store.cc:276] 0709 12:15:30.663919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.765962 (+102043us) service_pool.cc:224] Handling call
0709 12:15:30.766231 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:32.944764 (+2178533us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5800809cd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66480aa 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66eeb49 00007f3ec66ef416 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4
0709 12:15:33.120513 (+175749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.121356 (+   843us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1389122816}
I20260709 12:15:33.100504 18665 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:32.932003 18399 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.187134 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:32.966849 18691 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.158s
I20260709 12:15:33.098856 18856 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:33.114595 18840 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:33.120971 18809 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.271227 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 322863us
W20260709 12:15:33.246070 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59103.4us
I20260709 12:15:33.272382 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:33.091253 18689 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:33.241559 18867 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 12:15:33.115103 18832 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:33.276566 11182 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:33.149364 18866 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.242919 18794 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.003s
I20260709 12:15:33.185225 18851 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.156121 18713 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.243520 18520 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.130s	user 0.011s	sys 0.002s
W20260709 12:15:33.281733 18791 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.000s
W20260709 12:15:33.271260 18801 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.128s	user 0.009s	sys 0.003s
I20260709 12:15:33.238300 18856 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:33.126336 18846 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.188822 18665 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:33.113982 18857 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.290671 18775 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.194823 18614 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:33.306023 18879 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:15:33.306515 18873 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 12:15:33.335112 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62860.2us
I20260709 12:15:33.342123 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:33.293776 18709 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.319515 18876 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:15:33.311075 18890 raft_consensus.cc:2764] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election decision vote started in defunct term 24: won
I20260709 12:15:33.282078 18809 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:33.353178 18871 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:15:33.290985 18794 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.289144 18866 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.371932 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2961) took 1760 ms. Trace:
W20260709 12:15:33.345364 18878 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 12:15:33.360215 18877 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:15:33.360042 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1451) took 1516 ms. Trace:
I20260709 12:15:33.293430 18851 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.348697 18840 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:33.362823 18689 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.363765 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2960) took 1773 ms. Trace:
I20260709 12:15:33.367638 18614 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.297871 18801 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:33.370436 18869 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 12:15:33.371059 18691 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:33.372370 11331 rpcz_store.cc:276] 0709 12:15:31.611308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.932200 (+320892us) service_pool.cc:224] Handling call
0709 12:15:31.932558 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:32.069884 (+137326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.371693 (+1301809us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b54001c52d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:33.371871 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220296576}
I20260709 12:15:33.373005 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:33.295061 18713 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.288044 18832 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.311591 18791 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.361546 18630 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.376124 18775 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:33.353892 18399 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:33.378172 18690 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:33.360142 18858 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.351878 18709 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.382329 18840 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:33.386016 18691 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:33.397176 18875 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:15:33.399017 18791 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.398629 18642 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:33.408131 18866 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.394484 18650 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.396785 18794 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.392227 18847 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.398023 18862 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.393548 18775 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:33.395787 18713 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:33.412357 18827 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.675s	user 0.000s	sys 0.014s
I20260709 12:15:33.395587 18851 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.403806 18840 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:33.419672 18856 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.421411 18856 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:33.426267 18857 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.426463 18840 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.428074 18832 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:33.428329 18840 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.428704 18801 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.429265 18846 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:33.383937 11248 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:15:33.472286 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1163) took 1814 ms. Trace:
I20260709 12:15:33.376439 11325 rpcz_store.cc:276] 0709 12:15:31.843524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:32.002794 (+159270us) service_pool.cc:224] Handling call
0709 12:15:32.003205 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:32.665282 (+662077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:32.665329 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:32.665962 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:15:32.866148 (+200186us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:15:32.874588 (+  8440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:32.874781 (+   193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:33.359808 (+485027us) raft_consensus.cc:1685] finished
0709 12:15:33.359829 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.359973 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":295291,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":53,"thread_start_us":861318,"threads_started":2,"wal-append.queue_time_us":524561}
I20260709 12:15:33.472965 11251 rpcz_store.cc:276] 0709 12:15:31.657995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.662068 (+  4073us) service_pool.cc:224] Handling call
0709 12:15:31.662499 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.668994 (+  6495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.669019 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.669416 (+   397us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.737713 (+ 68297us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:31.737777 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.738491 (+   714us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:33.401478 (+1662987us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:33.471455 (+ 69977us) raft_consensus.cc:1685] finished
0709 12:15:33.471496 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.472019 (+   523us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46106,"raft.queue_time_us":14245,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":16526,"spinlock_wait_cycles":168130176,"thread_start_us":67956,"threads_started":1,"wal-append.queue_time_us":67396}
W20260709 12:15:33.480103 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 107010us
I20260709 12:15:33.379868 11327 rpcz_store.cc:276] 0709 12:15:31.590305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.926119 (+335814us) service_pool.cc:224] Handling call
0709 12:15:31.948279 (+ 22160us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:32.642292 (+694013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:32.642347 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:32.643014 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 12:15:32.877871 (+234857us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:32.877938 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.363233 (+485295us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b54000e5e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:33.363288 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:33.363347 (+    59us) raft_consensus.cc:1685] finished
0709 12:15:33.363367 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.363696 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":306272,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":100511488,"thread_start_us":846277,"threads_started":2,"wal-append.queue_time_us":621338}
I20260709 12:15:33.487509 18691 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.401888 18690 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.490080 18713 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.498842 18713 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.508039 18846 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:33.508917 18846 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:33.384368 18665 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.499564 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:33.516302 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:33.421377 18788 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.000s
I20260709 12:15:33.522195 18840 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:33.522534 18795 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:33.522743 18840 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:33.523391 18840 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:33.523469 18795 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:33.530062 18691 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:33.538491 18832 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:33.539039 18713 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.539393 18832 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.541894 18665 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:33.543366 18832 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.509620 18846 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:33.552211 18775 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:33.554005 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55389.4us
I20260709 12:15:33.555260 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:33.557569 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:33.557776 18795 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.559423 18788 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:32.462707 11326 rpcz_store.cc:276] 0709 12:15:30.862215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.958646 (+ 96431us) service_pool.cc:224] Handling call
0709 12:15:30.958965 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:30.959321 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.959357 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.604926 (+645569us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b2c00126d18. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:31.605481 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.954708 (+349227us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:32.387789 (+433081us) log.cc:844] Serialized 14 byte log entry
0709 12:15:32.425888 (+ 38099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:32.425973 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:32.426014 (+    41us) raft_consensus.cc:1685] finished
0709 12:15:32.434968 (+  8954us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:32.435203 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23066,"prepare.queue_time_us":215754,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":80493,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":29164,"spinlock_wait_cycles":64109824,"thread_start_us":704784,"threads_started":2,"wal-append.queue_time_us":612965}
I20260709 12:15:33.579807 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1388) took 2736 ms. Trace:
I20260709 12:15:33.580291 11932 rpcz_store.cc:276] 0709 12:15:30.843247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.845667 (+  2420us) service_pool.cc:224] Handling call
0709 12:15:30.900527 (+ 54860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:31.326342 (+425815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.579285 (+2252943us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b5400650ed0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:33.579554 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125248384}
I20260709 12:15:33.586853 18690 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:33.617420 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.618602 18690 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.619441 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.619551 18690 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.621352 18791 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.622366 18791 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.624840 18689 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.625671 18689 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.626283 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:33.626403 18832 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:33.627030 18832 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:33.627267 11328 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:33.627518 18832 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:33.628202 18689 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.628784 18689 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.629824 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.017s	sys 0.030s
W20260709 12:15:33.630277 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.017s	sys 0.030s
I20260709 12:15:33.625412 18791 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:33.631453 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.631709 18856 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:33.632711 18856 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.633601 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:33.634637 11256 raft_consensus.cc:2368] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18.
I20260709 12:15:33.639818 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.641399 18526 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.645606 18791 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.648952 18791 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.652468 11932 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:33.646898 18847 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:33.647708 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 90356us
I20260709 12:15:33.642637 18788 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:33.641371 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.645965 18526 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.666674 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:33.672673 11329 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:33.629454 18689 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.681795 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.682495 18526 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:33.682757 11327 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
W20260709 12:15:33.683785 18590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.426s	user 0.015s	sys 0.004s
W20260709 12:15:33.684340 18590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.436s	user 0.015s	sys 0.005s
I20260709 12:15:33.684049 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.685066 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.686074 11331 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:33.686488 11327 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:33.659271 18713 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.690306 11256 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:33.699093 11929 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:33.700017 18689 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.705435 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:33.709585 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:33.714205 11326 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:33.716629 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:33.723652 11250 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:33.723045 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1459) took 1562 ms. Trace:
I20260709 12:15:33.711112 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:33.736405 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:33.710259 18690 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.732824 11322 rpcz_store.cc:276] 0709 12:15:32.042984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:32.891479 (+848495us) service_pool.cc:224] Handling call
0709 12:15:33.571244 (+679765us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:15:33.571636 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:33.571998 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:33.572022 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:33.572102 (+    80us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:33.572130 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.582330 (+ 10200us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.605232 (+ 22902us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6878976}
I20260709 12:15:33.747355 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:33.748567 18900 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:33.758248 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:33.759696 18817 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 1.990s	user 0.008s	sys 0.005s
I20260709 12:15:33.755555 18846 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.763522 11248 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:33.766342 11932 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:33.766891 18846 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.768267 18846 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:33.769512 18828 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.514s	user 0.003s	sys 0.007s
I20260709 12:15:33.710000 18788 raft_consensus.cc:2764] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:15:33.733245 18847 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:33.730014 18840 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.782761 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1242) took 2345 ms. Trace:
I20260709 12:15:33.785543 18840 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.786724 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:33.787613 11932 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:33.788582 11248 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:33.794204 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1153) took 2864 ms. Trace:
I20260709 12:15:33.794499 18809 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.794821 11255 rpcz_store.cc:276] 0709 12:15:30.931515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.143283 (+211768us) service_pool.cc:224] Handling call
0709 12:15:31.143578 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:31.164596 (+ 21018us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:15:31.265840 (+101244us) log.cc:844] Serialized 14 byte log entry
0709 12:15:31.344253 (+ 78413us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:31.344307 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:31.344360 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.793511 (+2449151us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b54002a49d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:33.793644 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.793956 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117192,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":1162,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10562,"spinlock_wait_cycles":47582592,"thread_start_us":149403,"threads_started":2,"wal-append.queue_time_us":172887,"wal-append.run_cpu_time_us":1372,"wal-append.run_wall_time_us":1060703}
I20260709 12:15:33.801824 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:33.806352 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59120.9us
I20260709 12:15:33.807852 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:33.808854 18840 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:33.810600 18840 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:33.812208 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:33.815205 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2958) took 2280 ms. Trace:
I20260709 12:15:33.820109 11325 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:15:33.821991 18665 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:33.822347 18627 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:33.823055 18627 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:33.783452 11176 rpcz_store.cc:276] 0709 12:15:31.437098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.437250 (+   152us) service_pool.cc:224] Handling call
0709 12:15:31.437663 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.451243 (+ 13580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.451273 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.458225 (+  6952us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.603193 (+144968us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:31.603244 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.603319 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:33.758608 (+2155289us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:33.773551 (+ 14943us) raft_consensus.cc:1685] finished
0709 12:15:33.773572 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.782393 (+  8821us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19870,"raft.queue_time_us":14663,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":11565,"spinlock_wait_cycles":58467712,"thread_start_us":125328,"threads_started":1,"wal-append.queue_time_us":180480}
I20260709 12:15:33.822590 18665 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:33.815655 11323 rpcz_store.cc:276] 0709 12:15:31.535192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.655995 (+120803us) service_pool.cc:224] Handling call
0709 12:15:31.656301 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.669696 (+ 13395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.669735 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.670259 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.929245 (+258986us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:31.929288 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.929353 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:33.768391 (+1839038us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:33.773277 (+  4886us) raft_consensus.cc:1685] finished
0709 12:15:33.773297 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.815007 (+ 41710us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7833,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":11543,"spinlock_wait_cycles":10191104,"thread_start_us":258441,"threads_started":1,"wal-append.queue_time_us":429131}
I20260709 12:15:33.821290 18609 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:33.789260 11182 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45486 (request call id 1245) took 1890 ms (client timeout 2058 ms). Trace:
I20260709 12:15:33.710505 18713 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.731575 18689 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:33.831924 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.845897 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.847723 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.848618 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.849485 11326 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:33.850070 11324 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:33.850430 11900 leader_election.cc:400] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18.
I20260709 12:15:33.850829 11900 leader_election.cc:403] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:15:33.851411 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70708.9us
I20260709 12:15:33.852228 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.853035 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:33.853873 18689 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.854537 18585 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:33.855370 18795 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:33.856278 18689 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.857162 18698 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:33.859004 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1155) took 2897 ms. Trace:
W20260709 12:15:33.829906 11182 rpcz_store.cc:269] 0709 12:15:31.898799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.899540 (+   741us) service_pool.cc:224] Handling call
0709 12:15:33.788737 (+1889197us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001f40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:33.788995 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087052672}
I20260709 12:15:33.824771 18627 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.829165 18665 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:33.840065 18791 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.879379 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:33.880055 11325 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:33.888590 18520 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.889467 18520 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:33.892000 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:33.893446 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.896874 11329 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:33.900470 11931 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:15:33.903980 18851 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.904722 18851 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.906093 18540 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.906471 18805 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.906948 18540 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.907191 18851 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.908528 18735 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:33.909345 18735 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.910650 18627 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:33.910673 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:33.911923 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:33.912899 18735 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.913571 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:33.914137 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.914800 11326 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:33.915102 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:33.915925 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:33.916613 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:33.917333 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:33.918377 18585 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:33.918897 18585 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.919025 11322 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:33.919338 18585 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:33.917474 18885 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:33.920831 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:33.919835 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:33.921789 18709 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.922060 18555 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:33.920580 18885 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.917220 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.922926 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:33.927625 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.907258 18805 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.943063 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:33.860515 11257 rpcz_store.cc:276] 0709 12:15:30.960749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.160505 (+199756us) service_pool.cc:224] Handling call
0709 12:15:31.180172 (+ 19667us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:31.184758 (+  4586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.184793 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:31.212763 (+ 27970us) log.cc:844] Serialized 52 byte log entry
0709 12:15:33.532333 (+2319570us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b180017ee40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:33.663630 (+131297us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:33.840551 (+176921us) log.cc:844] Serialized 14 byte log entry
0709 12:15:33.853191 (+ 12640us) log.cc:844] Serialized 14 byte log entry
0709 12:15:33.856559 (+  3368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.857021 (+   462us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:33.857535 (+   514us) raft_consensus.cc:1685] finished
0709 12:15:33.857551 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.858365 (+   814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5502,"prepare.queue_time_us":35447,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":121,"raft.queue_time_us":11294,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":22652,"spinlock_wait_cycles":7896448,"thread_start_us":203555,"threads_started":2,"wal-append.queue_time_us":2373302}
I20260709 12:15:33.950021 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.949515 18786 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.955818 18786 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.960116 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:33.922799 18328 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.960395 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:33.922850 18555 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:33.970456 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:33.969535 11175 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:33.978706 18713 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.979249 11175 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:15:33.980386 11173 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:33.981539 11178 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:15:33.983359 18786 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.983784 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.984503 18845 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.984781 11327 raft_consensus.cc:2410] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:15:33.985502 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:33.986117 11327 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:33.986542 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51463.3us
W20260709 12:15:33.987056 18895 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.573s	user 0.002s	sys 0.000s
I20260709 12:15:33.989275 18885 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:33.972436 11328 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:33.963600 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:33.973006 18328 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:33.974539 18805 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:33.975093 18789 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.034s	user 0.016s	sys 0.009s
I20260709 12:15:34.007287 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:33.974774 18690 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:33.976917 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:33.977458 11180 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 18. (term mismatch)
I20260709 12:15:34.011610 11180 pending_rounds.cc:85] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 17
I20260709 12:15:33.959337 18662 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:33.971896 18555 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.968269 11331 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:33.989459 18540 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:33.998653 11326 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:33.973440 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:34.020290 11322 raft_consensus.cc:2410] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:15:34.020290 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.022763 18328 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:34.023496 18713 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:34.023996 18713 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.024441 18713 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:34.002732 18585 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:34.030534 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57169us
I20260709 12:15:34.015638 11180 pending_rounds.cc:107] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:15:34.035426 11180 raft_consensus.cc:2889] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:34.039887 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.043964 11325 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:34.046182 11176 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:34.049220 18824 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:34.050078 18824 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.051863 18585 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:34.054046 11174 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:34.054211 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.055085 11179 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:34.033707 18709 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:34.058313 18709 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.058737 18709 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:34.028584 18840 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:34.067094 11933 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:34.047209 18900 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:34.083185 11176 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:34.046756 18689 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:15:34.085619 11182 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:34.091158 18797 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.092039 18797 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.090088 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.095578 11251 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:34.096055 11251 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:34.073328 18840 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.101989 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.094449 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:34.106314 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.109177 11175 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:34.109598 11175 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:34.109421 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.097496 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:34.111611 18726 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.112200 18726 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.113269 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.114519 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.114567 18726 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:34.116595 18328 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:34.117991 11178 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:34.122617 11175 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.125566 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.098127 18824 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:34.140312 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65633.5us
W20260709 12:15:34.146627 18832 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.518s	user 0.002s	sys 0.015s
I20260709 12:15:34.146929 11251 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.147254 18832 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.148236 11251 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:34.149811 18582 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.150000 18862 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.150597 18582 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.153843 11323 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:34.155754 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.154302 11328 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:34.158975 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.160355 18797 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:34.161036 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:34.166966 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.168089 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.167646 18816 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 12:15:34.149859 18866 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:34.175839 11253 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:34.176159 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.176770 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.153589 18582 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:34.177898 11328 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:34.170782 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:34.179410 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.212760 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2966) took 1510 ms. Trace:
I20260709 12:15:34.181522 18713 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.183856 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.170527 18328 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:34.185073 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.185209 18520 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:34.185901 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.186586 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.170724 18885 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:15:34.174116 18866 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:34.150604 18862 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.191655 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.198459 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.199632 18856 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:34.200433 11932 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:34.201732 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.202385 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1395) took 1317 ms. Trace:
I20260709 12:15:34.202672 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:34.203176 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.204952 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.206154 11175 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:34.206472 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.208519 18555 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:34.210620 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.211450 18840 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:34.212268 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:34.212766 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.216624 11250 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:34.201419 18521 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:34.176395 18832 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:34.242354 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.217128 11330 rpcz_store.cc:276] 0709 12:15:32.702655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:32.934151 (+231496us) service_pool.cc:224] Handling call
0709 12:15:32.934544 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:33.336209 (+401665us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:15:33.351458 (+ 15249us) log.cc:844] Serialized 14 byte log entry
0709 12:15:33.807791 (+456333us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18004c1ec0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a 00007f3ec66f6a2a
0709 12:15:34.020380 (+212589us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:34.020470 (+    90us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:34.020579 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:34.212363 (+191784us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54000c6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:34.212414 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:34.212570 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8576,"prepare.queue_time_us":15064,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":12511,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":190,"spinlock_wait_cycles":926869632,"thread_start_us":143507,"threads_started":1,"wal-append.queue_time_us":753031}
I20260709 12:15:34.238713 11933 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:15:34.241158 11180 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:34.367465 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.337098 11325 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
W20260709 12:15:34.403373 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 318283us
I20260709 12:15:34.363335 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:34.348614 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 81639.1us
I20260709 12:15:34.311650 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.336716 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.327199 11934 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:34.343605 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:34.324182 11327 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:34.364259 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:34.381198 11251 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:15:34.334686 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.239367 11929 rpcz_store.cc:276] 0709 12:15:32.884767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:32.884924 (+   157us) service_pool.cc:224] Handling call
0709 12:15:32.885236 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:33.699766 (+814530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:34.202063 (+502297us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540041aad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:34.202306 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1870278528}
I20260709 12:15:34.388928 11927 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:15:34.243028 11173 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:15:34.319934 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 178518us
I20260709 12:15:34.425830 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:34.348379 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.322024 11176 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:15:34.386240 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:34.326473 18918 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:15:34.427778 11251 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:34.222354 18328 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.361718 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:34.360387 18914 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:15:34.370229 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:34.361096 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.365175 18713 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:34.386075 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:34.365212 18862 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:34.336987 18520 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:34.435494 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.433207 18328 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:34.437036 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:34.434352 11252 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:34.245394 11323 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:34.451285 11248 raft_consensus.cc:2393] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:34.443378 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:34.452854 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82904.9us
I20260709 12:15:34.454255 18265 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:34.454846 11251 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:34.487973 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.366252 18665 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.477725 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.326635 18582 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.491108 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:34.491446 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60090us
I20260709 12:15:34.369189 18709 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.492061 11178 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:15:34.500373 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:34.491034 11180 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:34.497357 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.506603 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.509040 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:34.512410 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.519662 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.520310 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.522351 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:34.523730 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.524210 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.524684 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.505299 18795 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.649s	user 0.014s	sys 0.000s
I20260709 12:15:34.532491 11250 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:34.521075 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:34.534127 11329 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:34.366034 18866 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:34.539294 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 96039.7us
I20260709 12:15:34.505664 18520 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:34.539206 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.537679 11327 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:34.535075 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.550164 11248 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:34.550184 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:34.540889 11255 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:34.541260 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.552165 11325 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:34.545846 11252 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:15:34.546504 11254 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:34.546398 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.535876 18795 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 17. Current term is 18.
I20260709 12:15:34.552157 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:34.567852 11178 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:34.569262 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:34.669567 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.670558 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 105136us
I20260709 12:15:34.674412 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.516840 18582 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.552238 18816 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:34.552731 18791 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:34.720209 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48795.4us
W20260709 12:15:34.670243 11296 connection.cc:256] RPC call timeout handler was delayed by -1.4942585369999506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 12:15:34.734891 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.735824 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.738284 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 190525us
I20260709 12:15:34.721282 18797 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:34.741813 11178 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:34.562147 18840 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:34.710417 18555 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:34.738812 18828 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:15:34.763223 11250 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:15:34.714223 18816 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.765373 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.713721 18856 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:34.771850 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.771068 11254 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:34.780282 11178 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:34.781586 18643 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.782336 18643 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.764312 18555 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.783103 18520 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.785451 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.786223 11248 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:34.787411 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:34.788628 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:34.790582 11296 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.193:45115 timed out after 2.112s (SENT) [suppressed 7 similar messages]
I20260709 12:15:34.789376 11248 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:34.793080 11926 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:15:34.794971 18840 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.795362 18792 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:34.797772 18861 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:34.764153 18797 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:34.767998 18627 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:15:34.778720 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:34.762086 18832 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.776511 18816 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:34.778720 18862 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:34.785394 18555 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:34.844791 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:34.848340 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.814163 18840 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:34.876549 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.880806 11250 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:34.859637 18862 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:34.882217 18862 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:34.885840 11177 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:34.810992 18795 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:15:34.877259 11926 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:34.888160 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:34.866039 18856 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:34.897106 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 213954us
I20260709 12:15:34.898207 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:34.899111 18698 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:34.899355 18856 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:34.907403 11930 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:34.909394 11254 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:34.936674 11255 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:34.941324 11327 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:15:34.943400 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:34.944451 11331 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:34.833827 18791 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.953764 18794 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:34.953224 18791 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:34.938364 18665 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:34.919857 18856 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.932437 18709 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:34.871003 18832 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:34.958534 18840 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:34.983861 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.979971 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76470us
I20260709 12:15:34.933889 18698 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
W20260709 12:15:34.988998 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 198433us
I20260709 12:15:35.001360 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:35.006855 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 487085us
I20260709 12:15:35.011724 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:35.013476 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.015265 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.020586 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:34.981601 18856 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.024798 11255 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:35.003937 18582 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.030009 11327 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:35.025420 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.027005 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:35.034214 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:35.037220 11255 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:15:35.036710 11254 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:35.043466 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:35.051913 11175 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.002s
I20260709 12:15:35.055830 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.058784 11250 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:15:35.022217 18840 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.071946 11327 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:15:35.073676 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:35.077081 11254 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:35.079449 11175 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:15:35.080117 11175 pending_rounds.cc:85] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 12
I20260709 12:15:35.080945 11175 pending_rounds.cc:107] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:15:35.081390 11175 raft_consensus.cc:2889] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:35.097645 11255 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:35.072940 18540 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.053293 18665 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.104611 18540 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.109802 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3336) took 1005 ms. Trace:
I20260709 12:15:35.107517 18941 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.110936 11175 rpcz_store.cc:276] 0709 12:15:34.100322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.107226 (+  6904us) service_pool.cc:224] Handling call
0709 12:15:34.108961 (+  1735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:35.105351 (+996390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:35.105863 (+   512us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:35.126044 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.130146 18941 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.073007 18709 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:35.135727 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:35.138877 18941 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.139834 18941 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:35.140228 18941 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.140595 18941 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:15:35.144057 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74478.3us
I20260709 12:15:35.145041 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.104902 18856 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:35.146173 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 2967) took 1233 ms. Trace:
I20260709 12:15:35.145190 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:35.155776 11327 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:35.158229 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:35.160482 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:35.161968 11179 raft_consensus.cc:2368] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:15:35.164851 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:35.141703 18785 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:15:35.167253 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:35.181700 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 172121us
I20260709 12:15:35.176501 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:35.165668 18932 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:15:35.189028 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.134182 18665 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.164103 18792 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:35.193109 11175 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:35.206784 18540 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.210479 18832 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.211977 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.212910 11176 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:35.202960 18792 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.201524 11174 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:35.150393 11324 rpcz_store.cc:276] 0709 12:15:33.912740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.132888 (+220148us) service_pool.cc:224] Handling call
0709 12:15:34.155899 (+ 23011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:34.188001 (+ 32102us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:34.432276 (+244275us) log.cc:844] Serialized 14 byte log entry
0709 12:15:35.145784 (+713508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:35.145849 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:35.145895 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:35.145968 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:35.146106 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44996,"prepare.queue_time_us":239422,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":229383,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2986,"thread_start_us":835667,"threads_started":3,"wal-append.queue_time_us":517646}
I20260709 12:15:35.214141 18792 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:35.164427 18935 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:35.223827 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:35.227095 11178 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:15:35.165987 18856 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 12:15:35.241739 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:35.249899 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 114258us
I20260709 12:15:35.226830 18939 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:35.232298 18935 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.266289 11178 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:15:35.262774 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:35.273785 18705 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:35.274701 18705 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.289278 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:35.301295 18928 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 12:15:35.301885 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:35.264905 18939 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:35.297792 18630 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:35.305343 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:35.303663 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.296427 18705 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.297194 18935 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:35.305229 18328 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.788s	user 0.015s	sys 0.000s
I20260709 12:15:35.311019 18832 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:35.313885 18328 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.314462 18832 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.314896 18832 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:35.315595 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1256) took 1078 ms. Trace:
I20260709 12:15:35.316013 11181 rpcz_store.cc:276] 0709 12:15:34.237594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.559957 (+322363us) service_pool.cc:224] Handling call
0709 12:15:34.684939 (+124982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:34.738342 (+ 53403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:34.738380 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:34.769915 (+ 31535us) log.cc:844] Serialized 26 byte log entry
0709 12:15:35.159683 (+389768us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:35.159748 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:35.159870 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:35.315361 (+155491us) raft_consensus.cc:1685] finished
0709 12:15:35.315387 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:35.315536 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26031,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":10816,"thread_start_us":300296,"threads_started":1,"wal-append.queue_time_us":428240}
I20260709 12:15:35.317935 11182 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:35.320667 18934 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:15:35.322184 18643 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.316293 18328 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:35.325284 18938 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.188s	user 0.000s	sys 0.002s
I20260709 12:15:35.327147 18840 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:35.310894 18939 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:35.331524 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 107467us
I20260709 12:15:35.338785 11254 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:35.329428 18862 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.344307 18665 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:35.350011 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:35.352272 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47006.5us
I20260709 12:15:35.354208 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:35.359009 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.360363 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:35.362777 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:35.363618 11322 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:35.304931 18940 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:35.369009 11175 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:35.370007 11176 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:35.362445 18643 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:35.357239 18862 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.366695 18665 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:35.369875 18840 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:15:35.381027 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 139389us
I20260709 12:15:35.383288 18936 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:35.384582 18936 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:35.385617 18936 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.389887 11926 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:35.389988 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.390904 11928 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:35.391412 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.392110 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.393045 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.394373 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.400988 18944 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.401698 18944 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.402056 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.402457 18885 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:35.403023 11175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:35.403782 11175 raft_consensus.cc:606] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:35.404069 18944 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.404333 18885 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:15:35.407722 18521 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.985s	user 0.008s	sys 0.004s
I20260709 12:15:35.372342 18856 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:15:35.405429 18885 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:15:35.410501 18328 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:35.375594 18709 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:15:35.378624 18789 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.411548 18652 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:35.410559 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:35.413833 18906 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:35.382503 18665 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:35.404109 11175 raft_consensus.cc:614] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:35.379128 18940 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
W20260709 12:15:35.423889 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55816us
I20260709 12:15:35.428087 11322 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:35.431483 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.434203 11325 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:35.435144 11323 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:35.415378 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.437749 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:35.439759 11324 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 12:15:35.404659 18944 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:15:35.443766 11329 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:15:35.446817 11330 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:35.445255 11324 pending_rounds.cc:85] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:15:35.435290 18792 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.430998 18652 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:35.448134 18944 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:35.448693 18944 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:35.449081 18652 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.449769 18944 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.450906 11249 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:35.451488 18944 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:35.452008 18789 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.453992 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.455367 18832 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.456595 18939 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:35.454133 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 71828.7us
I20260709 12:15:35.430334 18328 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:35.448158 11324 pending_rounds.cc:107] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 20.11
W20260709 12:15:35.478317 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.484299 11324 raft_consensus.cc:2889] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:35.458806 11178 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:15:35.461159 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:35.483309 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:35.459844 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:35.487103 11929 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 27: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:35.450336 18792 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:35.442759 18665 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:35.472486 18952 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
I20260709 12:15:35.494577 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:35.496483 11177 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:35.498242 11331 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:35.450371 18652 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:35.457837 18794 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:15:35.488082 18816 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.602s	user 0.015s	sys 0.001s
I20260709 12:15:35.457563 18832 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:35.506788 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47042.8us
I20260709 12:15:35.509528 18328 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.511999 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:35.513285 11331 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:35.502440 18521 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:35.497457 18949 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:35.501981 18940 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:35.518123 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.508378 18845 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:35.536389 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.518572 18521 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.546501 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:35.544893 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 144882us
I20260709 12:15:35.547456 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:35.510113 18816 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.554021 18789 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.554186 18816 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.554602 18521 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:35.554803 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:35.556408 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:35.537460 18643 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.558013 18643 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:35.558900 18951 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:15:35.560802 18643 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:35.561774 18948 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:35.562196 18805 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:35.562783 18805 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:35.563306 18805 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:35.565428 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.566883 18862 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:35.547086 18582 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:35.567624 18862 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.569815 11322 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:35.557442 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:35.566908 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:35.538944 18845 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:35.544637 18520 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:35.538308 18953 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:35.572141 11175 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:15:35.593076 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.594395 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.595440 11324 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:35.598105 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:35.585297 18328 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:35.579495 18936 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:35.590025 18939 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.597239 18642 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.602578 18698 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:35.665503 18698 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:35.626144 18630 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.584649 18816 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:15:35.630985 18797 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.582798 18906 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 12:15:35.666519 18941 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.000s
I20260709 12:15:35.537765 18265 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.065s
I20260709 12:15:35.638998 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.640376 18520 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.640944 18643 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.641054 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.635905 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3132) took 1287 ms. Trace:
I20260709 12:15:35.684289 11252 rpcz_store.cc:276] 0709 12:15:34.348545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.495654 (+147109us) service_pool.cc:224] Handling call
0709 12:15:34.542259 (+ 46605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:34.556046 (+ 13787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:35.635719 (+1079673us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b5400109850. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:35.635849 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36071808}
I20260709 12:15:35.590453 18940 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:35.653932 18861 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:35.686614 18861 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:35.658450 11930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 1217) took 1766 ms (client timeout 1998 ms). Trace:
I20260709 12:15:35.659246 18840 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:35.659976 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1216) took 1768 ms. Trace:
I20260709 12:15:35.660110 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.622081 18804 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:35.539901 18949 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.621061 18627 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:35.635529 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:35.712059 11179 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:35.676465 18794 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.672729 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:35.637902 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 71121.1us
I20260709 12:15:35.683108 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.657720 18642 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.740499 11178 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:35.693334 11174 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:15:35.687511 11930 rpcz_store.cc:269] 0709 12:15:33.891706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:33.910204 (+ 18498us) service_pool.cc:224] Handling call
0709 12:15:35.658244 (+1748040us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b540069e454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:35.658363 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904969984}
I20260709 12:15:35.650012 18936 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:35.698832 11180 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:35.698530 11927 rpcz_store.cc:276] 0709 12:15:33.891243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:33.910810 (+ 19567us) service_pool.cc:224] Handling call
0709 12:15:33.911310 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:34.420169 (+508859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:35.659742 (+1239573us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b540069fad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:35.659922 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955042688}
I20260709 12:15:35.670192 18582 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:35.673749 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62811.1us
I20260709 12:15:35.718225 18643 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.719926 18861 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.754869 11177 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:35.756055 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:35.756693 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 197000us
I20260709 12:15:35.694825 18840 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:35.760730 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 195422us
I20260709 12:15:35.738493 18698 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.753706 18906 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:35.759806 18936 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:35.676834 18630 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.756405 18794 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.755921 18845 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.700179 18941 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.782284 18862 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:35.790216 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 87382.8us
I20260709 12:15:35.697896 18520 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.799588 11179 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:35.806212 18906 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:35.790655 18956 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:35.789722 18940 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:15:35.814534 18643 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.808295 18939 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:15:35.817543 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.853174 18792 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:35.853519 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:35.854238 11252 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:35.854668 18521 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.836117 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:35.858269 18642 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.858736 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.858894 18642 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:35.859755 18642 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:35.860807 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.826084 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.826403 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:35.863801 11324 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:15:35.864735 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2717) took 1708 ms. Trace:
W20260709 12:15:35.865298 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.865175 11182 rpcz_store.cc:276] 0709 12:15:34.156294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.156458 (+   164us) service_pool.cc:224] Handling call
0709 12:15:35.864512 (+1708054us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:35.864666 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2642240384}
I20260709 12:15:35.860803 18861 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:35.834354 18797 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 12:15:35.831264 18941 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.869015 11176 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 12:15:35.869836 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58758.5us
I20260709 12:15:35.826680 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.870961 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:35.828101 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:35.840832 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 84845.8us
W20260709 12:15:35.820900 11249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3113) took 4458 ms (client timeout 1860 ms). Trace:
I20260709 12:15:35.853596 18792 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:35.872624 11249 rpcz_store.cc:269] 0709 12:15:31.362807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.381582 (+ 18775us) service_pool.cc:224] Handling call
0709 12:15:35.820598 (+4439016us) spinlock_profiling.cc:243] Waited 4.01 s on lock 0x7b54001cf7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:35.820786 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9231408384}
I20260709 12:15:35.881705 11248 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:15:35.881799 11324 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:35.880586 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:35.883850 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:35.885568 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.886291 11327 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:35.886570 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.830060 18845 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:35.888226 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.888824 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:35.889571 11296 leader_election.cc:400] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:15:35.835356 18520 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.891212 11296 leader_election.cc:403] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:15:35.891227 11253 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:35.839200 18956 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:35.821316 18265 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.878027 18797 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:35.842391 18630 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:35.829566 18794 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:35.906873 18792 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.908524 18792 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:35.827788 18949 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:35.806914 18582 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:35.907500 18555 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:35.901021 11251 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:15:35.914441 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:35.929534 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.888262 18941 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.933666 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:35.937652 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56480.1us
I20260709 12:15:35.938591 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:35.831800 18935 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.903800 18866 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:35.943259 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:35.944233 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 145538us
I20260709 12:15:35.944631 11248 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:35.944629 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.945039 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:35.904637 18956 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:35.951627 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.952148 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.953881 11250 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:35.931499 18939 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 18. Current term is 19.
I20260709 12:15:35.953946 11930 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:35.947048 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.933948 18953 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:35.959800 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.951049 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:35.963574 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.959762 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.965672 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.966437 11329 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:35.930948 18555 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:35.968108 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:35.963338 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:35.964526 11255 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:35.973829 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:35.974491 11933 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:35.974869 11252 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:35.976464 11252 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:35.965821 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:35.968959 18953 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:35.973506 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 390245us
I20260709 12:15:35.947525 18627 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:35.988983 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.950887 18713 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:35.994302 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.988821 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:35.947553 18866 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.989573 11330 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:35.991532 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.838850 18906 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:35.992837 11181 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:15:36.000056 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:35.933050 18265 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:35.847008 18328 consensus_queue.cc:1175] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fecfc03de52a4308bc9b045cad5aac1e" responder_term: 22 status { last_received { term: 20 index: 11 } last_committed_idx: 11 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:35.998055 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.956782 18832 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.991885 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:36.000993 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:35.989650 18627 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.001296 11179 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 16.
I20260709 12:15:36.002636 11331 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:36.003816 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:35.959506 18940 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:35.996714 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:36.010933 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.011065 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:36.132817 11254 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:35.988247 18953 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:36.073812 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.008183 18794 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:35.948251 18941 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.008386 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:35.985283 18642 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:35.947290 18630 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:15:36.094310 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:35.946892 18585 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:15:36.011207 11930 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:15:36.001437 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:36.150966 11932 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:15:36.114452 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 153866us
W20260709 12:15:36.128594 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.002660 18866 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:35.997795 18944 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.967094 18885 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.114s
W20260709 12:15:36.154436 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 165664us
I20260709 12:15:36.008423 18521 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:36.143435 11182 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:36.132149 11180 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:36.011934 18805 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.131865 18627 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:36.097469 18713 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:35.995229 18665 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.195636 18665 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:36.153177 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:36.154343 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57549.2us
I20260709 12:15:36.197873 11177 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:36.198467 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:36.198992 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.075829 18705 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:15:36.097996 18709 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:36.197376 18789 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:15:35.985567 18959 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.163s
W20260709 12:15:36.176055 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.176324 18939 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:36.178876 18582 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:36.182337 18840 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.183018 11073 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:36.185772 18936 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.190444 18969 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:36.191895 18713 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.199522 18785 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:36.238919 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.202739 11325 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
W20260709 12:15:36.173035 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 195704us
I20260709 12:15:36.242514 11180 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:36.175634 18964 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:36.199545 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:36.225764 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.123087 18832 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.230085 11253 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:36.231354 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:36.196393 18665 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.231678 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:36.232322 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 79197.1us
I20260709 12:15:36.232561 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.232337 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.257072 11176 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:36.160584 18944 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.195756 18805 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:36.240530 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.234782 18885 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
W20260709 12:15:36.245880 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73059.5us
I20260709 12:15:36.253899 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.254333 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:36.256898 11182 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:36.258767 11328 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:36.235090 18960 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:36.271016 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:36.237942 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 229644us
I20260709 12:15:36.153916 18555 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.274533 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:36.245682 18785 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:36.276394 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 164520us
I20260709 12:15:36.282157 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:36.278348 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78854.5us
I20260709 12:15:36.245429 18964 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.285799 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:36.292840 11223 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.290664 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:36.284904 18521 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:36.296877 11075 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 245) took 52792 ms (client timeout 15000 ms). Trace:
I20260709 12:15:36.265448 18789 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:36.305456 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:36.298763 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:36.281411 18804 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.570s	user 0.005s	sys 0.005s
I20260709 12:15:36.261814 18795 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:15:36.285221 18785 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:36.295470 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:36.290819 18540 consensus_queue.cc:1175] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 19 status { last_received { term: 19 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 19 index: 11 } }
W20260709 12:15:36.315851 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.304039 18643 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:36.298025 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.264119 18665 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.279769 18582 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:36.293897 18555 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:36.322929 11331 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:36.293412 18840 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:36.314522 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57732.8us
I20260709 12:15:36.329054 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:36.289157 18939 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:36.287324 18964 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:36.333444 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:36.333776 11933 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:15:36.303067 18971 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:36.308502 18794 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:36.280499 18792 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 12:15:36.341534 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46164.7us
W20260709 12:15:36.374022 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 76076.8us
W20260709 12:15:36.374325 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 45375.9us
I20260709 12:15:36.374431 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:36.375241 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.338312 18804 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:15:36.304261 18662 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:15:36.347047 18969 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:36.344692 18399 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.285709 18975 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.063s
I20260709 12:15:36.323630 18265 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:36.376087 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.338178 18789 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:36.388993 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:36.343969 18936 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:36.446206 18936 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:36.446734 18936 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:36.447654 18936 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.449661 18966 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:36.450255 18966 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:36.450620 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:36.451483 11322 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:36.451283 18966 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.452374 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:36.452772 18966 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:36.297427 11075 rpcz_store.cc:269] 0709 12:14:43.504415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.535116 (+ 30701us) service_pool.cc:224] Handling call
0709 12:15:36.291721 (+52756605us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:24.266979 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:24.267692 (+   713us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:24.267750 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:15:24.267785 (+    35us) tablet.cc:662] Decoding operations
0709 12:15:24.269254 (+  1469us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:24.269410 (+   156us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:24.269446 (+    36us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:15:24.433009 (+163563us) tablet.cc:701] Row locks acquired
0709 12:15:24.447410 (+ 14401us) write_op.cc:260] PREPARE: finished
0709 12:15:24.480057 (+ 32647us) write_op.cc:270] Start()
0709 12:15:24.504222 (+ 24165us) write_op.cc:276] Timestamp: P: 1783599324455899 usec, L: 0
0709 12:15:24.504275 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 12:15:24.505071 (+   796us) log.cc:844] Serialized 6415 byte log entry
0709 12:15:25.344258 (+839187us) op_driver.cc:464] REPLICATION: finished
0709 12:15:25.526266 (+182008us) write_op.cc:301] APPLY: starting
0709 12:15:25.526380 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:25.933907 (+407527us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:25.934040 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:26.641532 (+707492us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:26.808433 (+166901us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:15:26.846211 (+ 37778us) write_op.cc:312] APPLY: finished
0709 12:15:26.846873 (+   662us) log.cc:844] Serialized 148 byte log entry
0709 12:15:27.103044 (+256171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:27.119045 (+ 16001us) write_op.cc:454] Released schema lock
0709 12:15:27.127735 (+  8690us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:27.325529 (+197794us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:27.333218 (+  7689us) write_op.cc:454] Released schema lock
0709 12:15:27.630952 (+297734us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
Metrics: {"mutex_wait_us":3524478,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":171870,"apply.run_cpu_time_us":54720,"apply.run_wall_time_us":1782754,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71135,"num_ops":16,"prepare.queue_time_us":57207,"prepare.run_cpu_time_us":16169,"prepare.run_wall_time_us":563218,"raft.queue_time_us":106515,"raft.run_cpu_time_us":24122,"raft.run_wall_time_us":116666,"replication_time_us":839759,"spinlock_wait_cycles":127927040,"thread_start_us":719878,"threads_started":5,"wal-append.queue_time_us":901317,"wal-append.run_cpu_time_us":19154,"wal-append.run_wall_time_us":2705820}]]}
I20260709 12:15:36.456120 18935 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:36.441022 18399 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.462066 18399 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:36.462177 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 87835.1us
I20260709 12:15:36.462484 18805 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:36.462966 18805 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:36.463130 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:36.463469 18805 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:36.464031 18328 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:36.464476 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.465298 18328 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:36.392313 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:36.386256 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:36.431077 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:36.465499 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:36.484212 11331 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:15:36.435410 11248 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:36.151814 18940 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.437197 18662 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:36.437052 18985 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
W20260709 12:15:36.440382 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.371073 18971 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:36.443013 18977 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:36.385617 18520 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:36.365136 18643 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.383060 18959 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 12:15:36.383622 18885 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:15:36.393486 18521 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:36.444989 18789 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:36.467427 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.383848 18794 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.386756 18939 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.483315 11330 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:36.457428 18935 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.383486 18582 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:36.466363 11257 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:36.443876 18978 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:36.582796 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 277401us
I20260709 12:15:36.482853 18328 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.519990 18804 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:36.627034 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.604671 18949 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:36.526429 18973 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:15:36.656919 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:36.659247 11221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.016s
W20260709 12:15:36.659907 11221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.004s	sys 0.016s
I20260709 12:15:36.430541 18941 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:36.526273 18971 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:36.510039 18662 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:36.512948 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 126790us
W20260709 12:15:36.616668 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 107668us
I20260709 12:15:36.594898 18520 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:36.594103 18521 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.637872 18328 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:36.637425 18804 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.534356 18969 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:15:36.610263 18794 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:36.486316 18967 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:36.609534 18885 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:36.726888 18885 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:36.727610 18976 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 12:15:36.727357 18662 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.728150 18885 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.729166 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3346) took 1092 ms. Trace:
I20260709 12:15:36.729868 18885 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:36.562948 18832 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:36.663748 18970 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.393s	user 0.001s	sys 0.001s
W20260709 12:15:36.633773 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 257737us
I20260709 12:15:36.734601 18643 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:36.695016 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.667235 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:36.739140 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71998.7us
I20260709 12:15:36.739279 18832 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:36.739879 18832 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:36.739943 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:36.740417 18832 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:36.741201 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.741990 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:36.742632 11257 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:36.672191 18944 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:36.722579 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:36.729568 11173 rpcz_store.cc:276] 0709 12:15:35.636477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:35.643349 (+  6872us) service_pool.cc:224] Handling call
0709 12:15:35.649891 (+  6542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:35.941849 (+291958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:35.941880 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:35.965704 (+ 23824us) log.cc:844] Serialized 26 byte log entry
0709 12:15:36.268200 (+302496us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:36.268235 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:36.268306 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:36.728879 (+460573us) raft_consensus.cc:1685] finished
0709 12:15:36.728904 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:36.729079 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30201,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":21479,"thread_start_us":441342,"threads_started":2,"wal-append.queue_time_us":324076}
I20260709 12:15:36.641168 18949 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:36.510108 11073 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:36.705758 18804 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:36.751590 11073 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 17 to 19, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 19 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:36.754113 18816 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:36.754730 18816 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:36.755910 18816 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.759272 18652 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:36.759876 18652 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:36.761829 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:36.762816 11330 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:15:36.760874 18652 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.766345 18630 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:36.766884 18630 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:36.693971 18521 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:36.770466 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:36.778098 11222 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:36.771952 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.625592 18935 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.762598 18993 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:36.771762 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.699018 18963 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:36.703990 18965 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 12:15:36.795717 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.796499 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.637010 18866 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.694744 18973 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:36.692401 18986 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:36.808615 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:36.746487 18998 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.770902 18662 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:36.714218 18941 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:36.781114 18816 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:36.636626 18978 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:36.821102 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 128397us
I20260709 12:15:36.815826 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:36.827025 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:36.834049 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.703579 18985 consensus_queue.cc:1059] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.110s
I20260709 12:15:36.806309 18936 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:36.829989 11322 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:36.768003 18630 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:36.835767 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:36.836966 11249 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:15:36.814867 18643 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:36.806090 18642 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:15:36.831470 18652 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:36.821352 18861 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.851949 11327 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:36.837435 18665 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:36.840540 18936 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.849325 18643 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:36.879297 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 107594us
W20260709 12:15:36.898178 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.852120 18990 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:36.857931 18265 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:36.827853 18998 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.836112 18941 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:36.949436 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.957487 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:36.821614 18963 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:36.946851 18643 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:36.963768 18963 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:36.804032 11073 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 15 to 20, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 20 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:36.881311 18936 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:37.017865 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2712) took 1071 ms. Trace:
I20260709 12:15:37.018303 11249 rpcz_store.cc:276] 0709 12:15:35.946529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:35.969704 (+ 23175us) service_pool.cc:224] Handling call
0709 12:15:35.991493 (+ 21789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:36.846972 (+855479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.017544 (+170572us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54002a4ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:37.017796 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1943883008}
I20260709 12:15:36.965292 11073 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 13 to 14, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 14 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:36.959076 18990 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:36.979406 18785 consensus_meta.cc:287] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 12:15:36.944417 18665 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:37.039944 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 217031us
I20260709 12:15:37.077615 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:37.042639 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 208511us
I20260709 12:15:36.959548 18642 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:37.045935 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.849112 18520 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:37.078956 11324 raft_consensus.cc:2410] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:15:36.956936 18940 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:36.979573 18939 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:36.946305 18861 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.073716 18990 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:36.980515 18832 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:37.220391 18994 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 12:15:37.248082 18949 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:37.207829 19002 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
I20260709 12:15:36.959759 18265 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:37.297135 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:37.505591 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:37.323428 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.016896 18969 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 12:15:37.036952 19006 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.043113 18987 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:37.196914 18992 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:37.220860 19011 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:37.237774 19003 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:37.437086 11327 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.570s	user 0.016s	sys 0.010s
I20260709 12:15:37.106037 18997 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.402s
I20260709 12:15:36.961154 18973 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.525277 11327 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
W20260709 12:15:37.484577 11178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.005s	sys 0.005s
W20260709 12:15:37.485829 11328 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.002s	sys 0.007s
W20260709 12:15:37.486146 11325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.003s	sys 0.007s
W20260709 12:15:37.486397 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.003s	sys 0.007s
W20260709 12:15:37.486776 11330 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.003s	sys 0.006s
W20260709 12:15:37.487370 11181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.004s	sys 0.006s
W20260709 12:15:37.488786 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50558.9us
I20260709 12:15:37.155571 18642 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:37.108371 18785 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:37.493117 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.003s	sys 0.007s
W20260709 12:15:37.560139 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.003s	sys 0.008s
W20260709 12:15:37.487104 18975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.012s	sys 0.014s
I20260709 12:15:37.502655 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:37.511401 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.028s	sys 0.112s
W20260709 12:15:37.560885 11330 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.004s	sys 0.007s
W20260709 12:15:37.566207 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.028s	sys 0.112s
W20260709 12:15:37.554991 11178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.005s	sys 0.005s
W20260709 12:15:37.511897 11177 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.061s	sys 0.078s
W20260709 12:15:37.567431 11177 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.061s	sys 0.078s
I20260709 12:15:37.474601 18963 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:37.560531 11325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.003s	sys 0.007s
W20260709 12:15:37.565478 11328 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.003s	sys 0.007s
I20260709 12:15:37.114504 18665 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:37.564977 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 613333us
W20260709 12:15:37.574739 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.579114 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:37.578119 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:37.555858 11181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.004s	sys 0.006s
I20260709 12:15:37.515578 18265 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:37.587723 11182 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.006s	sys 0.018s
W20260709 12:15:37.590814 11182 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.007s	sys 0.018s
W20260709 12:15:37.555553 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.003s	sys 0.007s
W20260709 12:15:37.594328 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 758589us
W20260709 12:15:37.559127 19001 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.558s	user 0.006s	sys 0.000s
I20260709 12:15:37.524148 18866 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:37.597275 18963 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:37.620162 19022 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.620704 19022 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.621881 18997 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:37.622516 18997 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.623001 19022 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:37.623301 18986 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:37.623708 19022 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:37.623855 18986 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:37.624142 19022 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:37.624634 19022 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:37.624984 18997 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:37.565315 18975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.013s	sys 0.014s
I20260709 12:15:37.529189 19006 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:37.626744 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.527904 18987 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:37.629729 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.630002 18967 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.630551 18967 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:37.632434 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:37.632663 19009 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:15:37.633446 18999 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.680s	user 0.002s	sys 0.005s
I20260709 12:15:37.634148 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.634778 11176 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:37.635656 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.635752 18998 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:37.636217 11176 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:37.637987 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:37.597213 19011 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:37.642983 19017 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.643550 19017 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.644273 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:37.644932 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:37.645422 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:37.566222 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63675.2us
I20260709 12:15:37.647207 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:37.598649 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:37.599323 11173 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:37.563364 19003 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.607031 19029 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.674904 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:37.589053 18994 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 12:15:37.607697 18995 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.459s	user 0.001s	sys 0.000s
I20260709 12:15:37.608021 19010 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:37.608322 18981 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:15:37.612196 18665 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:37.614475 19005 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:15:37.672049 18992 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.621160 18963 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:37.509727 18940 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:37.631332 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:37.672034 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.557931 18935 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:37.608062 18709 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.408s	user 0.006s	sys 0.010s
I20260709 12:15:37.575127 18960 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:37.678846 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:37.607043 18805 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.143s	user 0.012s	sys 0.004s
I20260709 12:15:37.688941 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3009) took 1368 ms. Trace:
I20260709 12:15:37.692588 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:37.697402 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 98815.8us
I20260709 12:15:37.701277 18978 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:37.675204 18866 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.784602 18866 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.698803 18989 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:37.699452 18642 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:37.699692 18555 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.334s	user 0.017s	sys 0.003s
I20260709 12:15:37.527787 18949 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:37.709566 18906 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:37.727638 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.732527 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.813943 11173 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:37.734751 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.814670 11176 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:37.816004 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.737164 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:37.693223 18399 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:37.740320 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:37.740768 11328 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:37.692499 18981 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:37.741981 11928 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:37.676396 18797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.010s	sys 0.019s
W20260709 12:15:37.897933 18797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.010s	sys 0.019s
I20260709 12:15:37.742338 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:37.899919 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 157680us
I20260709 12:15:37.694082 18785 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:37.746050 18662 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.912s	user 0.009s	sys 0.013s
I20260709 12:15:37.746099 18585 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:15:37.741255 11329 rpcz_store.cc:276] 0709 12:15:36.320509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.321643 (+  1134us) service_pool.cc:224] Handling call
0709 12:15:36.322005 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:36.537649 (+215644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:36.537710 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:36.586674 (+ 48964us) log.cc:844] Serialized 26 byte log entry
0709 12:15:36.685745 (+ 99071us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:36.685808 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:36.685925 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:37.688701 (+1002776us) raft_consensus.cc:1685] finished
0709 12:15:37.688722 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:37.688869 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134600,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":270536,"threads_started":2,"wal-append.queue_time_us":102823}
I20260709 12:15:37.748499 18972 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:37.950462 11928 raft_consensus.cc:2410] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:15:37.751333 18967 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:37.902230 19006 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:37.756614 18840 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.387s	user 0.011s	sys 0.000s
I20260709 12:15:37.761793 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:37.762569 19004 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:15:37.769241 18964 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.382s	user 0.014s	sys 0.002s
W20260709 12:15:37.966676 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 320814us
W20260709 12:15:37.967366 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 205653us
W20260709 12:15:37.950073 18953 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.786s	user 0.008s	sys 0.014s
W20260709 12:15:37.770939 18643 consensus_meta.cc:287] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.808s	user 0.006s	sys 0.006s
I20260709 12:15:37.772476 18861 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:37.772737 19017 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:37.774618 11327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3377) took 1393 ms (client timeout 1757 ms). Trace:
I20260709 12:15:37.775213 18630 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:37.740162 18709 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:37.782032 18805 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:37.782998 18794 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.971s	user 0.016s	sys 0.006s
W20260709 12:15:37.783618 18845 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.883s	user 0.002s	sys 0.013s
I20260709 12:15:37.687929 19029 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.792932 11175 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:37.719489 18969 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:37.796782 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3381) took 1028 ms. Trace:
W20260709 12:15:37.760166 18804 consensus_meta.cc:287] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.009s	user 0.007s	sys 0.002s
I20260709 12:15:37.727164 18665 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.651573 19011 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:37.730690 18941 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:37.846555 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:38.005230 18627 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.781s	user 0.017s	sys 0.000s
I20260709 12:15:37.816825 11176 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:37.602635 11073 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 12 to 18, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:15:38.031641 18944 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.053s	user 0.012s	sys 0.019s
I20260709 12:15:37.834159 18885 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.732092 18940 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:37.742414 18540 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 12:15:37.723487 18992 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:37.867619 18936 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.844s	user 0.008s	sys 0.003s
I20260709 12:15:38.042448 18998 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.876925 11174 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:37.884462 18832 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:37.887194 18966 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.434s	user 0.022s	sys 0.039s
I20260709 12:15:37.901422 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:38.132979 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 231594us
I20260709 12:15:38.134101 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:37.910822 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1284) took 1099 ms. Trace:
I20260709 12:15:37.933077 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:37.953199 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:37.749249 11295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.009s	sys 0.018s
I20260709 12:15:37.933552 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:38.138388 11295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.010s	sys 0.018s
W20260709 12:15:37.954175 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.850871 18399 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:37.959913 11255 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 12:15:38.139447 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.960081 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:37.872411 19031 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.963742 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:37.769593 19014 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:37.911674 18978 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:37.953774 18972 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:38.002295 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:38.158856 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.027663 11181 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:15:38.026222 11327 rpcz_store.cc:269] 0709 12:15:36.381600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.388637 (+  7037us) service_pool.cc:224] Handling call
0709 12:15:37.774388 (+1385751us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b54000c67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:37.774518 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975999360}
W20260709 12:15:38.040343 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 71944.6us
I20260709 12:15:38.160810 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:37.874818 18987 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:15:37.966769 18789 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:15:38.048138 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 399909us
W20260709 12:15:37.968228 18816 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.132s	user 0.015s	sys 0.004s
I20260709 12:15:38.188355 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:37.721807 18862 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.091s
W20260709 12:15:38.031908 18956 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 2.042s	user 0.014s	sys 0.000s
I20260709 12:15:38.039942 11330 rpcz_store.cc:276] 0709 12:15:36.768814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.794158 (+ 25344us) service_pool.cc:224] Handling call
0709 12:15:36.815118 (+ 20960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:36.815313 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:37.566395 (+751082us) log.cc:844] Serialized 14 byte log entry
0709 12:15:37.796002 (+229607us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:37.796343 (+   341us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:37.796452 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.796547 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:37.796718 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35300,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":251,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":36401,"spinlock_wait_cycles":58014592,"thread_start_us":963314,"threads_started":2,"wal-append.queue_time_us":311469}
I20260709 12:15:37.968465 18973 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:37.849936 18971 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.152s	user 0.011s	sys 0.012s
I20260709 12:15:38.044742 18969 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.114137 18709 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:38.048765 18328 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.336s	user 0.012s	sys 0.000s
I20260709 12:15:38.042887 18861 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:38.054939 18944 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.095s	user 0.013s	sys 0.019s
I20260709 12:15:38.140110 11249 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:38.138422 11253 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:38.217836 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:38.138593 11257 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:37.987882 18964 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.135947 11177 rpcz_store.cc:276] 0709 12:15:36.811335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.824058 (+ 12723us) service_pool.cc:224] Handling call
0709 12:15:36.837479 (+ 13421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:36.837731 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:15:37.573058 (+735327us) log.cc:844] Serialized 14 byte log entry
0709 12:15:37.910362 (+337304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:37.910438 (+    76us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:37.910503 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.910592 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:37.910747 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10172,"prepare.queue_time_us":788139,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":1569,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7051,"thread_start_us":1044172,"threads_started":2,"wal-append.queue_time_us":457568}
I20260709 12:15:38.034518 18794 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:38.147157 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.013s	sys 0.058s
W20260709 12:15:38.221030 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.013s	sys 0.058s
I20260709 12:15:38.151162 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:38.147346 11248 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:38.042398 18643 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.049489 11173 raft_consensus.cc:2410] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:15:38.043773 18953 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.198064 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:38.050000 18804 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.054338 18885 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.142824 18941 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:38.049288 18665 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.106079 18997 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:15:38.158118 18840 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:15:38.187448 19024 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:15:38.055248 11933 raft_consensus.cc:2376] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18. Re-sending same reply.
I20260709 12:15:38.226569 18709 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:38.247416 18265 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.003s
I20260709 12:15:38.247735 18709 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.247833 18265 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.237875 18789 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:38.221328 18328 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.251521 18328 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.224372 18816 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:38.252398 18816 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:38.253334 18816 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.159258 18866 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.103881 18627 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:38.213048 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 225022us
I20260709 12:15:38.224647 18520 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:38.143577 18998 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.126847 18936 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.232311 18978 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:15:38.212736 18652 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.350s	user 0.016s	sys 0.006s
I20260709 12:15:38.223389 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:38.204282 18785 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:38.155342 19031 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:38.238394 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.280295 11930 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:38.280732 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.241603 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:38.256160 11323 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
W20260709 12:15:38.257535 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.199477 18642 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:38.261359 11223 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:38.225406 19046 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:38.271404 18992 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:38.272358 19038 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:15:38.272580 18936 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:38.211530 18956 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:38.224964 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 73905.7us
I20260709 12:15:38.226655 18861 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.244020 18840 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:38.218631 18971 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:38.309717 19034 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.434s	user 0.003s	sys 0.006s
I20260709 12:15:38.286477 11934 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:15:38.310223 11324 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:38.311272 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1286) took 1492 ms. Trace:
I20260709 12:15:38.312613 18940 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:38.311671 11178 rpcz_store.cc:276] 0709 12:15:36.819337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.819506 (+   169us) service_pool.cc:224] Handling call
0709 12:15:36.831542 (+ 12036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:36.876516 (+ 44974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:36.876554 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:36.877047 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:15:37.567172 (+690125us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:37.567212 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.567318 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:38.309087 (+741769us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:38.310953 (+  1866us) raft_consensus.cc:1685] finished
0709 12:15:38.310979 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:38.311210 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38760,"raft.queue_time_us":142564,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":21470,"spinlock_wait_cycles":111507712,"thread_start_us":588738,"threads_started":1,"wal-append.queue_time_us":846596}
I20260709 12:15:38.315946 11249 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:38.218861 18973 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:38.291097 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49599.5us
I20260709 12:15:38.292375 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.263197 19049 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.292870 18966 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:38.286453 18642 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:38.297911 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.299278 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:38.335398 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:38.336345 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:38.275628 18978 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:38.292507 19036 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:15:38.307685 18969 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.309669 18971 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:38.317788 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56523us
I20260709 12:15:38.317236 18840 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:38.290794 18652 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:15:38.294926 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.322590 11328 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:15:38.323307 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.325742 18987 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.325894 19003 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.326545 11330 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:38.327134 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.377322 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.377955 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.378592 11251 raft_consensus.cc:2368] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18.
I20260709 12:15:38.379036 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:38.328606 19010 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.000s
I20260709 12:15:38.328920 19040 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:15:38.329052 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.329805 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.330425 18953 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.380549 11249 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:15:38.332120 18956 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.332571 11179 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:38.338438 11926 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:38.261821 18972 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.341326 18936 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:15:38.341787 19023 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:38.342794 11253 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:38.343391 19029 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.342716 11073 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 12 to 19, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 19 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:38.343742 18832 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:38.432729 11180 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:38.433713 11180 raft_consensus.cc:606] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:38.345338 18861 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.346660 18964 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.347776 18627 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:38.301468 18805 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.356812 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:38.357739 11256 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
W20260709 12:15:38.362278 19042 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:38.305249 18265 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.279609 18520 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.320843 18941 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.341430 18978 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:38.348524 18966 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:38.285022 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61728.7us
I20260709 12:15:38.453938 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:38.366556 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.454861 11328 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:38.455737 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.376361 11928 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:38.327735 18940 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:38.339926 19022 consensus_meta.cc:287] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.000s
I20260709 12:15:38.385051 11326 raft_consensus.cc:2410] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 12:15:38.389240 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46112.4us
I20260709 12:15:38.443390 19023 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:38.335521 19049 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.384315 18969 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:38.390872 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.446075 18797 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:38.392819 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.392948 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.402091 18785 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:38.405105 18973 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.406329 18665 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:38.406999 18794 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.434144 11180 raft_consensus.cc:614] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:38.442394 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.387372 19010 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:38.440255 18986 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.808s	user 0.014s	sys 0.000s
I20260709 12:15:38.500504 18986 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.410989 19040 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:38.432840 18936 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:38.458189 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.463241 11328 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:38.502341 19040 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.503139 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:38.389537 19060 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:38.503750 18665 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:38.504405 18665 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.504863 19060 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:38.432667 18832 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:38.466948 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:38.468274 19046 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:38.469413 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1282) took 1662 ms. Trace:
I20260709 12:15:38.452710 18941 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.511932 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.469858 19022 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.513451 18399 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.515455 19022 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.469568 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.474184 11254 raft_consensus.cc:2410] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:15:38.518282 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:38.519080 11253 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:38.522060 19055 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:15:38.522724 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.523416 11251 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:38.524420 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.525038 11251 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:38.475205 18520 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.479245 11181 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:15:38.526912 18973 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:38.448019 19041 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.528906 19041 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.531360 19041 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.531976 19023 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.535219 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3386) took 1740 ms. Trace:
I20260709 12:15:38.535620 11324 rpcz_store.cc:276] 0709 12:15:36.795399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:37.492765 (+697366us) service_pool.cc:224] Handling call
0709 12:15:37.539170 (+ 46405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:38.310655 (+771485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:38.534961 (+224306us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54004597d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:38.535147 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772808576}
W20260709 12:15:38.452975 18949 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.000s
I20260709 12:15:38.485594 11330 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:38.538298 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.390869 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.539883 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.540546 11927 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:38.387742 18987 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.451179 18972 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:38.545851 18966 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.550315 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:38.550925 18981 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.552201 18981 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:38.552191 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.510103 18832 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:38.508203 11073 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:38.509792 19046 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:38.557417 19046 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:38.559008 18990 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.032s	user 0.008s	sys 0.008s
I20260709 12:15:38.559520 18990 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.560830 11255 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:38.562467 18885 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.567239 18804 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.446527 18642 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.509898 11182 rpcz_store.cc:276] 0709 12:15:36.806941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.807392 (+   451us) service_pool.cc:224] Handling call
0709 12:15:36.821796 (+ 14404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:37.591228 (+769432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:37.591275 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:37.685758 (+ 94483us) log.cc:844] Serialized 52 byte log entry
0709 12:15:38.010909 (+325151us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:38.010971 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:38.011086 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:38.469176 (+458090us) raft_consensus.cc:1685] finished
0709 12:15:38.469197 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:38.469357 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17303,"raft.queue_time_us":115367,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":12398,"thread_start_us":1059130,"threads_started":2,"wal-append.queue_time_us":572734}
W20260709 12:15:38.573731 18521 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.759s	user 0.007s	sys 0.007s
I20260709 12:15:38.574433 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1281) took 1778 ms. Trace:
I20260709 12:15:38.574252 18521 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.574826 11179 rpcz_store.cc:276] 0709 12:15:36.796481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.797320 (+   839us) service_pool.cc:224] Handling call
0709 12:15:36.815351 (+ 18031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:38.386359 (+1571008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:38.574144 (+187785us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540016eed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:38.574367 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3488873216}
I20260709 12:15:38.576761 11256 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:38.579123 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.581952 18521 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.582933 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.583622 11926 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:38.584286 18986 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:38.476574 18940 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.545336 18987 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.452644 18866 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.489944 11331 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:38.463967 18643 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.489003 19044 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.115s
I20260709 12:15:38.491300 18797 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.497854 19049 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:38.497884 19035 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.620s	user 0.007s	sys 0.002s
I20260709 12:15:38.494921 18785 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:38.498947 18977 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.897s	user 0.017s	sys 0.011s
I20260709 12:15:38.489020 18969 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.608541 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:38.660858 11176 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:38.465173 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:38.503764 11328 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:38.510627 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 43782.3us
I20260709 12:15:38.505450 19060 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.513339 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:38.608192 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.664919 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.502708 18936 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.632998 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.673516 11324 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:38.619086 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:38.674621 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 209501us
W20260709 12:15:38.501431 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.606372 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:38.603597 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.632423 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.602932 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1490) took 2639 ms. Trace:
I20260709 12:15:38.676776 11323 rpcz_store.cc:276] 0709 12:15:35.963676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:35.967380 (+  3704us) service_pool.cc:224] Handling call
0709 12:15:35.991493 (+ 24113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:38.290871 (+2299378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:38.597447 (+306576us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54000c1cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:38.602853 (+  5406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5207801728}
I20260709 12:15:38.594146 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.584630 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.659802 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.680020 11173 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:38.659375 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:38.605522 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.626981 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.509356 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:38.709889 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46344.7us
I20260709 12:15:38.718051 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.678813 11931 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 16.
I20260709 12:15:38.717370 11180 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:38.675441 11182 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:38.720985 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.722669 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.723531 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:38.723598 11257 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:38.724392 11331 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:38.682169 11175 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:38.687563 11179 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:38.686604 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 177328us
W20260709 12:15:38.596580 19027 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.708s	user 0.010s	sys 0.002s
I20260709 12:15:38.731813 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.732475 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.733080 18785 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.733649 18845 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.733798 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.734226 18845 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.735240 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:38.735819 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.736716 11933 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
W20260709 12:15:38.740453 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.740756 11255 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:38.740465 11251 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:38.732556 11249 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:38.743975 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:38.677801 11177 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:38.739593 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:38.746059 11256 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:38.681661 11928 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:38.732618 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.748164 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.680830 11253 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:38.751374 11176 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:38.665647 18642 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:38.595745 18941 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:38.588811 19031 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.672802 18885 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.625201 18998 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.672266 18977 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:38.710714 18964 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:38.587049 18665 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.760574 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:38.762928 11251 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:15:38.664760 18967 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:38.767848 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.774343 11257 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:15:38.775753 11249 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:15:38.782037 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.784689 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.724392 18521 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:15:38.784740 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 110071us
I20260709 12:15:38.786712 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.684304 18990 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:38.802433 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.806265 11328 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:38.722884 18805 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:38.808974 11927 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:38.810026 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.811373 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:38.813019 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.817124 19031 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:38.817833 11073 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 14 to 16, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 16 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:38.820335 18845 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:38.821803 18969 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.826328 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.838970 11250 raft_consensus.cc:2410] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:38.850167 19022 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:38.850783 18941 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.851132 19011 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.851293 18989 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:38.852039 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:38.854087 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.989567 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:38.853996 19060 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:38.858624 18885 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.881594 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.883176 18816 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:15:38.882977 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 122485us
I20260709 12:15:38.998606 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:38.879201 18940 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.887426 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:38.888389 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.889014 11174 raft_consensus.cc:2410] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:15:38.890195 11324 raft_consensus.cc:2410] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:15:38.890899 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.007009 11329 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:39.007939 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.891851 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:39.008836 11328 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:15:38.892977 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.893596 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:39.010262 11930 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:38.917986 11935 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
I20260709 12:15:38.894168 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.013011 11248 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:38.894196 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:38.896273 11926 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
I20260709 12:15:38.896282 11255 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:38.896692 11182 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:38.929317 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.016532 11927 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:38.756918 18987 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:38.758502 18520 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:38.996978 18969 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:15:39.008692 18713 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:39.018959 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.020093 11928 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:38.996838 19011 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.020255 11073 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:39.022547 19011 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:39.023602 11181 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
W20260709 12:15:39.023855 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 130343us
I20260709 12:15:39.021479 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:38.970757 18665 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:39.025159 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 213904us
I20260709 12:15:39.014223 11257 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:39.028221 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.028831 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.029259 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.029453 11174 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:39.028851 11248 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:38.683234 18866 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:38.976718 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.038017 11176 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:38.892239 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:38.980640 11179 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:39.038650 11933 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:38.982452 11253 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:38.978664 11326 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:38.987756 11178 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:39.039824 11178 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.040206 11178 raft_consensus.cc:614] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:39.039835 11179 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:15:38.988617 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:39.043447 11248 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
W20260709 12:15:38.880492 18906 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.033s	user 0.010s	sys 0.001s
I20260709 12:15:38.884752 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:39.044204 11251 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:39.041414 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:39.001307 11147 leader_election.cc:400] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18.
I20260709 12:15:39.045441 11147 leader_election.cc:403] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:39.003192 11175 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:15:39.004930 11900 leader_election.cc:400] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:39.025944 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:39.048050 11331 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:39.033789 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:39.034018 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.049350 11182 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:39.034392 19060 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:39.034657 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.050393 11930 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:39.035332 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3010) took 2709 ms. Trace:
W20260709 12:15:39.034032 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 182062us
I20260709 12:15:39.051627 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:39.052197 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.052937 11329 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:38.883441 19039 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.317s
I20260709 12:15:39.061507 11323 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:39.051144 11325 rpcz_store.cc:276] 0709 12:15:36.326423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.327017 (+   594us) service_pool.cc:224] Handling call
0709 12:15:36.341400 (+ 14383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:36.604333 (+262933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:36.604366 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:36.625055 (+ 20689us) log.cc:844] Serialized 26 byte log entry
0709 12:15:37.583573 (+958518us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:37.583810 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.583923 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:38.595727 (+1011804us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:39.010020 (+414293us) raft_consensus.cc:1685] finished
0709 12:15:39.010053 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:39.035265 (+ 25212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115514,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":34857856,"thread_start_us":1129672,"threads_started":2,"wal-append.queue_time_us":1171501}
I20260709 12:15:39.062870 11248 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:15:39.032982 18265 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:39.053125 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:39.068272 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.053855 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.054040 11177 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:39.054888 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.069193 11932 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:15:38.885061 18967 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:39.057494 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.060034 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.021867 11073 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 13 to 14, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 14 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:38.877610 18840 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.112200 18840 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:38.892159 18805 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.117897 11327 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:39.119958 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:39.070062 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:39.124449 11251 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:39.113435 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:38.898970 19046 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.126730 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:39.127797 11331 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:39.128296 19046 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:39.128961 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.023140 18862 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.080873 11249 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:39.001744 18845 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:15:39.082125 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.077862 18944 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:15:39.082815 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.083096 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:39.037508 18969 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:39.090986 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:39.091569 18989 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:39.092801 11328 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:39.187465 11175 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:39.092980 11182 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:39.093935 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1238) took 1450 ms. Trace:
I20260709 12:15:39.188639 11932 rpcz_store.cc:276] 0709 12:15:37.643817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:37.645571 (+  1754us) service_pool.cc:224] Handling call
0709 12:15:37.645937 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:39.069902 (+1423965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.093603 (+ 23701us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400673ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:39.093869 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3264556800}
I20260709 12:15:39.094362 18885 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:39.095860 11935 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:15:39.097920 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.099977 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.101888 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:39.191653 11324 raft_consensus.cc:2368] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27.
I20260709 12:15:39.102481 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:39.191917 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 100988us
I20260709 12:15:39.192412 11325 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
W20260709 12:15:39.102627 18972 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.001s
I20260709 12:15:39.103490 18797 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:39.103955 11179 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:15:39.104302 18990 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:39.198952 11329 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:15:39.108958 18832 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.000s
I20260709 12:15:39.208040 18520 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.208601 18520 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.129268 11929 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:39.210443 18944 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18.
I20260709 12:15:39.211097 11329 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:39.213289 11329 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:15:39.036940 18987 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.213617 11329 pending_rounds.cc:85] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 10
I20260709 12:15:39.215366 11329 pending_rounds.cc:107] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:15:39.215592 11329 raft_consensus.cc:2889] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:39.215619 18987 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:39.131793 18967 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:39.027535 18642 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.978246 19031 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:38.986057 19022 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.077600 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.121447 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:39.223062 11327 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:38.993376 18941 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.225700 11934 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:39.133394 11176 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:39.142195 11256 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:39.180967 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:39.227541 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46646.1us
I20260709 12:15:39.151369 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:39.184804 11933 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:39.193183 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:39.057160 19060 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:39.165690 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:39.194063 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 113708us
I20260709 12:15:39.230199 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.232331 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:39.068825 18265 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:39.196540 18989 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:39.201920 11324 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:39.201797 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.203536 18990 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:39.206197 11073 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 11 to 18, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:39.200210 11331 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:39.109669 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 115466us
I20260709 12:15:39.056440 18964 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.140s
I20260709 12:15:39.207892 18832 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.117363 18805 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:39.228813 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 77573.4us
I20260709 12:15:39.186961 18969 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.283643 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.078162 18960 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.284343 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.080550 18940 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:39.234087 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.240944 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.297325 11256 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:39.251281 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.224596 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.298910 11179 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:39.220815 18966 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.088s	sys 0.218s
I20260709 12:15:39.042204 18665 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:39.258227 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.254572 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:39.309213 11330 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:39.265394 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.265982 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.309962 11181 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:39.268008 18987 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:39.199061 19011 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:39.269999 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.312364 11175 raft_consensus.cc:2410] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:39.217965 18642 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.272137 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.285683 11929 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:39.280288 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:39.318506 11178 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:39.319617 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:39.233150 19060 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:39.290308 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.289849 11323 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:15:39.321326 18978 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.834s	user 0.011s	sys 0.000s
I20260709 12:15:39.290380 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:39.327773 11327 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:39.290300 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.249969 18265 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:39.238862 19046 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:39.263360 19023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.005s	sys 0.015s
I20260709 12:15:39.332826 11148 leader_election.cc:400] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27.
I20260709 12:15:39.232767 18643 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:39.303815 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:39.304684 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.305610 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.249529 18797 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:39.351740 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.265100 18944 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:15:39.266355 11928 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:39.249604 18989 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:39.309505 11251 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:39.313355 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62153.4us
I20260709 12:15:39.358455 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.318467 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:39.252985 19040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.002s	sys 0.017s
W20260709 12:15:39.291484 18981 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.005s
I20260709 12:15:39.329828 18805 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:39.375367 11327 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:39.199659 18862 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.330637 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:39.283910 18986 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:39.312692 18966 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.088s	sys 0.218s
I20260709 12:15:39.301301 18969 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:39.326602 19006 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:39.333233 11148 leader_election.cc:403] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:15:39.249524 18977 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:15:39.333755 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.319883 18990 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:15:39.307575 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:39.398588 11927 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:39.353914 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.399808 11182 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:39.354166 11935 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:39.348954 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.401919 11180 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:39.355448 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.355818 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.356217 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:39.402583 11181 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:39.402865 11252 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:39.356460 18665 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:39.357156 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.403911 11253 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:39.318257 19011 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.319084 19032 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:39.364199 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.364638 11932 raft_consensus.cc:2393] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:39.367182 11249 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:39.358994 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.368345 11256 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:15:39.332837 19046 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.265077 18521 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:39.325152 18964 consensus_queue.cc:1243] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 25.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:15:39.332983 19023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.006s	sys 0.016s
I20260709 12:15:39.379056 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.381110 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:39.330629 18987 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:39.362843 19040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.002s	sys 0.018s
I20260709 12:15:39.318976 18816 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:15:39.388908 18986 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.420866 18816 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:39.421360 18816 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:39.422706 18986 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:39.422465 18816 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.423319 19006 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:39.424150 18816 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:39.424424 19006 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.386834 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:39.426136 19061 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:15:39.380468 18713 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:15:39.428107 18987 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.430248 11181 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:39.430711 11181 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:39.431483 11181 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.432716 11181 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:39.434422 18832 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:39.249780 19010 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 12:15:39.397505 11930 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
W20260709 12:15:39.368980 18971 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.015s	sys 0.021s
W20260709 12:15:39.435863 18971 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.015s	sys 0.021s
I20260709 12:15:39.439682 11177 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:39.403735 18652 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:39.314159 18960 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.375418 18805 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:39.414775 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55898.6us
I20260709 12:15:39.415052 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.410902 18521 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.419260 18642 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:39.419814 19022 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:39.426216 11324 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:39.329494 18978 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.441720 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:39.440197 18964 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:39.441707 11173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.053s	sys 0.055s
W20260709 12:15:39.487488 11173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.053s	sys 0.055s
I20260709 12:15:39.436664 18971 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:39.376817 19029 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:39.463763 11176 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:39.408113 19011 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:39.457868 18652 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:39.410627 19046 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:39.477692 18521 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:39.474686 11179 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:15:39.476322 11182 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:39.439327 18866 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:39.442569 18967 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27.
I20260709 12:15:39.469878 18805 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:39.493984 11176 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:39.538677 11176 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:39.535511 11322 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:39.468138 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.488787 18642 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:39.541265 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 73225.9us
I20260709 12:15:39.541491 11181 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:15:39.542189 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.490569 18971 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:39.493022 19029 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.498943 18941 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:39.549078 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:39.499524 18520 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:39.587496 18845 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:39.538184 19031 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:39.582547 11176 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.551044 19032 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:39.554828 18967 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:39.577531 19029 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:39.605430 19067 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:15:39.395152 11073 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 17 to 18, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:39.614655 11221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.004s	sys 0.022s
I20260709 12:15:39.589622 18866 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.593292 18845 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:39.615154 11221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.004s	sys 0.022s
I20260709 12:15:39.634064 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:39.639470 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 3020) took 1306 ms. Trace:
I20260709 12:15:39.640003 11322 rpcz_store.cc:276] 0709 12:15:38.332635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:38.578846 (+246211us) service_pool.cc:224] Handling call
0709 12:15:39.637590 (+1058744us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54000e6854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:39.639389 (+  1799us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2197915008}
I20260709 12:15:39.551302 18642 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:39.642786 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.604295 11176 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:39.596903 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.656205 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1416) took 1263 ms. Trace:
W20260709 12:15:39.656786 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 60015.7us
I20260709 12:15:39.656610 11930 rpcz_store.cc:276] 0709 12:15:38.392730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.061428 (+668698us) service_pool.cc:224] Handling call
0709 12:15:39.067449 (+  6021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:39.435562 (+368113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.655932 (+220370us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54006301d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:39.656141 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758447872}
I20260709 12:15:39.657694 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:39.632062 18845 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:39.683960 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.011s	sys 0.025s
W20260709 12:15:39.684330 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.011s	sys 0.025s
W20260709 12:15:39.684859 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 615225us
I20260709 12:15:39.685685 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:39.663409 19039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.019s	sys 0.014s
W20260709 12:15:39.683633 18936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.013s	sys 0.023s
I20260709 12:15:39.693727 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:39.663530 18967 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:39.684737 18944 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:39.697108 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.639022 18866 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:39.702277 11174 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.014s	sys 0.035s
W20260709 12:15:39.705202 11174 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.014s	sys 0.035s
I20260709 12:15:39.697355 19031 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:15:39.694957 19039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.019s	sys 0.014s
I20260709 12:15:39.713107 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:39.702013 18956 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.003s	sys 0.027s
I20260709 12:15:39.633246 18971 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.695472 19032 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.713799 19031 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.705701 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.700347 18944 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.730826 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:39.714170 18990 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:15:39.717340 18956 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.004s	sys 0.027s
I20260709 12:15:39.740996 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:39.699868 18520 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:15:39.717324 19044 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:39.750008 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:39.754930 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 424596us
I20260709 12:15:39.756662 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:39.747041 18944 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:39.714847 18967 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 12:15:39.735574 18971 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:39.726264 18978 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:39.773065 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:39.769359 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 465229us
W20260709 12:15:39.751471 19069 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:15:39.770514 11900 leader_election.cc:400] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:39.779282 11900 leader_election.cc:403] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:15:39.748862 18906 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:39.738678 19072 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:39.740985 19031 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:39.777791 19068 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:15:39.728556 19032 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:39.783980 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:39.784880 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.027s	sys 0.026s
W20260709 12:15:39.802112 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.027s	sys 0.026s
W20260709 12:15:39.802485 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 518206us
W20260709 12:15:39.696050 18936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.015s	sys 0.023s
I20260709 12:15:39.766371 18940 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:39.771447 18713 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:39.782100 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.768764 19073 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:39.812125 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.779506 18967 consensus_queue.cc:1243] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 20.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:15:39.785796 19049 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:39.786389 18643 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:39.776605 18861 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
W20260709 12:15:39.800808 19065 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:15:39.827265 11322 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:39.828373 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.837119 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.823413 18713 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.827026 19073 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.787407 18797 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:39.823673 11325 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: []
I20260709 12:15:39.832377 18977 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:39.821655 18978 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:39.837944 11327 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:39.790483 18972 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:39.714823 11073 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 11 to 18, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 18 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:39.840947 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73816.4us
I20260709 12:15:39.847774 11901 leader_election.cc:400] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
W20260709 12:15:39.847323 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.833694 19090 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:15:39.863183 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 164194us
I20260709 12:15:39.868714 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.835875 19049 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:39.832566 18643 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:39.847378 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67173.1us
W20260709 12:15:39.846453 18975 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.007s
I20260709 12:15:39.840197 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:39.831099 19039 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:15:39.841612 18713 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:39.840142 18941 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:15:39.858836 19066 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:15:39.840710 18520 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:39.854753 19022 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:39.831789 19072 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:15:39.834271 18967 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:15:39.899761 19083 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:15:39.832751 18936 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:15:39.885995 11176 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:39.854513 18978 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:39.844419 18977 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:39.888325 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 66924.7us
I20260709 12:15:39.846879 19073 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:39.857730 18642 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:39.895036 11177 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:15:39.881038 18975 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.006s	sys 0.007s
I20260709 12:15:39.881332 18643 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:39.912029 19070 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:15:39.906306 18866 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:39.937544 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:39.897657 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 115628us
W20260709 12:15:39.954424 19084 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:15:39.957221 18969 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.550s	user 0.007s	sys 0.003s
I20260709 12:15:39.958612 18998 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:15:39.959559 19075 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:15:39.838162 18990 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:15:39.891661 19087 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:39.963838 19049 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.882799 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:39.982410 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3369) took 1026 ms. Trace:
I20260709 12:15:39.993479 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:40.006898 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:40.037959 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 159836us
W20260709 12:15:39.957221 18816 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.008s
I20260709 12:15:40.040952 11257 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:40.041405 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.048866 11253 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: [21.10-21.10]
W20260709 12:15:40.073055 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 204410us
I20260709 12:15:39.999279 18967 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.050449 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.962703 18972 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.908103 18940 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:39.963593 19022 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.992096 18978 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:39.992590 18977 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:40.062649 18956 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:40.069236 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 206591us
I20260709 12:15:39.931524 18797 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.049046 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.074851 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.998106 19090 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:40.078306 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:40.115854 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:39.992843 18861 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.090544 11254 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:15:40.071722 11174 rpcz_store.cc:276] 0709 12:15:38.955829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.030105 (+ 74276us) service_pool.cc:224] Handling call
0709 12:15:39.061800 (+ 31695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:39.062025 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:15:39.707701 (+645676us) log.cc:844] Serialized 14 byte log entry
0709 12:15:39.981832 (+274131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:39.981911 (+    79us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:39.981973 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.982174 (+   201us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:39.982335 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":774207,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":141249,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":50641,"thread_start_us":898135,"threads_started":3,"wal-append.queue_time_us":265686}
I20260709 12:15:40.095259 11257 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 12:15:40.065186 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.118739 11255 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:40.061708 18862 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.119477 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.111220 11253 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
I20260709 12:15:40.120524 11253 pending_rounds.cc:85] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 17
I20260709 12:15:40.120818 11253 pending_rounds.cc:107] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:15:40.121097 11253 raft_consensus.cc:2889] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:40.070526 19088 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:40.112915 11249 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
I20260709 12:15:40.083310 19095 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.061491 18969 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.124323 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.091645 18520 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:40.131633 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:40.104429 19079 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 12:15:40.113193 19006 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:15:40.138399 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 63635.8us
I20260709 12:15:40.167579 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:40.137524 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:40.146270 18816 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.150779 11323 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:40.091938 18972 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.156489 19074 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.157171 18642 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:40.158881 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:40.159480 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:40.159667 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:40.124425 19077 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.265s	user 0.001s	sys 0.001s
I20260709 12:15:40.163476 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:40.097098 19097 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:15:40.150357 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:40.125905 19088 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:40.195631 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:40.199348 11323 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:15:40.200284 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:40.148175 19099 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:15:40.093364 18967 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.103065 19022 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:40.168869 11252 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:40.167090 19006 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:40.194319 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:40.198309 19046 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.658s	user 0.014s	sys 0.001s
I20260709 12:15:40.147171 18862 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:15:40.196326 11327 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:40.196345 11324 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:40.201251 11147 connection.cc:256] RPC call timeout handler was delayed by -1.2356568389998301s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:15:40.202008 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:40.223184 11934 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:15:40.223593 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:40.162580 19078 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:15:40.208065 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.149344 19102 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:40.215391 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:40.216441 11927 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: [19.17-19.17]
I20260709 12:15:40.216972 11928 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:40.138342 18866 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:40.217798 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.214742 11330 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:40.217231 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.194537 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1287) took 1475 ms. Trace:
I20260709 12:15:40.188422 19032 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.133903 18861 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.216960 18956 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:40.228226 11173 rpcz_store.cc:276] 0709 12:15:38.718730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:38.749349 (+ 30619us) service_pool.cc:224] Handling call
0709 12:15:38.749728 (+   379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:38.882520 (+132792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:38.882554 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:38.883228 (+   674us) log.cc:844] Serialized 52 byte log entry
0709 12:15:39.488263 (+605035us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:39.488308 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.638709 (+150401us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54002e54d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:39.638777 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:40.179296 (+540519us) raft_consensus.cc:1685] finished
0709 12:15:40.179320 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.194463 (+ 15143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12634,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":304384384,"thread_start_us":604542,"threads_started":1,"wal-append.queue_time_us":810303}
I20260709 12:15:40.229761 19090 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.231163 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1417) took 1838 ms. Trace:
I20260709 12:15:40.231765 18967 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.232800 18966 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:15:40.233562 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1507) took 1221 ms. Trace:
W20260709 12:15:40.238202 19104 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:15:40.241991 19110 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:40.242491 18960 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.243217 18885 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:40.244398 19088 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.250100 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3401) took 1662 ms. Trace:
W20260709 12:15:40.304008 18971 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
W20260709 12:15:40.267284 18805 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.002s
W20260709 12:15:40.273756 19096 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:15:40.274089 19041 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:40.224860 19107 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:15:40.322070 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.275933 18972 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.208328 19080 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.196122 18816 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.225674 19046 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.281890 19097 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:15:40.282897 19082 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:15:40.283308 18265 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.932s	user 0.004s	sys 0.007s
I20260709 12:15:40.349279 18265 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.349939 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.351595 18265 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.283087 19109 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:15:40.204806 19074 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.219174 18936 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:40.353207 18936 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:40.353771 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2761) took 1046 ms. Trace:
I20260709 12:15:40.354193 11178 rpcz_store.cc:276] 0709 12:15:39.307211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.330501 (+ 23290us) service_pool.cc:224] Handling call
0709 12:15:40.353491 (+1022990us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54001f3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:40.353709 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604217600}
I20260709 12:15:40.354223 18936 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.354350 19074 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.147804 19095 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.357836 18816 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.287015 19006 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.288100 18969 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.287021 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1248) took 1313 ms. Trace:
I20260709 12:15:40.347373 18885 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:40.358841 11926 rpcz_store.cc:276] 0709 12:15:38.973818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.101267 (+127449us) service_pool.cc:224] Handling call
0709 12:15:39.101605 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:39.204691 (+103086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:39.204733 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:39.205133 (+   400us) log.cc:844] Serialized 26 byte log entry
0709 12:15:39.704029 (+498896us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:39.726034 (+ 22005us) log.cc:844] Serialized 14 byte log entry
0709 12:15:39.726241 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.726317 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:40.286784 (+560467us) raft_consensus.cc:1685] finished
0709 12:15:40.286806 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.286960 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114772,"prepare.queue_time_us":45538,"raft.queue_time_us":112350,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":32382080,"thread_start_us":498485,"threads_started":1,"wal-append.queue_time_us":614078}
W20260709 12:15:40.289317 18845 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.000s
I20260709 12:15:40.359892 18885 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.360968 18845 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:40.295655 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:40.299642 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 163861us
I20260709 12:15:40.364169 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.303256 11329 rpcz_store.cc:276] 0709 12:15:39.012589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.217781 (+205192us) service_pool.cc:224] Handling call
0709 12:15:39.218069 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:39.218619 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:39.218646 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:39.219114 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:15:39.713076 (+493962us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:39.713116 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.713212 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:40.233331 (+520119us) raft_consensus.cc:1685] finished
0709 12:15:40.233356 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.233499 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":493607,"threads_started":1,"wal-append.queue_time_us":975562}
W20260709 12:15:40.303323 19060 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.947s	user 0.009s	sys 0.004s
W20260709 12:15:40.303229 19029 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.669s	user 0.006s	sys 0.005s
I20260709 12:15:40.314132 11326 rpcz_store.cc:276] 0709 12:15:38.587377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.210920 (+623543us) service_pool.cc:224] Handling call
0709 12:15:39.211252 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:39.211579 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:39.211602 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:39.212054 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:15:39.655713 (+443659us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:39.655758 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.655831 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:40.249873 (+594042us) raft_consensus.cc:1685] finished
0709 12:15:40.249899 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.250037 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":443243,"threads_started":1,"wal-append.queue_time_us":567271}
I20260709 12:15:40.060961 11073 catalog_manager.cc:5697] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 8 to 13, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 13 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:40.367568 19060 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.367815 19029 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.310271 19108 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.369894 19060 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:40.370175 19108 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.371512 18845 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.372205 11328 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:40.372068 18652 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.372802 19108 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.319715 19088 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:40.373595 18652 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:40.374580 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3016) took 2043 ms. Trace:
I20260709 12:15:40.324431 19041 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.297945 11931 rpcz_store.cc:276] 0709 12:15:38.393030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.064666 (+671636us) service_pool.cc:224] Handling call
0709 12:15:39.092215 (+ 27549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:39.203608 (+111393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:39.203639 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:39.204098 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:15:39.688066 (+483968us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:39.716393 (+ 28327us) log.cc:844] Serialized 14 byte log entry
0709 12:15:39.716608 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.718078 (+  1470us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:40.230934 (+512856us) raft_consensus.cc:1685] finished
0709 12:15:40.230957 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.231100 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111027,"prepare.queue_time_us":37530,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":121552,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":7799,"thread_start_us":483541,"threads_started":1,"wal-append.queue_time_us":583436}
I20260709 12:15:40.380095 19041 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.325230 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.326017 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.374951 11328 rpcz_store.cc:276] 0709 12:15:38.331650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.199480 (+867830us) service_pool.cc:224] Handling call
0709 12:15:39.199841 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:40.372678 (+1172837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:40.374349 (+  1671us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003228d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:40.374521 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2695748352}
I20260709 12:15:40.328614 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.331902 19072 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:40.323957 18956 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:40.395617 18521 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.854s	user 0.010s	sys 0.001s
I20260709 12:15:40.395277 18956 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.396205 18521 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.345216 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.399639 18861 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:40.400588 18964 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.908s	user 0.013s	sys 0.000s
I20260709 12:15:40.401140 18964 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:40.346287 19097 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:40.402294 11182 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:40.402477 19097 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.403924 19097 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:40.273382 18713 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.346199 19080 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.406178 19095 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.369066 11073 catalog_manager.cc:5697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 11 to 12, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 12 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:40.319813 18805 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:40.311125 18989 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.933s	user 0.012s	sys 0.000s
I20260709 12:15:40.323740 18967 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.383924 11181 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:40.408644 19080 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:40.411108 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 86036.3us
I20260709 12:15:40.434861 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.436484 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:40.442732 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 559955us
I20260709 12:15:40.437461 11178 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:15:40.447606 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:40.456806 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 92708.3us
I20260709 12:15:40.456831 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.458453 19060 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:40.461758 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:40.464459 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.408512 18713 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.459808 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:40.468047 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.474570 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:40.475689 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.460559 19060 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:40.477332 19060 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:40.479722 19022 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.465570 18861 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:40.460347 18967 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.482097 18861 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:40.482544 18861 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:40.484120 18967 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.484730 19044 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:15:40.485591 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.486203 18816 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:40.486441 18971 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:40.487010 18971 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.487447 18971 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:40.487908 18977 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.464638 19113 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:15:40.490473 18521 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:40.456007 18989 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.491060 11073 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:40.492789 18989 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:40.465724 18990 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:40.494366 11330 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:15:40.489749 19115 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:40.495492 11177 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
W20260709 12:15:40.465363 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 257854us
W20260709 12:15:40.496519 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.497107 11331 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:40.462456 19074 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:40.470013 19031 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.000s
W20260709 12:15:40.512038 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 44060.5us
I20260709 12:15:40.477382 18805 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:40.535553 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 100855us
I20260709 12:15:40.536300 19029 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:40.495532 19011 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.997s	user 0.014s	sys 0.000s
I20260709 12:15:40.537869 19108 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:40.530076 19039 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:40.465101 19087 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:40.542529 18944 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.000s
I20260709 12:15:40.562040 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:40.563493 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:40.564272 11325 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:40.474140 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:40.470337 18520 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:40.495128 19080 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:40.551184 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.552256 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.607362 11326 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:40.608074 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:40.608330 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.553288 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:40.609026 11256 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:40.552480 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:40.556425 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:40.556645 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:40.500677 18990 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:40.544947 19100 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.613306 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.544061 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:40.609387 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:40.611753 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 59403.2us
I20260709 12:15:40.609488 11328 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:15:40.612442 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 138429us
I20260709 12:15:40.616604 11900 leader_election.cc:400] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21.
I20260709 12:15:40.616506 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.612632 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:40.617787 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 73821.3us
I20260709 12:15:40.559630 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:40.620335 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60784.1us
I20260709 12:15:40.621618 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.548933 19108 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.549147 19039 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.666117 19115 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:40.671026 19039 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:40.667119 19100 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.672770 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53872 (request call id 10) took 1337 ms. Trace:
I20260709 12:15:40.673149 11175 rpcz_store.cc:276] 0709 12:15:39.335694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.417388 (+ 81694us) service_pool.cc:224] Handling call
0709 12:15:40.672465 (+1255077us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54002e3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f2735 00007f3ec66ea9fa 00007f3ecb6b24c9 00007f3ec4c2ec54 00007f3ec4c2ebe2 00007f3ec4c2eb11 00007f3ec4c2ea8c 00007f3ec4c2dd42 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:40.672684 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151707392}
W20260709 12:15:40.673923 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 55446.5us
I20260709 12:15:40.674350 19100 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:40.669021 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 52580.5us
I20260709 12:15:40.675866 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.677687 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.679384 19108 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:40.679034 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.681335 18940 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:40.681754 18940 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.682113 18940 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:40.682653 18520 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:40.684450 18520 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:40.560658 19022 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:40.690092 18940 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.607013 18944 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.693315 11178 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:40.669678 11900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.826s (SENT) [suppressed 5 similar messages]
I20260709 12:15:40.693008 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:40.698122 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 76540.4us
I20260709 12:15:40.699867 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:40.699918 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.669241 11255 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:40.546445 19088 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.078s
I20260709 12:15:40.703670 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:40.703733 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.704555 11325 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:40.708920 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.709774 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:40.678850 19090 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:40.710803 18845 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:40.710898 19121 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:40.636970 19118 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:40.711731 19118 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:40.712031 18940 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.712173 19118 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:40.545011 19074 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.547488 19031 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.607515 18967 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.617007 18990 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:40.710163 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:40.742069 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 64416.3us
I20260709 12:15:40.757242 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:40.554987 18713 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.124s
I20260709 12:15:40.619390 18797 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:40.553587 19011 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.624234 19029 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:40.706828 18987 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:15:40.825122 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 66859.1us
I20260709 12:15:40.832939 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:40.696959 19115 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:15:40.758678 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 89073.1us
I20260709 12:15:40.748744 19098 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:40.759102 18940 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:40.841622 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:40.492815 11073 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 16 to 18, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 18 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:40.784336 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:40.874567 11249 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:40.678522 19113 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:40.821875 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.741008 19022 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:40.823277 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:40.880780 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 57602.5us
I20260709 12:15:40.830835 19029 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.825757 11248 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:40.826736 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.884397 11257 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:40.760668 18967 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.825608 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:40.829891 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 220585us
I20260709 12:15:40.835922 18987 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.678395 19040 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:40.834030 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.834542 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:40.836131 11325 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:40.750908 18977 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:40.833580 11927 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:40.840889 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:40.759009 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49400.3us
I20260709 12:15:40.835594 18981 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:40.827921 18797 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.870743 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.749941 19101 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.133s
I20260709 12:15:40.808497 18845 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:40.844774 11926 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:40.881645 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:40.712196 18944 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.713889 18804 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:40.933938 18804 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:40.934962 18804 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.936124 18804 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:40.887356 18885 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:40.937860 18885 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.938344 18885 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:40.888763 18966 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:40.942133 18966 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:40.943136 18966 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.889451 18862 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:40.944334 18862 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:40.945363 18862 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.946683 18862 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:40.895259 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:40.898448 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:40.950964 11177 raft_consensus.cc:2368] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:40.951900 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.953008 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:40.906224 19022 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:40.953689 18972 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.954300 18972 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.954818 19022 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.899869 11249 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:40.900996 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:40.960386 11257 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
W20260709 12:15:40.694118 18643 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
I20260709 12:15:40.843617 18816 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.962414 18816 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:40.873973 19121 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:40.920379 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 86425.2us
I20260709 12:15:40.923359 18981 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:40.964938 18981 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:40.965114 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:40.966573 11222 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.967242 18520 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:40.969202 18520 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:40.881314 19113 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.970103 19113 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:40.917052 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:40.928646 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:40.971355 11179 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:40.971750 19121 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.972325 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:40.973173 11179 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:40.972851 19060 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:40.912034 18978 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.002s
I20260709 12:15:40.974009 19121 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:40.973929 18978 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.976286 18978 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
W20260709 12:15:40.977348 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 60369.1us
I20260709 12:15:40.978123 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:40.978986 19011 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.979130 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:40.980036 19011 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:40.980371 11179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:40.980582 19011 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:40.981050 11179 raft_consensus.cc:606] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:40.981168 19011 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:40.913843 18987 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:40.961454 18643 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.969245 19031 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.920336 11223 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:41.043318 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 123094us
I20260709 12:15:40.982679 11179 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:40.987377 18967 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:40.904201 19029 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:40.987349 18981 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:40.988768 18977 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:40.991745 11926 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:40.993166 19115 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:40.994773 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:40.995702 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.996403 19121 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.000139 11927 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:15:41.001001 19095 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:41.002592 19073 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.962s	user 0.009s	sys 0.002s
I20260709 12:15:41.003631 19072 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:41.003861 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:40.929970 18797 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:41.006568 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1300) took 1668 ms. Trace:
I20260709 12:15:41.012765 19006 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:41.055349 18977 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:41.014226 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 132680us
I20260709 12:15:41.027418 11176 raft_consensus.cc:1275] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:15:41.038108 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:41.047883 11179 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:15:40.926626 19126 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:41.093467 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:41.096074 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:41.097852 11222 leader_election.cc:400] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:41.098368 11222 leader_election.cc:403] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:15:41.041039 18652 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.667s	user 0.002s	sys 0.009s
I20260709 12:15:41.049908 11223 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:41.042686 19097 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.000s
W20260709 12:15:41.058300 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.057796 11931 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:41.058210 11256 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:41.058828 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.040266 18964 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:41.062155 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:41.051913 19131 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:15:41.061832 19049 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:41.070043 11932 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:41.045185 18885 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.135373 11180 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
W20260709 12:15:41.070173 18971 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.001s
I20260709 12:15:41.147607 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.148824 11249 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:41.149773 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.072539 11182 rpcz_store.cc:276] 0709 12:15:39.337984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.554295 (+216311us) service_pool.cc:224] Handling call
0709 12:15:39.594202 (+ 39907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:40.402758 (+808556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.006330 (+603572us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54002e3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:41.006503 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857919872}
I20260709 12:15:40.831422 19087 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.078068 11935 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 19: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:41.078912 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:41.058429 18978 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:41.157186 11933 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:15:41.157260 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:41.082492 19032 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:15:41.083631 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 016411114c6d42039bd04025378dfc98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:41.088415 11356 heartbeater.cc:646] Failed to heartbeat to 127.9.198.254:36219 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.198.254:36219 timed out after 15.000s (SENT)
I20260709 12:15:41.091060 18977 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:41.073491 19073 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:41.102154 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52310.7us
I20260709 12:15:41.104094 18804 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:41.103713 19108 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:41.229398 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:41.189971 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 106518us
I20260709 12:15:41.230135 11254 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:41.104736 19040 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:41.109681 18805 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.111063 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3033) took 1972 ms. Trace:
I20260709 12:15:41.112047 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:41.112739 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:41.113292 18972 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:41.113435 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:41.114188 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:41.116127 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:41.116843 19074 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:41.124672 11934 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:41.073814 19072 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:41.132095 19100 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.132332 19021 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.131814 18956 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:15:41.137303 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:41.138415 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 1206) took 1145 ms. Trace:
I20260709 12:15:41.147874 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3420) took 1015 ms. Trace:
I20260709 12:15:41.048804 18642 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:41.262274 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:41.177279 18816 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:41.074571 11928 raft_consensus.cc:2393] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:41.153005 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74160.9us
I20260709 12:15:41.150660 11255 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:41.127424 18941 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.168277 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.129148 19049 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:41.068058 19121 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.166388 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1295) took 1829 ms. Trace:
W20260709 12:15:41.173988 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53576.1us
I20260709 12:15:41.190579 11326 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:41.294727 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:41.299396 11254 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
I20260709 12:15:41.300796 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.120852 19031 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:41.224581 18804 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.225581 11325 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:41.170925 18978 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.157883 19091 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24.
I20260709 12:15:41.236709 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:41.240732 11176 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:41.317520 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:41.318969 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:41.162847 19097 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:41.243310 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.246265 11327 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:41.248124 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:41.248447 11251 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:41.239373 11331 rpcz_store.cc:276] 0709 12:15:39.138798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.303788 (+164990us) service_pool.cc:224] Handling call
0709 12:15:39.304075 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:40.529229 (+1225154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.110836 (+581607us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400459cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:41.111004 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2743117568}
I20260709 12:15:41.252218 11173 raft_consensus.cc:2368] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:41.163097 19032 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:41.252436 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:41.252898 11322 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:41.327515 11934 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:41.257911 11330 rpcz_store.cc:276] 0709 12:15:40.132204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.227374 (+ 95170us) service_pool.cc:224] Handling call
0709 12:15:40.234232 (+  6858us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:40.527763 (+293531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.147544 (+619781us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400331dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:41.147798 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597580672}
I20260709 12:15:41.191548 19041 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.263044 11177 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:41.150988 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53479.8us
I20260709 12:15:41.257151 11248 rpcz_store.cc:276] 0709 12:15:39.992977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.993195 (+   218us) service_pool.cc:224] Handling call
0709 12:15:41.138211 (+1145016us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:41.138335 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1729812736}
I20260709 12:15:41.271731 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.287418 11180 rpcz_store.cc:276] 0709 12:15:39.336797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.487235 (+150438us) service_pool.cc:224] Handling call
0709 12:15:39.497550 (+ 10315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:41.135882 (+1638332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.158919 (+ 23037us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54002b7d54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:41.166306 (+  7387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3765635968}
I20260709 12:15:41.303438 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:41.309087 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:41.253062 19040 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:41.338050 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:41.339066 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:41.321203 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.325008 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:41.160816 18971 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:41.332150 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 84089.7us
I20260709 12:15:41.270159 18642 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.347580 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:41.336696 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:41.311673 18804 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:41.288365 19006 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:41.332387 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.342291 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:41.343519 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.348114 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:41.359902 11254 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:41.362910 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:41.364401 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:41.350848 11324 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:41.252933 18805 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:41.305306 19145 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:41.335369 19060 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:41.336799 19097 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.368433 11180 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:41.375682 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:41.352921 18642 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:41.350538 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:41.345909 18989 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:41.354892 19090 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.349577 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:41.269863 19100 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.279289 19021 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.286507 18941 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:41.346549 19123 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:41.376322 11254 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:41.369300 11248 raft_consensus.cc:2368] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:41.406709 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:41.407516 11248 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:41.380057 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.356154 18972 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:41.425855 18972 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.426339 18972 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:41.355454 19098 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.355894 19006 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:41.366552 18643 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:41.396742 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47212.4us
I20260709 12:15:41.370816 18936 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:41.346041 19137 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:41.402429 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.383949 19097 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:41.408176 19123 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.347524 19041 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.374307 18981 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:41.407063 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:41.422653 19140 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 12:15:41.422868 18978 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:41.376782 19145 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:41.363013 18862 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:41.385811 18652 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:41.365725 19128 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:15:41.430939 11926 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:15:41.445466 18936 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.441110 11248 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:41.404565 18816 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:41.408916 18989 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:41.453651 11180 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:15:41.471385 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:41.471781 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:41.402177 19060 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:15:41.492676 11254 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
W20260709 12:15:41.627681 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 156001us
I20260709 12:15:41.628294 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:41.473405 19123 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:41.633617 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:41.394398 19032 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:41.402869 19090 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:41.472160 18861 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.989s	user 0.012s	sys 0.002s
W20260709 12:15:41.440290 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.573772 11257 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:41.696429 18861 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.480089 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:15:41.574699 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.402912 19049 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.479674 18966 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:41.639971 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:41.707928 11248 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:15:41.625870 18936 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:41.638665 18956 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:41.629171 19006 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:41.482429 19101 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.164s
W20260709 12:15:41.649655 19132 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.813s	user 0.010s	sys 0.000s
W20260709 12:15:41.652122 19039 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.980s	user 0.013s	sys 0.000s
I20260709 12:15:41.499166 18643 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.153s
I20260709 12:15:41.660837 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3425) took 1116 ms. Trace:
I20260709 12:15:41.630631 19108 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:41.641856 18520 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:41.623183 18816 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.668102 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:41.668365 19011 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.000s
W20260709 12:15:41.644954 19118 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.900s	user 0.011s	sys 0.000s
I20260709 12:15:41.643075 19072 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.669775 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:41.671233 19157 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.671877 18862 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:41.878027 18862 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.881090 19126 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.675021 18845 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:41.883082 19072 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:41.676458 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:41.678414 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:41.884806 11327 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:41.683583 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:41.885701 11325 raft_consensus.cc:2393] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:41.684530 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.686264 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:41.688304 19145 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:41.689320 11174 pending_rounds.cc:85] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 13
I20260709 12:15:41.890895 11174 pending_rounds.cc:107] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:15:41.891187 11174 raft_consensus.cc:2889] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:41.884433 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:41.691499 19046 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:41.435927 19103 consensus_queue.cc:1048] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 12:15:41.695194 18977 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:41.588558 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 224193us
I20260709 12:15:41.862064 19006 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:41.698009 19137 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:41.493530 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 156887us
I20260709 12:15:41.945933 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:41.947664 11147 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:41.710625 11248 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:15:41.498183 18981 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.868877 19073 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:41.720570 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 1207) took 1714 ms (client timeout 1826 ms). Trace:
W20260709 12:15:41.721951 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 293933us
W20260709 12:15:41.959306 11251 rpcz_store.cc:269] 0709 12:15:40.006232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.006519 (+   287us) service_pool.cc:224] Handling call
0709 12:15:41.720351 (+1713832us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400277c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:41.720515 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704638208}
I20260709 12:15:41.436476 19098 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.706264 19107 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.728753 11324 rpcz_store.cc:276] 0709 12:15:40.544284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.544424 (+   140us) service_pool.cc:224] Handling call
0709 12:15:40.548777 (+  4353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:41.365165 (+816388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.660492 (+295327us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b540026cad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:41.660756 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844113280}
I20260709 12:15:41.966239 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:41.709197 19091 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:41.862897 19011 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.863140 11322 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:41.735855 18643 consensus_queue.cc:1243] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:15:41.869917 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.869133 19118 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.961234 19095 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:41.743705 18975 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:41.730461 19049 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:41.734099 18885 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:41.899827 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:41.977147 19091 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:41.883504 18845 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.044584 18845 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.045957 19095 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:41.933840 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:41.933540 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:42.048473 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:42.049837 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:42.051116 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:42.051748 11325 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:41.900985 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:42.054025 11327 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:42.055102 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:42.055984 11327 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:41.943401 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:42.056948 18972 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.620s	user 0.006s	sys 0.001s
I20260709 12:15:41.988678 19107 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.058068 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:42.059674 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:42.060029 19107 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.060453 11254 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:41.944900 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:42.061802 19090 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.062027 11330 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:42.063282 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:42.064262 11330 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 19: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:42.056943 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:41.995172 18956 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.065263 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:41.725585 19123 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.949536 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:42.125103 19150 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.397s	user 0.000s	sys 0.000s
W20260709 12:15:42.126078 19156 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 12:15:42.126502 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:42.126953 19123 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:41.890286 19022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.032s	sys 0.032s
I20260709 12:15:41.435850 19040 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.129170 18977 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.128098 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:42.128510 19022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.032s	sys 0.032s
I20260709 12:15:42.132349 19091 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.133210 11325 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:42.133671 19091 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:42.133786 19095 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.066008 19100 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:42.140673 18797 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.051s	user 0.006s	sys 0.003s
I20260709 12:15:41.960930 18981 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.154443 19098 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.038568 18941 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:42.057462 18972 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.162523 19118 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.163548 18972 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:42.165094 19115 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.100s	user 0.008s	sys 0.005s
I20260709 12:15:42.167644 18956 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.170219 18956 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.173152 19011 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:42.174525 19155 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:15:42.176374 18936 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:42.178097 19029 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.120s	user 0.000s	sys 0.012s
I20260709 12:15:41.737015 19039 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.180111 19039 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:42.180955 18804 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.811s	user 0.008s	sys 0.000s
I20260709 12:15:41.962316 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3422) took 1707 ms. Trace:
I20260709 12:15:42.198381 11329 rpcz_store.cc:276] 0709 12:15:40.255207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.365121 (+109914us) service_pool.cc:224] Handling call
0709 12:15:40.365435 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:40.365840 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:40.365867 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:40.377966 (+ 12099us) log.cc:844] Serialized 26 byte log entry
0709 12:15:40.461854 (+ 83888us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:40.461905 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:40.461986 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:41.648993 (+1187007us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:41.962018 (+313025us) raft_consensus.cc:1685] finished
0709 12:15:41.962042 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:41.962255 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4900,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":5952256,"thread_start_us":81180,"threads_started":1,"wal-append.queue_time_us":288113}
I20260709 12:15:41.890563 19120 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:41.967097 11178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.011s	sys 0.000s
I20260709 12:15:41.906744 18805 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:41.969696 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:42.215806 11251 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:41.881894 19126 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.845291 18520 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.975759 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:41.781411 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:42.219293 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:42.219103 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 243422us
I20260709 12:15:42.220588 11296 leader_election.cc:400] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:42.220961 11296 leader_election.cc:403] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:15:41.982518 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:42.221369 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:41.982986 11326 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:41.983315 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:42.223089 11176 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:42.223459 19011 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:42.223946 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:41.986987 11173 raft_consensus.cc:1275] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:41.888338 19157 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:41.953387 19128 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:42.005339 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.012724 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:42.228680 11251 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:15:42.039070 11328 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
W20260709 12:15:41.915825 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.988396 19153 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.040078 18975 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.042297 19041 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.068333 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:42.087620 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.190091 19115 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:42.243129 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:42.243294 11294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.000s	sys 0.013s
W20260709 12:15:42.227928 18987 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.166s	user 0.011s	sys 0.001s
W20260709 12:15:42.124286 19149 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:15:42.178177 11327 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:15:42.015223 19121 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:41.958253 19137 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.180881 19029 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.181833 19090 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:41.961736 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.007s	sys 0.007s
I20260709 12:15:42.196100 11325 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:42.284165 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2745) took 1108 ms. Trace:
W20260709 12:15:42.202517 11178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.011s	sys 0.000s
I20260709 12:15:41.887120 18816 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:42.207226 11322 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:42.289994 11322 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:42.207875 11181 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.008s	sys 0.006s
I20260709 12:15:42.222085 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:42.222458 11933 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:15:42.222973 19118 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:42.291887 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:42.292210 19118 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:42.292747 19118 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:42.293063 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:42.294291 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:42.295997 11177 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 24. Current term is 25. Ops: []
W20260709 12:15:41.987233 18990 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.180s	user 0.009s	sys 0.002s
I20260709 12:15:42.013445 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:42.229998 11251 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:42.298664 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1230) took 1045 ms. Trace:
I20260709 12:15:42.298393 18990 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.299057 11251 rpcz_store.cc:276] 0709 12:15:41.253042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.229519 (+976477us) service_pool.cc:224] Handling call
0709 12:15:42.229813 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.298443 (+ 68630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.298617 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:42.300464 11251 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:15:42.300542 11175 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:15:42.301110 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2747) took 1037 ms. Trace:
I20260709 12:15:42.231875 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:42.159704 18981 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.233975 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:42.234454 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:42.302635 11250 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:42.236395 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:42.218954 18804 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.234022 11180 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:42.237459 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 363203us
W20260709 12:15:41.873773 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.311316 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:42.312297 11928 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:42.312883 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.141197 18797 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.246084 11254 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:42.245244 19146 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:15:42.250701 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:42.250732 11328 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:15:42.243763 11294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.000s	sys 0.013s
I20260709 12:15:42.251726 11330 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
W20260709 12:15:42.322806 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:42.324596 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 456806us
I20260709 12:15:42.272221 11326 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:42.274425 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1229) took 1034 ms. Trace:
I20260709 12:15:42.327153 11248 rpcz_store.cc:276] 0709 12:15:41.239766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.130547 (+890781us) service_pool.cc:224] Handling call
0709 12:15:42.130911 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.236293 (+105382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:42.236322 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:42.236364 (+    42us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:42.236398 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.274193 (+ 37795us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:42.274241 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.274369 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241899776}
I20260709 12:15:42.276971 19021 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.330988 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:42.335052 11931 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:15:42.280176 11331 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:42.284547 11256 rpcz_store.cc:276] 0709 12:15:41.176281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.182183 (+  5902us) service_pool.cc:224] Handling call
0709 12:15:41.182649 (+   466us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:41.183124 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.183173 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:41.183913 (+   740us) log.cc:844] Serialized 52 byte log entry
0709 12:15:41.553908 (+369995us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:41.553970 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.554090 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.283960 (+729870us) raft_consensus.cc:1685] finished
0709 12:15:42.283980 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.284110 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7717,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8294,"thread_start_us":291903,"threads_started":1,"wal-append.queue_time_us":495643}
W20260709 12:15:42.297945 11181 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.008s	sys 0.006s
I20260709 12:15:42.236675 11926 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:42.304342 11329 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:15:42.301455 11251 rpcz_store.cc:276] 0709 12:15:41.263658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.300011 (+1036353us) service_pool.cc:224] Handling call
0709 12:15:42.300253 (+   242us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:42.300859 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.301057 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:42.316992 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:42.318068 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:42.319131 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:42.307086 18642 consensus_meta.cc:287] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.906s	user 0.010s	sys 0.000s
I20260709 12:15:42.340692 11176 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:42.353582 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:42.357839 19123 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:42.224571 19120 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.360316 11249 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 27. Current term is 28. Ops: [27.18-27.18]
W20260709 12:15:42.360522 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 109917us
I20260709 12:15:42.303550 18990 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.362865 11926 raft_consensus.cc:1275] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:15:42.360869 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:42.236071 19128 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:42.354794 19113 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.384s	user 0.010s	sys 0.004s
W20260709 12:15:42.361217 18966 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.000s
I20260709 12:15:42.390475 11253 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:42.242177 19153 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.392810 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:42.405222 11926 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:42.361696 19021 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:15:42.409915 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1225) took 1221 ms. Trace:
I20260709 12:15:42.402618 19113 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.410308 11257 rpcz_store.cc:276] 0709 12:15:41.188292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.698875 (+510583us) service_pool.cc:224] Handling call
0709 12:15:41.699209 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.699447 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.699471 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.706448 (+  6977us) log.cc:844] Serialized 26 byte log entry
0709 12:15:42.198242 (+491794us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:42.198296 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.198392 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.409665 (+211273us) raft_consensus.cc:1685] finished
0709 12:15:42.409690 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.409859 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":449764,"threads_started":1,"wal-append.queue_time_us":608876}
I20260709 12:15:42.411873 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:42.412413 11257 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:15:42.366966 19121 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:42.413395 19121 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.413826 19121 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:42.413951 11179 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:42.414999 19165 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:15:42.362071 18643 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:15:42.416813 18643 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:42.417135 18643 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:42.417925 18643 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.419333 18643 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:42.420475 11257 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:42.421478 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:42.421815 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:42.422134 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1222) took 1234 ms. Trace:
I20260709 12:15:42.422545 11255 rpcz_store.cc:276] 0709 12:15:41.187436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.680678 (+493242us) service_pool.cc:224] Handling call
0709 12:15:41.681068 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:41.681435 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.681456 (+    21us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:41.682125 (+   669us) log.cc:844] Serialized 52 byte log entry
0709 12:15:41.941919 (+259794us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:41.941970 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.942042 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.421890 (+479848us) raft_consensus.cc:1685] finished
0709 12:15:42.421913 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.422070 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23938,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6056,"thread_start_us":259250,"threads_started":1,"wal-append.queue_time_us":599828}
I20260709 12:15:42.424492 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:42.425999 11255 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:42.426028 18971 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:42.426282 11250 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:15:42.426602 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3195) took 1152 ms. Trace:
I20260709 12:15:42.426988 11255 rpcz_store.cc:276] 0709 12:15:41.274681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.425510 (+1150829us) service_pool.cc:224] Handling call
0709 12:15:42.425802 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.426377 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.426554 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:42.428579 11255 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:15:42.429153 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3196) took 1154 ms. Trace:
I20260709 12:15:42.429621 11255 rpcz_store.cc:276] 0709 12:15:41.274975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.427929 (+1152954us) service_pool.cc:224] Handling call
0709 12:15:42.428205 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.428932 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.429106 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:42.430560 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:42.394735 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:42.286185 19022 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 12:15:42.429016 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:42.367192 11928 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:42.283123 18987 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.346596 19098 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:42.307348 19090 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.354071 18977 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.061s
I20260709 12:15:42.330323 19006 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.436513 11255 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:15:42.436868 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.126s	sys 0.159s
I20260709 12:15:42.437083 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:42.366621 19103 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:15:42.438042 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:42.439679 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:42.441751 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:42.441987 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47331.1us
I20260709 12:15:42.443410 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:42.435366 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.447060 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:42.422946 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:42.354307 18805 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:15:42.347956 18713 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 12:15:42.363466 18642 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.390679 18866 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:15:42.399941 19039 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 12:15:42.400514 19073 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:15:42.406441 19145 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:15:42.669483 11257 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:42.536803 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:42.669822 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:42.544035 11324 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:42.451043 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:42.671499 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:42.570748 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3043) took 1199 ms. Trace:
I20260709 12:15:42.672744 11327 rpcz_store.cc:276] 0709 12:15:41.371328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.124509 (+753181us) service_pool.cc:224] Handling call
0709 12:15:42.124924 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:42.267434 (+142510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.570412 (+302978us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b54001c3ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:42.570673 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121876224}
W20260709 12:15:42.602303 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.484659 11322 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:42.581544 11174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.027s
W20260709 12:15:42.675036 11174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.002s	sys 0.037s
W20260709 12:15:42.550331 19163 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
I20260709 12:15:42.674233 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:42.676806 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.126s	sys 0.190s
I20260709 12:15:42.563175 11926 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:15:42.670528 11933 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:42.432266 19041 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.673349 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:42.693573 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 251912us
W20260709 12:15:42.685909 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 248883us
I20260709 12:15:42.726119 11148 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:42.450719 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:42.725768 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:42.431977 18956 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:42.738698 11255 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:42.440560 19113 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.446403 19046 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:42.573577 19097 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.088s	user 0.008s	sys 0.000s
I20260709 12:15:42.753346 19103 raft_consensus.cc:2764] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:15:42.753803 19097 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.755174 11177 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:42.581867 19049 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.541s	user 0.007s	sys 0.003s
W20260709 12:15:42.547907 19168 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:15:42.590451 19162 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.445569 19123 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:42.643532 19022 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:42.573386 19148 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 1.092s	user 0.007s	sys 0.001s
W20260709 12:15:42.664160 18265 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.015s	sys 0.016s
I20260709 12:15:42.445847 19159 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.454202 19100 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.320s
I20260709 12:15:42.730329 19174 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:42.737197 19095 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:42.752532 19039 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.774454 18966 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:42.753027 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:42.826774 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:42.756366 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.655580 19157 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.655851 18804 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.588271 19098 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:42.659045 19029 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.453212 18972 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:15:42.480355 19087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.015s	sys 0.030s
I20260709 12:15:42.549746 19177 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.438235 18971 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.305s
I20260709 12:15:42.672102 18998 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:42.672909 19137 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.727466 18713 consensus_queue.cc:1243] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer 016411114c6d42039bd04025378dfc98 log is divergent from this leader: its last log entry 19.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:15:42.883188 18971 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:42.451143 19072 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:42.883634 18971 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:42.884042 19072 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.884548 18971 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.820530 19159 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.888465 18804 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:15:42.889180 19178 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:15:42.892181 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1315) took 1774 ms. Trace:
I20260709 12:15:42.892776 11174 rpcz_store.cc:276] 0709 12:15:41.117867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.638808 (+520941us) service_pool.cc:224] Handling call
0709 12:15:41.689141 (+ 50333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.892021 (+202880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.892068 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.973617 (+ 81549us) log.cc:844] Serialized 26 byte log entry
0709 12:15:42.675685 (+702068us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:42.675751 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.675854 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.891864 (+216010us) raft_consensus.cc:1685] finished
0709 12:15:42.891899 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.892098 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":634073,"threads_started":1,"wal-append.queue_time_us":651056}
I20260709 12:15:42.869378 19098 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.898847 19098 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.486545 18936 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:42.760087 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 85975.1us
I20260709 12:15:42.730110 18797 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.685469 19108 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:42.900358 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:42.900731 19108 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.831207 19039 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.903496 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:42.900382 18797 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:42.904928 18797 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:42.888495 19173 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:15:42.906100 19029 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:42.906512 19097 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.906787 18713 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:42.908360 18713 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:42.867138 18805 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:42.908942 18713 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:42.909240 18805 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:42.909914 18797 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.910276 18805 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.911293 18797 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:42.913048 19118 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.620s	user 0.001s	sys 0.008s
I20260709 12:15:42.752586 19176 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:42.816970 11257 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:15:42.835613 19090 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.784803 19123 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:42.790483 18265 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.016s	sys 0.017s
W20260709 12:15:42.825281 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.831532 11327 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:42.832672 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3192) took 1559 ms. Trace:
I20260709 12:15:42.825362 18987 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:42.865420 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139409us
I20260709 12:15:42.748646 18956 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.742986 19041 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.733335 19006 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.812965 19126 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.869678 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2777) took 1754 ms. Trace:
I20260709 12:15:42.945868 11178 rpcz_store.cc:276] 0709 12:15:41.114776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.364740 (+249964us) service_pool.cc:224] Handling call
0709 12:15:41.378383 (+ 13643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.403084 (+ 24701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.403127 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.403635 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:15:42.285122 (+881487us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:42.285164 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.571466 (+286302us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b54003d22d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:42.571521 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.840326 (+268805us) raft_consensus.cc:1685] finished
0709 12:15:42.840357 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.869568 (+ 29211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1775,"raft.queue_time_us":11242,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":19559,"spinlock_wait_cycles":36870400,"thread_start_us":847467,"threads_started":1,"wal-append.queue_time_us":894526}
I20260709 12:15:42.821058 19174 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:42.949843 19174 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:42.950197 19095 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:15:42.872390 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 194067us
W20260709 12:15:42.875011 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 149473us
I20260709 12:15:42.951627 19095 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:42.952291 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:42.952924 11296 leader_election.cc:400] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:42.788992 19162 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.789865 19113 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:15:42.825906 19164 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:42.880478 19177 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.831321 19153 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:42.882296 18816 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.001s
I20260709 12:15:42.900118 18936 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.742461 19107 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.915048 11257 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
W20260709 12:15:42.967248 18643 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.548s	user 0.007s	sys 0.003s
I20260709 12:15:42.920922 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1258) took 1790 ms. Trace:
I20260709 12:15:42.983263 11929 rpcz_store.cc:276] 0709 12:15:41.130660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.130815 (+   155us) service_pool.cc:224] Handling call
0709 12:15:41.137002 (+  6187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.139468 (+  2466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.139508 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.142194 (+  2686us) log.cc:844] Serialized 26 byte log entry
0709 12:15:41.354487 (+212293us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:41.354528 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.354612 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.572863 (+1218251us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:42.876399 (+303536us) raft_consensus.cc:1685] finished
0709 12:15:42.876423 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.920839 (+ 44416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13995,"raft.queue_time_us":2209,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":10390,"spinlock_wait_cycles":487548160,"thread_start_us":211896,"threads_started":1,"wal-append.queue_time_us":285715}
I20260709 12:15:42.872704 18972 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.919289 19118 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.961355 19120 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.919658 19176 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:43.015146 19118 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:43.014592 19176 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.973481 19164 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.016263 19176 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:42.953905 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:43.017422 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2781) took 1028 ms. Trace:
I20260709 12:15:42.906574 19029 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:42.960477 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 509777us
I20260709 12:15:42.953619 19095 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:15:43.017920 11177 rpcz_store.cc:276] 0709 12:15:41.988704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.297076 (+308372us) service_pool.cc:224] Handling call
0709 12:15:42.297358 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.755629 (+458271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.017178 (+261549us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54002a0154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:43.017362 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052417280}
I20260709 12:15:43.018410 19029 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:42.998190 18972 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:43.019641 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:43.020713 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:43.021044 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:43.018437 19164 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.958138 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:42.924336 19185 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:43.024852 11324 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 1521) took 1571 ms (client timeout 1881 ms). Trace:
I20260709 12:15:43.024921 19185 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:42.973537 18713 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.932374 19123 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:43.025482 11324 rpcz_store.cc:269] 0709 12:15:41.453112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.965963 (+512851us) service_pool.cc:224] Handling call
0709 12:15:43.024650 (+1058687us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54003237d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:43.024793 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326398848}
I20260709 12:15:43.027575 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:42.941028 18971 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:43.038486 11324 raft_consensus.cc:2368] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26.
I20260709 12:15:42.978826 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3198) took 1019 ms. Trace:
I20260709 12:15:42.915048 11255 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:15:42.987293 18944 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:42.977995 18816 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.988008 19041 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.943851 19193 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:15:42.978142 19189 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.988294 19107 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:42.988945 18936 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:42.990537 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3187) took 2230 ms (client timeout 1912 ms). Trace:
I20260709 12:15:42.991250 19162 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.991482 18956 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:42.992090 11328 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:15:42.992944 19181 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:15:42.994966 19108 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:42.978662 19126 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.925537 11256 rpcz_store.cc:276] 0709 12:15:41.272751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.348380 (+1075629us) service_pool.cc:224] Handling call
0709 12:15:42.348816 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.349088 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:15:42.376065 (+ 26977us) log.cc:844] Serialized 14 byte log entry
0709 12:15:42.832180 (+456115us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:42.832270 (+    90us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:42.832334 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.832433 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.832597 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26949,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":1837,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":29399,"thread_start_us":439273,"threads_started":1}
I20260709 12:15:42.870752 19184 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:43.001701 19128 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.002355 19098 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:43.003296 19153 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:43.003866 19090 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:15:43.003358 19113 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:15:43.007072 19121 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.000s
I20260709 12:15:43.007534 19006 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.010541 18975 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:43.011466 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2768) took 1904 ms (client timeout 2295 ms). Trace:
I20260709 12:15:42.957813 18998 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:15:43.025275 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:43.031790 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.032397 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:43.034487 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:42.940949 19145 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:43.038985 11257 rpcz_store.cc:276] 0709 12:15:41.959257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.914553 (+955296us) service_pool.cc:224] Handling call
0709 12:15:42.914883 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.978534 (+ 63651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.978766 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:43.039337 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:43.041355 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:43.052210 11253 rpcz_store.cc:269] 0709 12:15:40.759946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.810142 (+ 50196us) service_pool.cc:224] Handling call
0709 12:15:42.990310 (+2180168us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400088ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:42.990445 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3432337792}
I20260709 12:15:43.056380 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:42.977041 19100 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:43.041885 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 88051.5us
I20260709 12:15:43.068483 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:43.076768 11223 leader_election.cc:400] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26.
I20260709 12:15:43.040206 18944 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:43.081660 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.081780 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:43.039352 18643 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:43.055439 19164 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:43.085438 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:43.166348 19182 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:15:43.090410 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 87575.7us
I20260709 12:15:43.166674 11324 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:43.167536 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:43.168128 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:43.168891 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:43.170075 19193 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:43.170447 19193 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:43.170517 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:43.170912 19193 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:43.140223 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:43.171350 19108 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:43.172199 19108 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:43.172585 19108 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.172947 19108 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:43.161319 11929 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:43.177166 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.181901 19193 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.184440 19193 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.190307 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3194) took 1916 ms. Trace:
I20260709 12:15:43.190765 11251 rpcz_store.cc:276] 0709 12:15:41.273885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.350119 (+1076234us) service_pool.cc:224] Handling call
0709 12:15:42.350539 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:42.351067 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:42.351108 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:42.361951 (+ 10843us) log.cc:844] Serialized 26 byte log entry
0709 12:15:42.751631 (+389680us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:42.751696 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.751798 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:43.190018 (+438220us) raft_consensus.cc:1685] finished
0709 12:15:43.190041 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:43.190237 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4582,"raft.queue_time_us":10959,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":26901,"thread_start_us":322652,"threads_started":1,"wal-append.queue_time_us":419795}
I20260709 12:15:43.192345 18936 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.194875 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.055331 19189 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.198992 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:43.200464 19189 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.162648 11934 raft_consensus.cc:2368] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26.
I20260709 12:15:43.163483 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:43.202111 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.162352 11223 leader_election.cc:403] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:15:43.163157 11932 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:43.202972 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 138428us
W20260709 12:15:43.161931 11179 rpcz_store.cc:269] 0709 12:15:41.107250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.115900 (+  8650us) service_pool.cc:224] Handling call
0709 12:15:43.011272 (+1895372us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002571d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:43.011408 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2681499904}
I20260709 12:15:43.164103 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:43.055716 19041 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:43.068723 19162 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.185334 19006 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.202203 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:43.141394 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:43.062340 18642 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.205097 11935 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:43.099751 18265 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:43.205905 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.169018 11324 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:43.217448 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 77378us
I20260709 12:15:43.218739 11256 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:43.101083 19121 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.230378 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:43.221444 11257 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:43.230136 11930 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:43.232409 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:43.218973 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:43.230635 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:43.231338 11327 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:43.233875 11256 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:43.234982 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:43.237761 11256 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:15:43.238556 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3201) took 1275 ms. Trace:
I20260709 12:15:43.239015 11256 rpcz_store.cc:276] 0709 12:15:41.962777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.236810 (+1274033us) service_pool.cc:224] Handling call
0709 12:15:43.237256 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:43.238265 (+  1009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.238495 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:43.240597 11256 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:43.235564 18643 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:43.228014 18998 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:43.241376 19040 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:43.242327 18643 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.235087 19121 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.243718 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3203) took 1278 ms. Trace:
I20260709 12:15:43.244454 11256 rpcz_store.cc:276] 0709 12:15:41.965144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.240005 (+1274861us) service_pool.cc:224] Handling call
0709 12:15:43.240364 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:43.243401 (+  3037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.243653 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:43.245810 18643 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.247083 19040 consensus_queue.cc:1048] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:43.233777 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:43.247431 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:43.250329 11257 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:43.252084 11257 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 12:15:43.253825 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.254263 18643 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:43.256615 18998 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.258652 11935 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:43.259011 11257 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:43.262099 19040 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:43.260500 18643 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.263690 18643 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:43.264541 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.264541 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:43.265985 19128 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:43.266502 19128 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.266582 18643 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.266993 19128 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:43.271886 18975 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.272686 18975 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.277120 18975 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.278523 11222 leader_election.cc:400] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26.
I20260709 12:15:43.280663 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:43.280857 19145 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:43.197359 19107 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:43.281365 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51104.2us
I20260709 12:15:43.367244 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:43.368953 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:43.282757 11223 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:43.283071 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 96694us
I20260709 12:15:43.084263 19126 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:43.370437 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87783.4us
I20260709 12:15:43.091341 19153 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.218191 19031 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26.
I20260709 12:15:43.222290 18940 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.293726 18944 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.225363 19162 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.295423 19091 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:15:43.299253 19141 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.871s	user 0.006s	sys 0.003s
I20260709 12:15:43.302537 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:43.393157 11326 raft_consensus.cc:2410] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:15:43.304919 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3197) took 1347 ms. Trace:
I20260709 12:15:43.281216 19006 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.098862 19113 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.308542 18816 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.087759 19098 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:43.311486 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 1220) took 2757 ms (client timeout 2137 ms). Trace:
I20260709 12:15:43.314458 18861 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.318282 19049 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:15:43.397187 11250 rpcz_store.cc:269] 0709 12:15:40.554237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.554356 (+   119us) service_pool.cc:224] Handling call
0709 12:15:43.311296 (+2756940us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54000f82d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:43.311412 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3874905856}
I20260709 12:15:43.320744 19032 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:43.380990 19146 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:43.322285 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3052) took 1899 ms. Trace:
I20260709 12:15:43.382563 19176 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:43.327003 19108 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.427420 19188 raft_consensus.cc:993] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: : Instructing follower bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:43.431141 19188 raft_consensus.cc:1081] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Signalling peer bc92f9d2c6c5477182fa33eba4368ec1 to start an election
I20260709 12:15:43.327322 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:43.327852 19185 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:43.433270 11328 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:43.388382 19193 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:43.328318 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2763) took 1354 ms. Trace:
I20260709 12:15:43.332280 19202 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.225476 19184 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.225607 19164 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:43.337212 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.338203 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:43.344527 11324 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:43.344826 11253 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:43.348135 11178 pending_rounds.cc:85] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 10
W20260709 12:15:43.346176 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 67836.9us
I20260709 12:15:43.492282 11178 pending_rounds.cc:107] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:15:43.432500 19108 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.351161 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.351265 18652 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:43.288698 19190 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:43.352124 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.284795 19196 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:43.358184 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:43.430327 19022 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.625s	user 0.011s	sys 0.000s
I20260709 12:15:43.360245 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:43.394797 19091 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:43.462502 18805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.009s	sys 0.042s
I20260709 12:15:43.361500 19095 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:43.361713 19145 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:43.363673 19143 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 1.883s	user 0.004s	sys 0.006s
I20260709 12:15:43.384126 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:43.385939 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1313) took 2268 ms. Trace:
W20260709 12:15:43.387585 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.210567 19100 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.390642 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:43.462308 19031 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:43.322280 19046 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:15:43.532246 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.533226 11182 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:43.386538 18971 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:43.425760 11329 rpcz_store.cc:276] 0709 12:15:41.423175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.201021 (+777846us) service_pool.cc:224] Handling call
0709 12:15:42.201273 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.339226 (+137953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.321987 (+982761us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b54002f67d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:43.322204 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197863808}
I20260709 12:15:43.282805 18265 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:43.398873 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:43.544503 11250 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:43.538453 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:43.434073 11254 rpcz_store.cc:276] 0709 12:15:41.974089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.061356 (+ 87267us) service_pool.cc:224] Handling call
0709 12:15:43.328129 (+1266773us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b54000f87d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:43.328253 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8044416}
I20260709 12:15:43.492589 11178 raft_consensus.cc:2889] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:43.348685 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:43.492722 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:43.494279 11177 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:43.494976 11176 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:43.496477 11174 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:43.358469 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1221) took 2171 ms. Trace:
I20260709 12:15:43.548185 11249 rpcz_store.cc:276] 0709 12:15:41.187036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.187213 (+   177us) service_pool.cc:224] Handling call
0709 12:15:41.226238 (+ 39025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:42.360875 (+1134637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.358177 (+997302us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540033ccd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:43.358405 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607520128}
I20260709 12:15:43.545495 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:43.550166 11250 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:43.550316 11935 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:43.542866 19205 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.510584 11322 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:43.538590 19108 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:43.462265 18944 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:43.393573 11255 rpcz_store.cc:276] 0709 12:15:41.957870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.750405 (+792535us) service_pool.cc:224] Handling call
0709 12:15:42.750997 (+   592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:43.039126 (+288129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.304576 (+265450us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400069854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:43.304851 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376711936}
I20260709 12:15:43.514791 11181 rpcz_store.cc:276] 0709 12:15:41.117353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.397569 (+280216us) service_pool.cc:224] Handling call
0709 12:15:41.398103 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.409446 (+ 11343us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:42.338231 (+928785us) log.cc:844] Serialized 14 byte log entry
0709 12:15:43.345963 (+1007732us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b180039db40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287
0709 12:15:43.346092 (+   129us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:43.347798 (+  1706us) log.cc:844] Serialized 26 byte log entry
0709 12:15:43.347966 (+   168us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:43.348004 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.348084 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:43.366035 (+ 17951us) raft_consensus.cc:1685] finished
0709 12:15:43.385701 (+ 19666us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:43.385871 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23641,"prepare.queue_time_us":957973,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2940,"raft.queue_time_us":136418,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":5956,"spinlock_wait_cycles":9894656,"thread_start_us":1315604,"threads_started":3,"wal-append.queue_time_us":468078}
W20260709 12:15:43.532344 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 148342us
I20260709 12:15:43.579839 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:43.535051 19091 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.541579 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:43.589033 11927 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:43.498430 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:43.398263 19049 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:43.397643 19128 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.545495 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 1524) took 1101 ms. Trace:
I20260709 12:15:43.511992 19022 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:43.552666 11934 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:15:43.553300 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 60683.5us
I20260709 12:15:43.516160 19196 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:43.554479 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.514375 19095 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:43.392577 18940 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:43.543651 19123 consensus_meta.cc:287] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.003s
I20260709 12:15:43.497040 19193 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.496762 18972 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.571244 11322 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:43.360471 19029 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.540104 19190 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:43.544552 19046 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:43.495755 19202 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.577878 19090 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.585285 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.586434 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:43.587564 11176 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
W20260709 12:15:43.569265 19174 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.004s
I20260709 12:15:43.555114 19176 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:43.497562 19184 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.594664 11328 rpcz_store.cc:276] 0709 12:15:42.443725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.510960 (+1067235us) service_pool.cc:224] Handling call
0709 12:15:43.545412 (+ 34452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:43.612942 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3181) took 3409 ms. Trace:
I20260709 12:15:43.543483 19031 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:43.593870 19115 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.171s	user 0.018s	sys 0.028s
I20260709 12:15:43.626789 18797 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:43.635031 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:43.636966 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:43.595803 18971 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:43.660127 11929 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:43.585352 19108 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:43.668525 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 82180.3us
I20260709 12:15:43.681105 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:43.576717 19205 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.669479 11933 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:15:43.670702 11928 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:43.672535 11927 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:43.678020 11326 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:15:43.673573 19115 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.250s	user 0.019s	sys 0.029s
I20260709 12:15:43.634013 19123 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.613965 18265 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.681375 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:43.670343 19190 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:43.681223 19091 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:43.624125 19164 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.676195 18797 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:43.679181 19126 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.670702 19174 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.670883 19090 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.702240 19202 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.703713 19090 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.667369 19049 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.704427 19031 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.705159 19123 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:43.705504 19046 consensus_queue.cc:1175] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 22 status { last_received { term: 22 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 22 index: 16 } }
I20260709 12:15:43.704990 19049 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:43.688452 19153 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:43.696214 19216 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:15:43.719664 19206 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:15:43.720551 19108 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:43.710245 19196 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.725092 19205 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.726449 19184 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:43.730284 19204 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:15:43.734508 19193 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:43.727415 19210 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 12:15:43.636610 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.785665 11175 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:43.790782 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.792919 11179 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:43.794706 11180 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:43.785789 11177 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:43.798789 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:43.799952 11179 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:43.614087 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:43.803078 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 188997us
W20260709 12:15:43.809401 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.813798 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:43.654596 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:43.817137 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 162498us
I20260709 12:15:43.640916 11248 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:43.820626 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:43.651823 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:43.668268 11252 rpcz_store.cc:276] 0709 12:15:40.203389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.213780 (+ 10391us) service_pool.cc:224] Handling call
0709 12:15:40.214131 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:40.220625 (+  6494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:40.220658 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.007588 (+786930us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b2c00000e98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec4fe5a11 00007f3ec4ff5372 00007f3ec6727a01 00007f3ec66ec43a 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2
0709 12:15:41.008015 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 12:15:41.337675 (+329660us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:41.451220 (+113545us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.943369 (+492149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.943464 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:43.363004 (+1419540us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:43.612649 (+249645us) raft_consensus.cc:1685] finished
0709 12:15:43.612682 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:43.612869 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":109244,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":139311,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":60225,"spinlock_wait_cycles":493112064,"thread_start_us":921478,"threads_started":3,"wal-append.queue_time_us":393023}
I20260709 12:15:43.705758 18265 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:43.707505 18797 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.690130 18816 raft_consensus.cc:2764] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:15:43.698707 18861 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:43.880119 18861 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.880808 18861 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:43.831746 19108 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.881878 19176 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:43.882227 19108 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:43.883294 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2762) took 1856 ms. Trace:
I20260709 12:15:43.883445 18940 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:43.832605 18972 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:43.832926 18643 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.833669 19174 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.834708 19123 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:43.902510 11179 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:43.899240 18643 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.902865 19123 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:43.690578 19095 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:43.847337 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.845377 11175 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:43.873481 19074 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:15:43.955291 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 123432us
W20260709 12:15:43.871117 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50779.1us
I20260709 12:15:43.897910 18642 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:15:43.898548 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.899204 19100 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.904148 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.959820 19041 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:43.960423 19224 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:15:43.961313 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:43.964308 11255 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:43.883839 11256 rpcz_store.cc:276] 0709 12:15:41.972998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.245391 (+1272393us) service_pool.cc:224] Handling call
0709 12:15:43.245855 (+   464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:43.255877 (+ 10022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:43.255922 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:43.256771 (+   849us) log.cc:844] Serialized 52 byte log entry
0709 12:15:43.414138 (+157367us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:43.431588 (+ 17450us) log.cc:844] Serialized 14 byte log entry
0709 12:15:43.432028 (+   440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.432101 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:43.822560 (+390459us) raft_consensus.cc:1685] finished
0709 12:15:43.822612 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:43.829770 (+  7158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16664,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":281,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":206,"thread_start_us":156794,"threads_started":1,"wal-append.queue_time_us":331504}
I20260709 12:15:43.951004 19108 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:43.957960 18642 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.971946 18642 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:43.977070 19184 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:15:43.977553 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 296336us
I20260709 12:15:43.977872 19184 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:43.978534 19184 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:43.991179 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:43.993944 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3449) took 1073 ms. Trace:
I20260709 12:15:43.994048 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:43.995462 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:43.994582 11322 rpcz_store.cc:276] 0709 12:15:42.920049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.649016 (+728967us) service_pool.cc:224] Handling call
0709 12:15:43.651381 (+  2365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:43.653357 (+  1976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:43.653389 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:43.659262 (+  5873us) log.cc:844] Serialized 26 byte log entry
0709 12:15:43.981974 (+322712us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:43.982088 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.982252 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:43.993313 (+ 11061us) raft_consensus.cc:1685] finished
0709 12:15:43.993359 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:43.993680 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1578,"raft.queue_time_us":2166,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":3228,"thread_start_us":265701,"threads_started":1,"wal-append.queue_time_us":312201}
I20260709 12:15:43.996554 19176 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:43.897918 19126 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:43.997855 19176 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:43.998459 19176 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.998989 19176 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:43.999823 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.000660 11930 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:44.002025 18971 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:44.002563 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.003716 19115 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:44.004046 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.007269 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.007617 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3442) took 1124 ms. Trace:
I20260709 12:15:44.008121 11329 rpcz_store.cc:276] 0709 12:15:42.883261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.539821 (+656560us) service_pool.cc:224] Handling call
0709 12:15:43.540133 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:43.540377 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:15:43.680597 (+140220us) log.cc:844] Serialized 14 byte log entry
0709 12:15:44.006071 (+325474us) log.cc:844] Serialized 14 byte log entry
0709 12:15:44.006923 (+   852us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:44.007019 (+    96us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:44.007109 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.007226 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:44.007439 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":148465,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":126,"raft.queue_time_us":107837,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":212,"spinlock_wait_cycles":20354688,"thread_start_us":298370,"threads_started":2,"wal-append.queue_time_us":282718}
I20260709 12:15:44.009469 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:43.960785 19041 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.027763 11179 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:44.033962 19095 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:44.036660 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.047521 11927 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:44.050318 11178 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:44.051940 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.057329 19233 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:43.983841 19074 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:44.058192 19233 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.059700 19074 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.060730 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:44.062031 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.029888 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.065871 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.068199 11256 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:44.069564 19233 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.070065 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.070312 19231 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:15:44.072278 19233 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:44.072727 19233 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:44.074075 11255 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:44.080449 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.083427 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.083770 11930 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:44.085011 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.086722 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.086942 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3211) took 1130 ms. Trace:
I20260709 12:15:44.087623 11253 rpcz_store.cc:276] 0709 12:15:42.956146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.198228 (+242082us) service_pool.cc:224] Handling call
0709 12:15:44.086478 (+888250us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400068454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.086711 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330918656}
I20260709 12:15:44.087019 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.094287 11173 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:44.098351 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.098915 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.099380 11173 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:44.101397 19090 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:44.101975 19090 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.102440 19090 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:44.094915 19201 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:15:44.105659 19201 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:44.106191 19201 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:44.106819 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 1264) took 1075 ms. Trace:
I20260709 12:15:44.106464 19108 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:15:44.102411 11327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 1526) took 1658 ms (client timeout 1711 ms). Trace:
W20260709 12:15:44.108173 11327 rpcz_store.cc:269] 0709 12:15:42.444244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.933458 (+489214us) service_pool.cc:224] Handling call
0709 12:15:44.101929 (+1168471us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400323cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.102165 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474512256}
I20260709 12:15:44.107302 11928 rpcz_store.cc:276] 0709 12:15:43.031422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.031547 (+   125us) service_pool.cc:224] Handling call
0709 12:15:44.106331 (+1074784us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400415fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.106572 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1169675136}
I20260709 12:15:44.109225 19201 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.110500 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1441) took 1130 ms. Trace:
W20260709 12:15:44.110797 18944 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.518s	user 0.016s	sys 0.000s
I20260709 12:15:44.110987 11935 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:15:44.115470 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.111593 19108 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 12:15:44.110981 11933 rpcz_store.cc:276] 0709 12:15:42.980284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.980458 (+   174us) service_pool.cc:224] Handling call
0709 12:15:42.980968 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:43.687915 (+706947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.109954 (+422039us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54003b5454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:44.110265 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577907584}
I20260709 12:15:44.117887 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.114097 11180 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:44.092263 18643 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.123267 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.123574 18944 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:44.124396 18944 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:44.124686 11935 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:44.126010 18944 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.127135 18862 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.127610 18944 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:44.128094 18861 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.131402 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.132434 11256 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:44.133868 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.134697 11251 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:44.139084 18990 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:44.139797 18990 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.140465 19128 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.150084 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.152890 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.153434 19128 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:44.154201 11257 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:44.154594 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.153143 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.155309 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:44.155551 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.157078 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.158310 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.158619 19072 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.159473 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.160112 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.160952 11249 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:44.161499 11933 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:44.162170 18998 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:44.162688 18998 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.163125 18998 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:44.163941 19176 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.154268 19128 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.177731 11179 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 12:15:44.178282 19162 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.179634 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.180667 11250 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:44.181708 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.153016 19039 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.166978 19098 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.186381 18990 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.185093 18998 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.191942 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.192770 19006 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.194070 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.194882 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.195487 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.195883 11327 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:15:44.196103 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.195703 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.196863 11175 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:44.196985 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.197365 11329 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 12:15:44.197958 11327 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:44.199996 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.200546 19128 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.201799 11329 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:15:44.202708 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.205098 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.206017 11322 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:44.207628 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.208416 11329 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:44.209088 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.209677 11926 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:15:44.209940 11327 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:44.210990 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.211370 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.211613 11177 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:44.211915 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:44.167482 19176 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.170107 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.214332 11252 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:44.215263 11926 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:44.215395 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.215590 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.171828 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.217303 11928 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:44.172032 19151 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 2.725s	user 0.003s	sys 0.008s
I20260709 12:15:44.172971 11935 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23. Ops: [21.16-22.17]
I20260709 12:15:44.170006 19113 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.219398 19233 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.168978 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.169466 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.223543 11249 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:44.221107 18972 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:15:44.171847 19032 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.744s	user 0.003s	sys 0.013s
I20260709 12:15:44.222975 11257 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:44.221966 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3435) took 2978 ms. Trace:
I20260709 12:15:44.222462 11255 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:44.176105 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.226433 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.207291 11251 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:15:44.216377 11182 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:44.231983 18972 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:44.233436 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.233215 18972 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.234748 18862 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:44.235199 18862 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:44.235452 18805 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.235607 18862 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:44.236049 18805 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.245734 11178 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:15:44.247717 19205 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:44.247996 19233 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:44.248328 19205 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:44.248832 19205 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:44.250203 11330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 1519) took 2818 ms (client timeout 2994 ms). Trace:
W20260709 12:15:44.250615 11330 rpcz_store.cc:269] 0709 12:15:41.432021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.064953 (+632932us) service_pool.cc:224] Handling call
0709 12:15:44.249819 (+2184866us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400285ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.249997 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422047616}
I20260709 12:15:44.251379 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.252343 11182 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:44.252507 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 1272) took 1199 ms. Trace:
I20260709 12:15:44.254654 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 1112) took 2023 ms. Trace:
I20260709 12:15:44.255064 11931 rpcz_store.cc:276] 0709 12:15:42.231185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.232484 (+  1299us) service_pool.cc:224] Handling call
0709 12:15:42.232837 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.335469 (+102632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.254195 (+1918726us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b5400650254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:44.254483 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215943936}
I20260709 12:15:44.255637 19041 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.256772 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.257715 18990 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:44.258184 18990 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.258594 18990 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:44.259568 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.259876 18652 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.260087 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.260398 18652 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:15:44.260836 11257 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:44.247782 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.262036 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.262322 18520 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.260908 19046 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:44.263576 11250 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:44.263875 19046 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.264272 19046 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:44.264946 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.263401 19157 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.253005 11929 rpcz_store.cc:276] 0709 12:15:43.053197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.176884 (+123687us) service_pool.cc:224] Handling call
0709 12:15:44.251977 (+1075093us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54000a7154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.252301 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109055360}
I20260709 12:15:44.267379 11249 raft_consensus.cc:2410] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:15:44.175247 19159 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.272657 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.274590 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.274866 11182 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:15:44.275221 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.279937 11173 raft_consensus.cc:2410] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 27 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:15:44.281493 11250 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:44.226464 11323 rpcz_store.cc:276] 0709 12:15:41.243624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.243826 (+   202us) service_pool.cc:224] Handling call
0709 12:15:41.244237 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.244520 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.244557 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.251205 (+  6648us) log.cc:844] Serialized 26 byte log entry
0709 12:15:41.447074 (+195869us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:41.447135 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.447270 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:44.171023 (+2723753us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:44.221325 (+ 50302us) raft_consensus.cc:1685] finished
0709 12:15:44.221345 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:44.221768 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601609472,"thread_start_us":143584,"threads_started":1,"wal-append.queue_time_us":175737}
I20260709 12:15:44.299397 11933 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:44.303284 11255 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: [23.16-25.17]
I20260709 12:15:44.305676 18971 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.307945 18861 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.308074 18971 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:44.309106 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.310590 18971 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.311554 19157 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.314204 19157 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.339679 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.300096 18940 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.345240 11173 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:44.345667 18940 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.316116 11180 pending_rounds.cc:85] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 16
I20260709 12:15:44.321053 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:44.324604 11327 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:44.324776 18642 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.333295 18862 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.336377 18998 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:44.337826 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 80166.7us
I20260709 12:15:44.339452 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3191) took 3066 ms. Trace:
I20260709 12:15:44.341460 19123 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.341912 19159 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:44.308880 11251 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:44.354298 11249 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:15:44.353997 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.365008 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.366214 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:44.367287 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46299.4us
I20260709 12:15:44.365365 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.346915 11180 pending_rounds.cc:107] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 19.17
I20260709 12:15:44.372982 11180 raft_consensus.cc:2889] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:44.364779 11248 rpcz_store.cc:276] 0709 12:15:41.272435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.351526 (+1079091us) service_pool.cc:224] Handling call
0709 12:15:43.328724 (+977198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:43.819797 (+491073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.338353 (+518556us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54000f87d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:44.339088 (+   735us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b54000f87d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.339237 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2963969792}
I20260709 12:15:44.376773 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.377660 11331 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.377715 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.378473 11933 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:44.378885 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:44.379348 19091 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:44.379813 19091 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:44.380246 19091 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:44.380367 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.382018 19031 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:15:44.382797 19031 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:44.383186 19031 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.383525 19031 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:44.385962 18816 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:44.385988 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.386863 19159 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:44.388142 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.387952 19159 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.388760 11253 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:44.379788 18652 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.389968 19205 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.391539 18652 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.392458 19205 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.392795 18797 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:44.394722 11252 raft_consensus.cc:2368] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22.
I20260709 12:15:44.381467 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:44.402467 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:44.403455 18805 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.403950 19046 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.404393 19091 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.406013 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:44.408011 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 140446us
I20260709 12:15:44.408846 18861 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:44.405354 11901 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:44.410601 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.411599 11255 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:44.412437 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.417321 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.426723 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.370272 19184 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.432998 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.433311 18862 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.434068 11330 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.435949 11324 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 12:15:44.436733 11251 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:44.440495 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.441758 11928 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:44.441913 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.442238 11926 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:44.414726 19205 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.436260 11327 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:44.443965 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.422312 19126 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:15:44.456569 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 75197.3us
I20260709 12:15:44.458032 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:44.469810 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59791.3us
I20260709 12:15:44.476585 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.476917 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1447) took 1292 ms. Trace:
I20260709 12:15:44.477558 11329 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.477274 11934 rpcz_store.cc:276] 0709 12:15:43.184113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:43.201868 (+ 17755us) service_pool.cc:224] Handling call
0709 12:15:44.476586 (+1274718us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54004c2554. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.476755 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":804615040}
W20260709 12:15:44.474612 19164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.006s	sys 0.009s
I20260709 12:15:44.482605 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.485337 11174 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
W20260709 12:15:44.489895 19164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.009s
I20260709 12:15:44.494719 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.494935 11251 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:44.495576 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.496285 11934 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:44.497808 18998 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:44.505986 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.507036 18652 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:44.452353 19126 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:44.507334 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.507750 19126 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:44.507718 18652 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:44.509114 18652 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.510046 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.512461 19123 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:44.513003 19123 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:44.513969 19123 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.514922 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.515566 19029 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.517048 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.517381 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.518204 11330 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:44.518635 19177 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.519886 11329 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.519662 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.520229 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.521746 11322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879"
dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:39356
I20260709 12:15:44.521974 11179 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:44.522507 11322 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:44.522933 11322 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:44.526306 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.531100 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.534125 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.537904 11323 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:44.466357 18805 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:44.538717 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.453559 19205 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.579772 11251 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:15:44.507273 18804 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.581902 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.643669 18804 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.584872 18805 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.645129 19029 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:44.645809 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.646106 18804 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.645748 19029 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.646621 18805 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.587795 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3423) took 1242 ms. Trace:
I20260709 12:15:44.588970 11181 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:44.648512 18972 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:44.650276 18861 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:44.650725 18861 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:44.651914 11248 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:44.651825 18861 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.653178 18861 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:44.653810 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1329) took 1118 ms. Trace:
I20260709 12:15:44.655680 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.644398 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3428) took 1251 ms. Trace:
I20260709 12:15:44.656656 18652 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:44.589221 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.590428 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1330) took 1054 ms. Trace:
W20260709 12:15:44.593736 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.593863 19174 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:44.594226 18642 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.594619 19090 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.657428 11179 rpcz_store.cc:276] 0709 12:15:43.535626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.522790 (+987164us) service_pool.cc:224] Handling call
0709 12:15:44.523163 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:44.651570 (+128407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:44.651601 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:44.651649 (+    48us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:44.651696 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.653514 (+  1818us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54002e4ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:44.653565 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:44.653731 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294756096}
I20260709 12:15:44.648401 11178 rpcz_store.cc:276] 0709 12:15:43.345519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.083033 (+737514us) service_pool.cc:224] Handling call
0709 12:15:44.587557 (+504524us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54001f36d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:44.587707 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353947904}
I20260709 12:15:44.598281 11329 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:44.581902 18990 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.599202 18940 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.599786 19091 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.601136 19184 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.603119 19205 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.672976 11322 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.603215 19046 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.604946 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.606308 11326 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:44.530941 18816 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:15:44.607538 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:44.608108 19177 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:44.608132 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.609289 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.610247 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3430) took 1200 ms. Trace:
W20260709 12:15:44.621109 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.623090 19201 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:44.623551 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.627465 11173 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:15:44.689091 11325 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:44.690032 11173 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:44.691468 11178 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 27. Current term is 29. Ops: []
I20260709 12:15:44.636332 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:44.692610 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:44.693208 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57124.4us
I20260709 12:15:44.656948 11177 rpcz_store.cc:276] 0709 12:15:43.393235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.212420 (+819185us) service_pool.cc:224] Handling call
0709 12:15:44.212941 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:44.213270 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:44.428092 (+214822us) log.cc:844] Serialized 14 byte log entry
0709 12:15:44.644033 (+215941us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:44.644088 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:44.644135 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.644213 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:44.644341 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101411,"prepare.queue_time_us":170672,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":36,"raft.queue_time_us":104532,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":10496,"thread_start_us":283675,"threads_started":2,"wal-append.queue_time_us":196222}
I20260709 12:15:44.694003 11148 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:44.696544 19113 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:44.697289 19113 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:15:44.637013 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 106052us
I20260709 12:15:44.698081 19113 consensus_queue.cc:1048] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:15:44.637193 19231 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:15:44.644534 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 105887us
I20260709 12:15:44.701962 19231 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.702486 19231 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:44.648031 19029 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.644647 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.659178 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.658178 11174 rpcz_store.cc:276] 0709 12:15:43.535951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.589239 (+1053288us) service_pool.cc:224] Handling call
0709 12:15:44.589667 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:44.589970 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:44.590009 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:44.590075 (+    66us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:44.590128 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.590197 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:44.590346 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:44.657128 18652 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:44.681231 11248 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:44.633355 19031 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.658795 19174 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:44.703913 11176 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:44.704710 11322 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:44.686856 11175 rpcz_store.cc:276] 0709 12:15:43.410102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.212417 (+802315us) service_pool.cc:224] Handling call
0709 12:15:44.212915 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:44.265743 (+ 52828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:44.265783 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:44.266324 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:15:44.471243 (+204919us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:44.471283 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.471364 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:44.610003 (+138639us) raft_consensus.cc:1685] finished
0709 12:15:44.610022 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:44.610165 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24824,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11307,"spinlock_wait_cycles":2304,"thread_start_us":204487,"threads_started":1,"wal-append.queue_time_us":339156}
W20260709 12:15:44.707221 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.683638 19090 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:44.708948 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51546.2us
I20260709 12:15:44.633291 19126 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:44.707432 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.708480 18652 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:44.716725 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:44.729135 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.728799 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.728359 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.732118 11929 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:44.714558 11253 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:44.732142 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.732447 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.715234 11328 raft_consensus.cc:2393] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:44.714434 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.732319 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.735272 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.735826 11934 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 28 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:15:44.738219 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.739806 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:44.744112 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.712852 18998 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:15:44.749480 11327 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.714437 19184 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:15:44.755524 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.719138 18816 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 12:15:44.805716 11181 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:15:44.751034 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:44.756318 19090 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.054s
W20260709 12:15:44.849036 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 98160us
I20260709 12:15:44.748056 18998 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:44.786605 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.744875 19205 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:44.810334 19187 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:15:44.772277 11330 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:44.744621 19177 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.741046 18940 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.771282 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.789615 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.752947 19029 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:44.757407 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.817024 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:44.805797 19153 raft_consensus.cc:2764] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election decision vote started in defunct term 25: won
I20260709 12:15:44.882000 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.805974 19188 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:44.822420 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 214998us
I20260709 12:15:44.823410 19031 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:44.805478 19239 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:15:44.855085 18998 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:15:44.824044 19174 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:44.826130 19126 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.830484 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:44.890928 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 74144.5us
W20260709 12:15:44.814345 19254 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:15:44.894362 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.841675 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.796878 11179 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:44.845016 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:44.846287 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 44224.7us
I20260709 12:15:44.796939 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.847846 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:44.872749 11328 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.878777 11182 raft_consensus.cc:1240] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 12:15:44.869803 11148 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Timed out: RequestConsensusVote RPC to 127.9.198.195:42967 timed out after 1.676s (SENT)
I20260709 12:15:44.870052 19123 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:44.879755 11325 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.883024 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.899678 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:44.888896 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:44.889412 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.891474 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:44.891667 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:44.893996 11248 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 12:15:44.894130 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:44.835989 18971 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.896500 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.896413 11223 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:44.879642 19029 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.906636 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:44.901928 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.903781 11250 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:44.904177 11254 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:15:44.906415 11257 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:44.908030 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:44.910523 11323 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:15:44.913868 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72314.2us
I20260709 12:15:44.915387 11252 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:44.910315 19029 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:44.916482 19193 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:44.920745 18972 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:44.921257 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.922122 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.915616 11330 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.921394 18972 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:44.922752 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.884289 19091 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:44.891191 19113 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:44.925612 11147 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.930591 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.930976 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:44.904847 19187 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:44.925560 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.899649 19123 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:44.934355 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.939563 19164 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:44.922365 19006 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.942560 19205 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:44.943305 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:44.912206 18990 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:44.938882 11325 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:44.936030 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.941608 11328 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:44.907539 19188 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.949949 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.926963 19091 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:44.932206 19174 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:44.955333 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59063.5us
I20260709 12:15:44.947163 19126 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:44.957495 19126 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:44.938751 19031 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:44.959160 11330 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:44.943890 19123 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:44.960764 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:44.926625 19113 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:44.968187 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.907816 18940 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.942678 18971 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.974388 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.981882 11327 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:44.976825 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.977887 18862 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:44.978332 19032 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:44.974839 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:44.975914 19205 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:15:44.985380 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92601.8us
I20260709 12:15:44.983696 18862 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.989998 18862 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:44.995191 11326 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:44.994858 19113 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:44.996435 19113 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:44.997815 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44207.6us
I20260709 12:15:44.998685 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:45.000000 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:45.000893 19153 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:45.001749 19153 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:45.002172 19153 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.002236 19205 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:45.002561 19153 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:45.002720 19205 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:45.003706 19205 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.005146 19205 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:44.967233 19126 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:45.001251 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:45.008776 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139123us
I20260709 12:15:45.010162 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.010808 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:44.969558 19031 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.011890 11325 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:44.999989 19095 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:45.029623 11329 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:45.030328 11327 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:44.987052 19032 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:45.017252 19031 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:45.042383 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72079.4us
I20260709 12:15:45.028597 19098 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:45.044596 19098 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:45.045058 19098 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:45.045837 18940 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:44.972488 19006 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.051422 19006 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.059113 18861 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:15:45.059675 18861 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:45.041751 11325 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:45.037110 19032 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:45.031116 18990 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.092094 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:45.031802 19095 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:45.029035 19177 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:45.094925 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.103441 11182 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:45.105094 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.101312 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.097744 11254 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 29.
I20260709 12:15:45.105798 11325 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:45.107580 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.099742 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.118698 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.119624 11175 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:45.122078 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.117861 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.131457 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:45.133548 11327 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:15:45.104826 19095 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:45.136679 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44657.2us
W20260709 12:15:45.138630 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 67361.6us
I20260709 12:15:45.149786 19098 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.152640 19098 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:45.157102 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60092.1us
I20260709 12:15:45.036675 19091 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.035145 18972 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.167243 18804 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.179955 18804 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.180401 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:45.182413 19257 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:15:45.182591 18804 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.183383 18804 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:15:45.183759 19248 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:15:45.124329 18797 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:45.175031 11328 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 17: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:45.172137 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:45.188477 18940 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:45.191740 18940 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:45.192294 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:45.180130 19188 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.167855 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.193984 11330 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:45.194223 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1327) took 1659 ms. Trace:
I20260709 12:15:45.194770 11180 rpcz_store.cc:276] 0709 12:15:43.534882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.266266 (+731384us) service_pool.cc:224] Handling call
0709 12:15:44.266738 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:44.406211 (+139473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:44.406257 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:44.407040 (+   783us) log.cc:844] Serialized 52 byte log entry
0709 12:15:44.720775 (+313735us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:44.720830 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.720924 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:45.193902 (+472978us) raft_consensus.cc:1685] finished
0709 12:15:45.193937 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:45.194143 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"thread_start_us":313210,"threads_started":1,"wal-append.queue_time_us":502787}
I20260709 12:15:45.198266 18971 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.198849 18971 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:45.199246 18971 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:45.200093 18971 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.202762 19231 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.199129 18804 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:15:45.204367 18804 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.205147 19231 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.206485 18804 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.202847 11330 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:45.139535 19177 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:15:45.168994 19046 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:45.210740 19046 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:45.137831 18990 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.211980 18862 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.166041 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:45.219278 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:45.221900 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55972.2us
I20260709 12:15:45.197085 18265 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
W20260709 12:15:45.229765 19256 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:15:45.230257 19188 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:45.233052 11325 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:15:45.216321 18805 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:45.243789 19091 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:45.243903 19029 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.249221 19159 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:15:45.251140 19255 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.317s	user 0.002s	sys 0.000s
I20260709 12:15:45.231149 19201 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:45.256614 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:45.272445 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.283197 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.225528 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:45.300966 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 75523us
I20260709 12:15:45.307659 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:45.305351 11175 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:45.284123 18862 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.311734 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.313434 11325 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:45.284750 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.310168 11182 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
W20260709 12:15:45.274744 19252 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
I20260709 12:15:45.321178 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:45.323561 11257 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:15:45.254647 19006 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:45.254632 18805 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:45.324122 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:45.314553 11324 raft_consensus.cc:2376] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25. Re-sending same reply.
I20260709 12:15:45.211944 19046 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.355119 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:45.322543 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 361878us
I20260709 12:15:45.359627 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.322947 19188 raft_consensus.cc:2764] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:15:45.362573 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:45.373462 11325 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:45.331030 19159 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:45.354547 19006 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.351995 18805 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:45.362399 19177 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.371145 11179 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 26: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:45.324877 19231 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:45.362296 11175 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:45.351748 19201 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:45.361524 18652 consensus_meta.cc:287] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.616s	user 0.008s	sys 0.003s
I20260709 12:15:45.261623 19123 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.403281 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.355234 18642 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:45.360080 19098 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.411494 11147 leader_election.cc:400] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22.
I20260709 12:15:45.408123 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.413913 11328 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:45.408838 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.312978 19193 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.050s
W20260709 12:15:45.420619 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.415778 11324 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:45.412546 18642 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:45.421888 19098 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.422936 18642 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.424222 19098 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:45.425139 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 72482.8us
I20260709 12:15:45.428313 19095 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.351413 19090 consensus_queue.cc:1059] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:15:45.430814 11175 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:45.430707 19095 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.432092 19095 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:45.433358 19123 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.438946 19126 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.412236 18652 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.458500 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3072) took 1002 ms. Trace:
I20260709 12:15:45.405392 19006 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:45.479506 11330 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:15:45.499421 11330 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:45.484189 11256 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:45.410332 19159 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:45.419780 19187 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:45.413275 19201 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:45.510747 19153 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.004s
I20260709 12:15:45.525358 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.527539 18652 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.528122 11174 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:45.496976 18804 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.530895 18990 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.531260 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:45.534272 19205 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.529s	user 0.014s	sys 0.002s
I20260709 12:15:45.557961 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.561367 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.560012 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:45.567571 11177 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:15:45.573179 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 534971us
W20260709 12:15:45.556618 18998 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.004s
I20260709 12:15:45.492273 11331 rpcz_store.cc:276] 0709 12:15:44.456251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.456388 (+   137us) service_pool.cc:224] Handling call
0709 12:15:44.473119 (+ 16731us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:44.504590 (+ 31471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:44.504622 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:44.505162 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:15:44.707841 (+202679us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:44.707883 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:44.708058 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:45.273994 (+565936us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:45.447497 (+173503us) raft_consensus.cc:1685] finished
0709 12:15:45.447533 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:45.458387 (+ 10854us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":12284160,"thread_start_us":202289,"threads_started":1,"wal-append.queue_time_us":338770}
I20260709 12:15:45.418776 19177 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.564459 19231 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:45.569367 11330 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 15, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:45.564018 19032 consensus_meta.cc:287] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.006s
W20260709 12:15:45.591730 11323 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.004s
I20260709 12:15:45.607533 19262 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:45.608284 19262 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.608888 19126 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.606894 11323 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:45.580430 19164 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:45.614792 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 434448us
I20260709 12:15:45.615877 11330 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:45.591744 11900 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:15:45.576165 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 388261us
W20260709 12:15:45.534705 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.009s	sys 0.039s
I20260709 12:15:45.553743 18862 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:15:45.591476 18998 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:45.623122 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:45.619531 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.009s	sys 0.039s
I20260709 12:15:45.624336 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:45.560739 19153 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:45.623708 19031 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.000s
W20260709 12:15:45.632262 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.028s	sys 0.033s
I20260709 12:15:45.617619 19231 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:45.639185 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.028s	sys 0.033s
W20260709 12:15:45.641552 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 214352us
W20260709 12:15:45.644745 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.645663 11178 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
W20260709 12:15:45.633023 19113 consensus_meta.cc:287] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.636s	user 0.014s	sys 0.001s
I20260709 12:15:45.651525 11174 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:45.605785 18804 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.626387 19184 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:45.659328 11330 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
I20260709 12:15:45.661311 11330 pending_rounds.cc:85] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 16
I20260709 12:15:45.660792 11181 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:45.656612 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:45.613665 19032 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.662374 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:45.664474 11330 pending_rounds.cc:107] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 26.17
I20260709 12:15:45.660257 19029 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:45.656595 19126 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.623972 19164 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.677184 11180 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:45.680850 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:45.647948 19031 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.675086 18804 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.676724 19113 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:45.670382 11330 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:45.667281 19266 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:45.692168 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.677125 19268 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.661881 19184 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.644981 19159 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.712970 11324 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:45.732594 19268 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:45.736778 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.739900 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:45.756076 11330 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:45.741940 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79647.2us
I20260709 12:15:45.758543 11223 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:45.760109 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:45.669924 18265 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:15:45.687101 18652 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:45.747840 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.748800 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.776942 11327 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 20: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:45.749730 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.754058 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.778219 11322 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:45.755141 11324 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:45.765359 11933 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:15:45.783569 11325 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:45.790622 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:45.792764 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:45.683883 19267 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:45.763983 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:45.775498 11328 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:15:45.809082 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 69330.7us
I20260709 12:15:45.809829 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:45.808943 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:45.783280 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:45.844301 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:45.847679 11254 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:45.843716 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:45.842906 11257 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:45.852066 11929 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:45.845203 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:45.814713 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69741.9us
I20260709 12:15:45.822588 11931 raft_consensus.cc:2368] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22.
W20260709 12:15:45.859956 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.681458 19126 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.859082 11326 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
W20260709 12:15:45.868739 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 178581us
I20260709 12:15:45.858242 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:45.864431 11331 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:15:45.882258 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 122248us
I20260709 12:15:45.883165 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:45.884420 11223 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:45.884447 11330 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:45.756202 19274 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:15:45.887755 11252 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:45.899979 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:45.796896 18652 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.886315 11324 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.756680 19266 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:45.821735 19267 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.860837 19090 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:45.869298 19091 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:45.911314 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:45.922345 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:45.923898 11254 raft_consensus.cc:2368] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22.
W20260709 12:15:45.925719 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67590.7us
I20260709 12:15:45.895310 18642 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:45.932078 11324 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:45.933511 18990 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:45.933595 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:45.933885 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 192726us
I20260709 12:15:45.895641 18804 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:15:45.906816 19205 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:45.908541 18652 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:45.741240 18861 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:45.912746 19277 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.913791 19269 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:45.952665 19264 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.340s	user 0.002s	sys 0.000s
W20260709 12:15:45.934974 19261 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.525s	user 0.002s	sys 0.002s
I20260709 12:15:45.877995 19231 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:45.965679 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65748.9us
I20260709 12:15:45.981981 11223 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:45.976264 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:45.951486 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 324249us
I20260709 12:15:45.913754 19267 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:45.946982 19184 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:45.952467 19265 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:45.992098 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80915.9us
I20260709 12:15:45.997524 19126 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.982997 19177 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:46.001305 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.001906 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.002713 11254 raft_consensus.cc:2368] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26.
I20260709 12:15:46.003693 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:45.989800 19205 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:46.006795 19278 raft_consensus.cc:2764] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:15:46.007165 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.008158 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.008801 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:45.954710 19090 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:46.010012 19090 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.011713 19275 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:45.972497 19269 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.012425 19090 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:45.995419 19268 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:15:46.009846 19266 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.009528 11331 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:15:46.012348 11328 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:15:46.004268 11254 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:45.996340 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 135606us
W20260709 12:15:46.030339 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:46.031037 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.031358 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.032269 11330 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:46.034907 18990 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:45.987015 18861 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:15:46.036064 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.027735 19184 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:46.016301 18998 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.035954 19187 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.054320 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.045796 19274 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 29 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:46.058249 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57028.7us
I20260709 12:15:46.061513 11900 leader_election.cc:400] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26.
I20260709 12:15:45.982575 19277 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.069726 19153 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.071673 11176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:46.072511 11176 raft_consensus.cc:606] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:46.072916 11176 raft_consensus.cc:614] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:46.039942 18990 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:46.065532 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:45.987332 18816 consensus_queue.cc:1175] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "016411114c6d42039bd04025378dfc98" responder_term: 29 status { last_received { term: 27 index: 18 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 27. Current term is 29. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:46.052829 19031 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:46.067028 19283 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:15:46.075173 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.077008 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:46.053306 19046 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:46.090534 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 82491.5us
I20260709 12:15:46.093084 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.093227 19201 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.083564 11330 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:46.084411 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.084899 11176 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:15:46.086664 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.087437 19159 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:46.064287 19187 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.087716 18990 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.088876 19289 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:46.089998 19269 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:46.123672 11250 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:46.093982 19282 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:15:46.134361 19279 log.cc:927] Time spent T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:15:46.095145 11256 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2766) took 1945 ms (client timeout 2069 ms). Trace:
I20260709 12:15:46.135952 19286 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:46.135881 11256 rpcz_store.cc:269] 0709 12:15:44.149484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.149726 (+   242us) service_pool.cc:224] Handling call
0709 12:15:46.094888 (+1945162us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400087ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:46.095040 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2429920640}
I20260709 12:15:46.136479 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.137202 11329 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:46.137413 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.031386 19265 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.138253 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.138909 11328 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:46.140173 11328 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:46.140708 19265 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.126075 19159 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.141908 11256 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:46.108906 19006 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
I20260709 12:15:46.142925 19006 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.103119 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.145143 11181 raft_consensus.cc:2393] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:46.109251 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.146226 11177 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:46.112546 18862 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.147081 11331 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:46.147364 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.149066 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:46.114265 19260 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.580s	user 0.001s	sys 0.002s
I20260709 12:15:46.115274 18652 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.150985 11252 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:46.151895 18990 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.116139 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:46.153199 19174 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.190s	user 0.010s	sys 0.006s
W20260709 12:15:46.089535 18805 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.001s
I20260709 12:15:46.157943 18652 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.158129 18805 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.158723 19006 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.159531 11177 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:46.160458 18805 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.161932 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.163996 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.164954 11329 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:46.111977 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.167608 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.168151 19164 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:46.090407 19262 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.170063 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.170485 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.171149 11931 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:46.171890 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.083134 11254 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:46.124013 11251 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:46.126792 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:46.130481 19187 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.131247 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.131971 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.075256 19184 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:46.133530 19029 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:46.096012 19274 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.142127 19159 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:46.059597 19278 consensus_queue.cc:1059] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:15:46.146229 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:46.111909 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:46.141021 19267 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.165433 11324 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
W20260709 12:15:46.187892 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 122509us
I20260709 12:15:46.172623 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.231904 11325 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:46.232582 11249 raft_consensus.cc:2376] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26. Re-sending same reply.
I20260709 12:15:46.231818 11324 pending_rounds.cc:85] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 15
I20260709 12:15:46.172744 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.233541 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.172888 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:46.234297 11330 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:46.124445 11253 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:15:46.173486 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.232782 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.238345 19287 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:46.193136 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.240023 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:46.241187 11329 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:46.233186 11324 pending_rounds.cc:107] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 22.16
W20260709 12:15:46.064831 19193 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.013s	sys 0.006s
I20260709 12:15:46.123769 11935 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:46.250999 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.234596 11257 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:46.249233 11181 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:46.250560 11329 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:46.250679 11324 raft_consensus.cc:2889] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:46.250667 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 138848us
W20260709 12:15:46.251545 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59261.2us
W20260709 12:15:46.240262 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 67462.8us
I20260709 12:15:46.253749 11181 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:46.256495 11249 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:15:46.254611 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.258395 11256 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:46.254647 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.260394 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:46.256950 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.262140 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.231971 19029 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.194167 19201 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:46.173252 19289 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:46.264189 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.265050 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:46.268589 11176 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:46.271267 11177 raft_consensus.cc:2368] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:46.265226 11181 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.266132 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.266776 11248 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: [24.11-24.11]
W20260709 12:15:46.252398 19193 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.014s	sys 0.006s
I20260709 12:15:46.153527 19032 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.267463 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.277305 11251 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:46.279358 19029 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.274610 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.255275 19265 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:46.270550 18804 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:46.285774 18971 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:46.283538 18862 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:46.270141 18972 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.261216 19187 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:46.252682 19164 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:46.272330 11329 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.255419 19031 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:46.190053 18990 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 12:15:46.303565 11181 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:46.261013 19282 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:46.311197 11176 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: [19.10-21.11]
I20260709 12:15:46.311851 11256 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:46.311903 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:46.314932 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 82312.6us
I20260709 12:15:46.334221 19265 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.314992 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.353963 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59670 (request call id 1534) took 1509 ms. Trace:
I20260709 12:15:46.354561 11251 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:46.354490 11323 rpcz_store.cc:276] 0709 12:15:44.844396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.920972 (+ 76576us) service_pool.cc:224] Handling call
0709 12:15:46.353743 (+1432771us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b54004290d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:46.353889 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224211200}
I20260709 12:15:46.268190 19231 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:46.327870 11177 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:15:46.272339 19289 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.330394 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.388883 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.389567 11254 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:15:46.330590 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.389756 11176 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:46.389351 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.390347 11173 raft_consensus.cc:2410] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:15:46.287400 19268 raft_consensus.cc:3055] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:46.263618 18861 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.062s
I20260709 12:15:46.276147 19201 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:46.286778 19277 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:46.336077 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3455) took 1018 ms. Trace:
W20260709 12:15:46.337560 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 83324.1us
I20260709 12:15:46.343175 11329 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:46.136605 19286 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.311039 19177 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:46.325977 18804 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.358253 19231 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.347424 19205 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.315582 19174 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:46.391688 19268 raft_consensus.cc:740] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:46.335563 19029 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:46.350692 18805 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:46.394749 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.394845 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.428567 11176 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:46.337061 19113 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:46.392691 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.336853 18972 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.336790 19164 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:46.436174 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.416486 19231 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:46.400992 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:46.438925 11257 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:46.402082 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:46.439241 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 178917us
I20260709 12:15:46.439467 11173 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:46.402693 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.440107 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:46.346033 11253 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:46.397727 11182 rpcz_store.cc:276] 0709 12:15:45.318088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.324271 (+  6183us) service_pool.cc:224] Handling call
0709 12:15:45.324567 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:45.324784 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:45.654460 (+329676us) log.cc:844] Serialized 14 byte log entry
0709 12:15:46.155753 (+501293us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b18003778c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c23a
0709 12:15:46.335673 (+179920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:46.335727 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:46.335769 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.335847 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:46.336008 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192561,"prepare.queue_time_us":388007,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":180747,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5797,"spinlock_wait_cycles":76930432,"thread_start_us":493253,"threads_started":2,"wal-append.queue_time_us":294265}
I20260709 12:15:46.406270 11254 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:46.391958 19289 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:46.350525 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 85574.2us
I20260709 12:15:46.396948 19188 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.415799 18265 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:46.425486 11329 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 10. (term mismatch)
I20260709 12:15:46.428625 11251 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:46.400152 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.427227 11326 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:46.431906 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.405546 18804 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:46.434880 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.435276 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.437834 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:46.455852 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.441533 19282 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:46.446439 19265 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:46.448675 19296 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:15:46.449016 19091 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:15:46.449937 19299 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:15:46.348933 19266 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:15:46.468053 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 68211.4us
I20260709 12:15:46.453840 11329 pending_rounds.cc:85] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 9
I20260709 12:15:46.451535 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.469152 11329 pending_rounds.cc:107] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 12:15:46.457086 11176 raft_consensus.cc:2393] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:46.469870 18861 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:46.470528 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:46.423803 19205 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.466864 11177 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:46.473105 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:46.468080 19267 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.480580 11257 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 27. Current term is 29. Ops: [27.18-27.18]
I20260709 12:15:46.481274 11249 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
W20260709 12:15:46.481648 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.459630 19031 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:46.473846 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.485003 11176 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:46.474231 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.485972 11182 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:15:46.486943 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.472479 11148 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.469449 11329 raft_consensus.cc:2889] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:46.487726 11329 pending_rounds.cc:107] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 12:15:46.488065 11329 raft_consensus.cc:2889] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:46.490368 11173 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:46.493806 18862 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.494431 18862 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.496620 18862 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.435884 18805 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.498020 19031 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:46.498209 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3448) took 1768 ms. Trace:
I20260709 12:15:46.498518 19031 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:46.498657 11174 rpcz_store.cc:276] 0709 12:15:44.712021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.844644 (+132623us) service_pool.cc:224] Handling call
0709 12:15:46.479766 (+1635122us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54000d9b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:46.479892 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1454188800}
I20260709 12:15:46.500260 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:46.500335 11174 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: [25.17-25.17]
W20260709 12:15:46.501559 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.465592 19091 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.502247 19091 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:46.460685 18265 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.507520 18265 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.508096 18265 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:46.508512 18265 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:46.511507 19205 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.513144 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:46.513975 11935 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:15:46.498046 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.509296 18265 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.536535 11181 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:46.522511 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.544483 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.545352 11257 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:46.352141 19184 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:46.549572 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 86521.3us
W20260709 12:15:46.551728 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79336us
I20260709 12:15:46.550746 18862 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:15:46.519546 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45504.7us
I20260709 12:15:46.554491 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.555207 11173 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:46.471225 19304 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:46.457479 19188 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.515937 18861 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:46.514873 19285 raft_consensus.cc:2764] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:15:46.514707 19266 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.570111 11935 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:15:46.550932 19123 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.535480 19090 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:46.480639 19286 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.465896 19282 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.578574 11182 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
W20260709 12:15:46.585714 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.535002 19046 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:46.583019 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:46.588927 11179 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:46.583751 11257 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:15:46.514978 19201 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.577615 19177 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:15:46.588179 19153 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
W20260709 12:15:46.592589 19301 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:15:46.593358 19300 log.cc:927] Time spent T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:15:46.584877 19281 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:15:46.593024 19266 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.594233 18990 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.603873 19032 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:46.616415 18990 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.617261 18805 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.618798 18990 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.594610 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2800) took 2223 ms. Trace:
I20260709 12:15:46.620492 11178 rpcz_store.cc:276] 0709 12:15:44.371456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.692184 (+320728us) service_pool.cc:224] Handling call
0709 12:15:44.937372 (+245188us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5800002a98. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:15:44.951404 (+ 14032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:45.649947 (+698543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.594265 (+944318us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54002e40d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:46.594513 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600675584}
I20260709 12:15:46.479776 19268 consensus_queue.cc:260] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.622291 19268 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:46.611574 19187 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:46.597951 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.626068 19159 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.581923 19123 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.627560 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.628898 19123 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.629856 11256 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:46.630033 19153 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:46.631208 19273 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.631727 19273 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.632848 19090 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:46.633283 19090 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:46.634003 19187 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.629536 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:46.634579 19187 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.635462 19006 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:46.635848 19006 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.636166 19006 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:46.607971 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.637755 18972 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.637894 19187 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.640293 11255 raft_consensus.cc:1240] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:46.641809 11249 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:46.628820 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.645195 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:46.577267 11901 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:46.656015 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 78823.2us
I20260709 12:15:46.661020 11178 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:46.629448 19123 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:46.664295 11255 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:15:46.604600 19286 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:46.664714 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53743.7us
I20260709 12:15:46.667755 11257 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:15:46.668963 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:46.672287 18862 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.676064 18862 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.679440 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.681053 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.681767 11329 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:46.682252 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.683679 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.684666 11325 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:46.670939 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.690232 11328 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 12:15:46.690789 11328 pending_rounds.cc:85] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 16
I20260709 12:15:46.691170 11328 pending_rounds.cc:107] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 12:15:46.691484 11328 raft_consensus.cc:2889] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:46.643810 18971 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:46.693259 11901 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.663949 18805 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.668119 18972 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:46.701794 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.699024 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.699218 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.703619 11323 raft_consensus.cc:2393] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:46.699834 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.704730 11327 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:46.661468 19159 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.595904 19046 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:46.700829 11222 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.617s (SENT) [suppressed 13 similar messages]
W20260709 12:15:46.742002 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63123.9us
I20260709 12:15:46.690655 19265 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.743333 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:46.743738 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.746481 11931 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:46.694834 19032 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:15:46.744717 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.749437 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3454) took 1442 ms. Trace:
I20260709 12:15:46.749873 11180 rpcz_store.cc:276] 0709 12:15:45.307115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.307396 (+   281us) service_pool.cc:224] Handling call
0709 12:15:46.749281 (+1441885us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54003d1dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:46.749384 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848147200}
I20260709 12:15:46.749930 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.678010 19273 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.634270 19090 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.696239 18861 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.693063 18971 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:46.701440 19153 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:46.751683 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.752471 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:46.753970 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:46.701344 19123 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:46.670848 19091 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.746953 11178 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:46.761977 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.745908 18972 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.800776 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.696815 19193 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 12:15:46.690955 19286 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.771572 11931 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:46.771938 11935 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:46.773926 19273 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.774068 19046 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:46.762395 19306 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:15:46.848271 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.848506 19029 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:46.743034 18652 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:46.849665 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.765695 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.851024 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3246) took 1129 ms. Trace:
I20260709 12:15:46.851437 11248 rpcz_store.cc:276] 0709 12:15:45.721349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.721518 (+   169us) service_pool.cc:224] Handling call
0709 12:15:45.782787 (+ 61269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:46.274547 (+491760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.850765 (+576218us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b5400067a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:46.850964 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105498880}
I20260709 12:15:46.851032 11325 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:46.801828 19159 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.853830 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:46.784435 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 44251.6us
I20260709 12:15:46.853981 19159 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.854357 19205 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:46.854993 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.855518 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:46.801859 18816 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:15:46.856474 19268 raft_consensus.cc:2764] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:15:46.856788 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.857462 11325 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:46.858201 18816 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:46.858436 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:46.858505 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:46.766990 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.859138 11325 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:46.778744 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.747710 19177 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.860379 19188 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.837744 18861 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.860675 19177 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:46.757402 19307 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:15:46.753356 19032 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.752172 19265 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.830824 19153 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.754168 19282 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.839937 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.841982 11327 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:46.892448 11933 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:46.707073 18805 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.845314 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.893608 11255 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:46.846627 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:46.860110 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.863627 11331 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:46.863637 11251 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 27. Ops: []
I20260709 12:15:46.868731 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.876511 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:46.860421 11929 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:46.880652 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:46.896068 11330 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 17.
I20260709 12:15:46.831895 19184 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:46.837347 19174 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:46.775264 19231 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:46.837689 19308 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:46.860067 19286 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:46.891202 19268 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:46.906227 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:46.891726 19262 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.875196 19164 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:46.888891 11935 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:46.883644 19032 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:46.882066 19159 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.862845 19177 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.803094 19123 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.831080 19031 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.914710 11176 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:15:46.916922 11927 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:15:46.858589 18816 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.844542 19266 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:15:46.864682 19273 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.838044 18971 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:46.925186 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.926512 11176 pending_rounds.cc:85] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 16
I20260709 12:15:46.964744 11176 pending_rounds.cc:107] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:15:46.965124 11176 raft_consensus.cc:2889] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:46.930680 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62048.5us
I20260709 12:15:46.920114 19277 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:46.937279 19309 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.107s
I20260709 12:15:46.966321 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.966970 19309 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:46.967466 19309 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.967811 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:46.968407 19309 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.970022 19177 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:46.916105 19268 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.970674 19268 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:46.970758 19177 raft_consensus.cc:2764] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:15:46.942986 18816 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.976415 18816 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:46.977720 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.979326 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:46.980082 11254 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:46.906963 18804 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.920588 19308 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:46.957269 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:46.964057 19184 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:46.943449 18652 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:46.948592 18805 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:46.941818 11325 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:46.974305 18971 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:46.945405 18972 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.988909 19309 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:46.982396 19177 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:47.112979 19095 raft_consensus.cc:3055] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:46.945079 19231 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:46.999408 11173 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:15:46.943475 19265 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:46.946466 19153 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:47.091754 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.947130 19091 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.000524 18652 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:47.083328 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.948323 19289 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:46.922089 19262 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.000295 19293 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:15:47.172435 19320 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:47.207984 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.098991 19267 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.148875 11931 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:15:47.165207 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:47.166326 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:47.169019 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:47.171448 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 359815us
I20260709 12:15:47.172986 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.218928 11249 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:47.098831 18804 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.175796 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.177055 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:47.179232 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:47.180176 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:46.994874 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.185853 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:47.148636 19159 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:47.188093 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 193872us
I20260709 12:15:47.193364 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.193490 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.223521 11255 raft_consensus.cc:2368] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25.
I20260709 12:15:47.223711 11250 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:47.199476 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.171092 19201 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.223995 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:47.097720 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 118468us
W20260709 12:15:47.096925 19311 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.182s	user 0.001s	sys 0.004s
I20260709 12:15:47.214017 11252 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:47.219861 11934 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
I20260709 12:15:47.222167 11256 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:47.224969 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.229005 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:47.225818 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:47.188815 19095 raft_consensus.cc:740] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:47.230259 11257 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:15:47.232132 11248 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:47.232789 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.233570 18265 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:47.234009 18265 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:47.234385 18265 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:47.237586 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:47.243920 11147 leader_election.cc:400] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22.
I20260709 12:15:47.244274 11147 leader_election.cc:403] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:15:47.245095 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.243721 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:47.257768 19317 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.133s	user 0.001s	sys 0.004s
I20260709 12:15:47.242367 19193 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:47.260284 19289 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.259692 19095 consensus_queue.cc:260] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.261297 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3460) took 1722 ms. Trace:
I20260709 12:15:47.230902 19159 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.261828 11175 rpcz_store.cc:276] 0709 12:15:45.538633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.550341 (+ 11708us) service_pool.cc:224] Handling call
0709 12:15:45.550729 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:45.551123 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:45.551165 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:45.551750 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:15:46.592244 (+1040494us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b18003fc580. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:15:46.740387 (+148143us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:46.740427 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.740572 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:47.260957 (+520385us) raft_consensus.cc:1685] finished
0709 12:15:47.260980 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.261227 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16089,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":5847,"spinlock_wait_cycles":2876160,"thread_start_us":119418,"threads_started":1,"wal-append.queue_time_us":1282746}
I20260709 12:15:47.263777 19174 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:15:47.215349 18861 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.230065 19177 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.265570 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.274155 19177 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.265751 19098 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:15:47.276292 19177 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.265774 19308 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.234412 19201 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.227571 19267 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.279266 19201 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.224844 19274 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.280143 19274 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.265918 19188 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22.
I20260709 12:15:47.280732 18804 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:47.281283 19193 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:47.261909 19031 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.282300 19274 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:47.282685 19193 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.286700 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.287106 11250 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:47.287403 11173 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 29.
I20260709 12:15:47.267295 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.289827 19174 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:15:47.267313 19184 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.261230 19095 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:47.268013 19153 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:47.294265 19153 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:47.294684 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.295337 11248 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:47.295161 19153 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.273953 19032 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.268271 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:47.274507 11257 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:47.273546 11253 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:47.304836 18990 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:47.280512 19267 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.305251 19031 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:47.305395 19164 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.272328 18861 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.271108 19123 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.306563 18990 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:47.306502 19164 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.307266 18990 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:47.305271 19098 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:15:47.308702 19164 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.311867 11252 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:15:47.312152 19201 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.312639 19201 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.312875 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.313812 11250 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 29: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:47.270082 19006 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.002s
I20260709 12:15:47.314442 19006 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.315361 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1257) took 1315 ms. Trace:
I20260709 12:15:47.315747 11251 rpcz_store.cc:276] 0709 12:15:45.999635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.453236 (+453601us) service_pool.cc:224] Handling call
0709 12:15:46.453538 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:46.894761 (+441223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.315131 (+420370us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:47.315303 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":942715264}
I20260709 12:15:47.302729 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.314008 11255 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:15:47.325349 19188 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:47.305886 19159 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.269364 19273 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.271939 19268 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.326601 18862 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:47.327237 19201 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.269155 19091 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:47.270678 19262 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:47.329336 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.337946 11147 leader_election.cc:400] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22.
I20260709 12:15:47.329838 19091 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:47.340647 19091 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:47.343676 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 118023us
I20260709 12:15:47.343853 19320 raft_consensus.cc:2764] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:15:47.329783 18990 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.344623 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.345305 18862 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:47.346160 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.346570 18990 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:47.347404 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.319308 19031 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:47.349138 19285 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:47.347680 19320 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:47.351290 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.305297 19032 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.316591 19006 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:47.359344 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:47.324115 18265 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.328348 19268 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.367472 11252 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 26. Current term is 27. Ops: []
I20260709 12:15:47.370064 19268 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:47.376044 18265 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:47.385426 19031 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.365080 19285 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:47.391057 19285 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.395352 19046 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:47.396041 19046 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:47.367188 18652 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.399884 19090 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:47.358417 19274 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:47.413118 19282 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:47.414433 19282 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:15:47.415030 19032 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:47.415496 19032 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:47.415958 19032 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:47.417896 19274 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:47.419029 11181 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:15:47.419167 19274 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.423409 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.425129 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.426275 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1256) took 1426 ms. Trace:
I20260709 12:15:47.426893 11254 rpcz_store.cc:276] 0709 12:15:45.999380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.441596 (+442216us) service_pool.cc:224] Handling call
0709 12:15:46.777835 (+336239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:46.980563 (+202728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.425354 (+444791us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400278154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:47.426075 (+   721us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b5400278150. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:47.426212 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236956672}
I20260709 12:15:47.426376 18816 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:47.397303 19046 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.432785 11072 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 18, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 18 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:47.434439 11254 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:15:47.439775 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.440243 18816 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:47.414253 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 55018.4us
I20260709 12:15:47.467988 11072 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:47.470042 11181 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:47.470323 11147 leader_election.cc:400] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25.
I20260709 12:15:47.470654 11147 leader_election.cc:403] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:15:47.471684 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.472038 19262 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:47.472496 19113 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:47.474124 19113 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:47.471266 11072 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:47.476055 11072 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:47.477857 19006 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:47.478624 19006 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:47.414922 19282 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:47.480068 19006 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 24.11, Last appended by leader: 11, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.481364 19282 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:47.401017 18652 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:47.430543 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 79365.5us
I20260709 12:15:47.324602 18861 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:47.490484 11181 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:15:47.441586 11073 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 241) took 45288 ms (client timeout 15000 ms). Trace:
I20260709 12:15:47.459555 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:47.465543 18862 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:15:47.506091 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46662.3us
I20260709 12:15:47.507149 19262 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:15:47.507586 19262 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:47.507958 19262 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:15:47.482509 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.492201 19273 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:47.493717 19282 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:47.494899 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.495982 19046 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:47.527688 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49297.2us
I20260709 12:15:47.527276 19304 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:15:47.534240 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.537201 11329 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:15:47.538833 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.539481 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:47.546152 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.547156 11329 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:47.548416 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.549947 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.551045 11329 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:15:47.525302 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:47.491451 11249 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:15:47.613360 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51453.9us
I20260709 12:15:47.613065 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.614500 11329 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:47.614432 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.617286 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:47.618780 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:47.619944 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.620457 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.622231 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:47.615710 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 59492.4us
I20260709 12:15:47.624253 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.492190 19006 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:47.625368 11322 raft_consensus.cc:2410] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:15:47.626871 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:47.629235 11331 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:47.659276 11329 raft_consensus.cc:2410] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 24 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 12:15:47.664143 19321 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:15:47.665072 11329 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:15:47.625679 19304 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:47.665808 19304 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:47.666389 19304 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:47.667586 19293 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:47.668349 19293 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:47.669446 19293 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:47.670010 11329 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:15:47.671021 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 145831us
I20260709 12:15:47.672334 19123 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:47.677731 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:47.677934 19123 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:47.678375 19123 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:47.678577 19320 raft_consensus.cc:2764] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:15:47.672348 11324 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:15:47.673946 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.532490 18652 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:47.680500 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:47.681146 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61322.3us
W20260709 12:15:47.697418 11222 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.024s	sys 0.039s
W20260709 12:15:47.697835 11222 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.024s	sys 0.039s
W20260709 12:15:47.698096 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 997224us
W20260709 12:15:47.698796 11182 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.025s	sys 0.038s
W20260709 12:15:47.700601 11182 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.026s	sys 0.038s
I20260709 12:15:47.658782 19098 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 12:15:47.685837 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.701349 11328 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:47.568877 19113 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25.
I20260709 12:15:47.689368 11326 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:15:47.732079 11326 pending_rounds.cc:85] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 14
I20260709 12:15:47.732373 11326 pending_rounds.cc:107] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:15:47.732646 11326 raft_consensus.cc:2889] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:47.733309 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:47.497779 11073 rpcz_store.cc:269] 0709 12:15:02.153339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.157865 (+  4526us) service_pool.cc:224] Handling call
0709 12:15:47.441305 (+45283440us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:42.876745 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:42.993992 (+117247us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:42.994072 (+    80us) write_op.cc:435] Acquired schema lock
0709 12:15:42.994102 (+    30us) tablet.cc:662] Decoding operations
0709 12:15:43.000856 (+  6754us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:43.000975 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:43.001020 (+    45us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:15:43.002825 (+  1805us) tablet.cc:701] Row locks acquired
0709 12:15:43.002867 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:15:43.003083 (+   216us) write_op.cc:270] Start()
0709 12:15:43.039869 (+ 36786us) write_op.cc:276] Timestamp: P: 1783599343003007 usec, L: 0
0709 12:15:43.051778 (+ 11909us) op_driver.cc:348] REPLICATION: starting
0709 12:15:43.193516 (+141738us) log.cc:844] Serialized 7119 byte log entry
0709 12:15:43.738597 (+545081us) op_driver.cc:464] REPLICATION: finished
0709 12:15:43.740104 (+  1507us) write_op.cc:301] APPLY: starting
0709 12:15:43.740257 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:44.088220 (+347963us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:44.088397 (+   177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:44.501076 (+412679us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:44.501439 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:15:44.501513 (+    74us) write_op.cc:312] APPLY: finished
0709 12:15:44.612672 (+111159us) log.cc:844] Serialized 164 byte log entry
0709 12:15:44.612993 (+   321us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:44.613618 (+   625us) write_op.cc:454] Released schema lock
0709 12:15:44.613715 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:44.614243 (+   528us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:44.614274 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1396720,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":1203,"apply.run_cpu_time_us":36718,"apply.run_wall_time_us":874133,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79511,"num_ops":18,"prepare.queue_time_us":495080,"prepare.run_cpu_time_us":142899,"prepare.run_wall_time_us":732614,"raft.queue_time_us":34894,"raft.run_cpu_time_us":1491,"raft.run_wall_time_us":1491,"replication_time_us":685327,"thread_start_us":707882,"threads_started":4,"wal-append.queue_time_us":464840,"wal-append.run_cpu_time_us":15662,"wal-append.run_wall_time_us":1963225}]]}
I20260709 12:15:47.701285 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3467) took 1351 ms. Trace:
I20260709 12:15:47.734272 11182 rpcz_store.cc:276] 0709 12:15:46.349968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.586154 (+236186us) service_pool.cc:224] Handling call
0709 12:15:46.586472 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:46.586710 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:15:46.698589 (+111879us) log.cc:844] Serialized 14 byte log entry
0709 12:15:47.700892 (+1002303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:47.700948 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:47.700993 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.701055 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.701166 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1756,"prepare.queue_time_us":157899,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":50320,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2377,"thread_start_us":1108578,"threads_started":2}
I20260709 12:15:47.703163 11249 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:47.737140 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.703842 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.738214 11253 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:47.703893 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:47.685837 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.739423 11329 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:47.721097 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.740531 11252 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:47.742038 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.661712 19287 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:15:47.730329 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.050s	sys 0.108s
W20260709 12:15:47.759672 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.050s	sys 0.108s
I20260709 12:15:47.752588 11331 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:47.529965 18861 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.155s
I20260709 12:15:47.756767 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:47.776575 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.776852 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.779637 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.509505 19032 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:47.787356 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43962.3us
I20260709 12:15:47.686739 18652 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:47.866448 11175 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:47.866881 11255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.015s	sys 0.002s
W20260709 12:15:47.879102 11255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.016s	sys 0.002s
I20260709 12:15:47.789266 11179 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:15:47.788208 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:47.880483 11173 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:47.790534 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.881891 11256 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:47.882678 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:47.883096 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.025s
I20260709 12:15:47.822640 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:47.883634 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.000s	sys 0.026s
I20260709 12:15:47.883715 11325 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:15:47.681778 18805 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:47.662060 18816 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.129s
I20260709 12:15:47.874976 11182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:47.877080 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:47.910933 11182 raft_consensus.cc:606] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.911113 11933 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:47.911517 11182 raft_consensus.cc:614] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:47.879381 11180 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:15:47.850546 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:47.797046 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.915630 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.790194 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.917119 11254 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:47.898456 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:47.873385 11249 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:47.886005 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:47.919945 11324 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:47.739960 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.659092 19309 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.167s
I20260709 12:15:47.891145 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:47.874696 19174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.013s
I20260709 12:15:47.925719 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.929958 11331 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:47.930428 11176 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:47.930740 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:47.944797 11174 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:47.946071 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.767895 19277 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:47.947685 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:47.950589 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78612.4us
I20260709 12:15:47.954980 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.951252 11176 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:47.950002 11326 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:47.956683 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.957616 11327 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:47.961177 11180 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:47.962491 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.886382 18652 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:47.979743 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.983820 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:47.985486 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:47.986212 11249 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:47.989388 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105015us
I20260709 12:15:47.990010 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:47.980482 11178 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:47.995307 11173 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
W20260709 12:15:47.953042 19174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.006s	sys 0.013s
I20260709 12:15:47.996924 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:47.756989 19268 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.196s
I20260709 12:15:47.997710 11326 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:47.998550 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:47.897491 19113 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.005383 11256 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:47.953474 19277 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:47.791285 19265 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.198s
I20260709 12:15:47.899114 18805 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:48.017401 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:47.865464 18971 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.191s
I20260709 12:15:47.864814 19286 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.125s
I20260709 12:15:48.054037 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:48.012789 19308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.036s	sys 0.045s
I20260709 12:15:48.023254 18265 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:48.056722 19277 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:48.026543 19267 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:15:48.023058 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:48.025734 19201 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.757180 19320 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.245s
I20260709 12:15:48.062584 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:48.095723 11256 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:48.011178 11324 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:48.097667 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:48.021442 11331 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
W20260709 12:15:48.062754 19193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.006s	sys 0.035s
W20260709 12:15:48.071245 19188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.024s	sys 0.011s
W20260709 12:15:48.055714 11257 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.014s	sys 0.019s
W20260709 12:15:48.055546 18972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.017s	sys 0.021s
W20260709 12:15:48.059912 19187 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.029s	sys 0.056s
I20260709 12:15:48.110313 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:48.075243 19159 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:48.067368 19091 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.726s	user 0.008s	sys 0.004s
I20260709 12:15:48.054024 19113 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:48.106585 18972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.017s	sys 0.022s
I20260709 12:15:48.100988 19098 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:48.125810 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:48.058794 18642 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:48.136695 11256 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:48.118427 19031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.010s	sys 0.033s
I20260709 12:15:48.151965 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:48.075692 11072 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 12 to 19, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 19 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:48.156376 11072 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 22, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 22 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:48.159871 19282 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.631s	user 0.006s	sys 0.003s
I20260709 12:15:48.160987 11935 raft_consensus.cc:2410] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:15:48.161087 11072 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:48.162166 19337 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:48.162883 11072 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 17 to 18, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 18 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:48.067886 18805 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:48.167145 19113 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:48.168403 11248 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:48.131119 19098 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:48.172051 19098 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:48.172802 19090 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.771s	user 0.006s	sys 0.001s
I20260709 12:15:48.166872 19304 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:48.151499 19031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.011s	sys 0.033s
I20260709 12:15:48.174592 19032 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:48.175341 11250 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 12:15:48.103041 11257 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.014s	sys 0.020s
W20260709 12:15:48.075683 19308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.040s	sys 0.045s
I20260709 12:15:48.012374 19177 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:15:48.086088 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:48.102432 19193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.006s	sys 0.035s
I20260709 12:15:48.120054 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:48.120436 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:48.229713 11325 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:15:48.093276 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:48.102682 19188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.024s	sys 0.012s
W20260709 12:15:48.114697 19187 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.029s	sys 0.056s
I20260709 12:15:48.130694 11249 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:48.122120 19159 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.151139 18990 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.125471 19091 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.163638 19337 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:48.176378 11329 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:48.100735 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:48.266187 11331 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:48.098526 11324 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:48.101639 19285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.000s	sys 0.018s
W20260709 12:15:48.149432 19262 consensus_meta.cc:287] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
W20260709 12:15:48.153192 19266 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.011s	sys 0.012s
W20260709 12:15:48.118659 19153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.024s	sys 0.019s
I20260709 12:15:48.250334 11327 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:48.114106 19201 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:48.311419 19159 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:48.215430 19031 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:48.305263 19304 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:48.312233 19031 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.314549 19031 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:48.280771 19337 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.318424 19337 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:48.275673 19285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.001s	sys 0.018s
I20260709 12:15:48.325017 19091 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:48.297755 19266 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.011s	sys 0.016s
W20260709 12:15:48.326957 19046 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.797s	user 0.009s	sys 0.000s
I20260709 12:15:48.326498 11072 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 19 to 20, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 20 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:48.329244 11325 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
W20260709 12:15:48.281204 18862 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.000s
I20260709 12:15:48.332221 11326 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:15:48.283154 19274 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.004s	sys 0.026s
W20260709 12:15:48.338603 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.016s	sys 0.028s
W20260709 12:15:48.348908 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.017s	sys 0.029s
W20260709 12:15:48.332705 19153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.025s	sys 0.020s
W20260709 12:15:48.353294 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 693488us
I20260709 12:15:48.354260 11326 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:15:48.349606 11323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.000s	sys 0.022s
W20260709 12:15:48.357529 11323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.000s	sys 0.023s
W20260709 12:15:48.359547 19329 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:15:48.360978 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3120) took 1673 ms. Trace:
I20260709 12:15:48.361423 11330 rpcz_store.cc:276] 0709 12:15:46.687824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.915654 (+227830us) service_pool.cc:224] Handling call
0709 12:15:46.924584 (+  8930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:46.948956 (+ 24372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:46.948990 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:46.949567 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 12:15:47.760225 (+810658us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:47.760268 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.760339 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:48.360680 (+600341us) raft_consensus.cc:1685] finished
0709 12:15:48.360704 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:48.360885 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16281,"raft.queue_time_us":17206,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":770412,"threads_started":1,"wal-append.queue_time_us":1161067}
I20260709 12:15:48.257771 18990 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:48.386334 11251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.005s	sys 0.010s
W20260709 12:15:48.346633 19274 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.004s	sys 0.027s
I20260709 12:15:48.346575 18862 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:15:48.411670 11180 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:15:48.298447 19262 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:48.413437 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.368180 19277 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.348766 18652 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.398799 19193 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.354470 18805 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:48.487248 11926 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:15:48.488591 11322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.000s	sys 0.015s
I20260709 12:15:48.392942 19046 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.089s
W20260709 12:15:48.508816 11934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.026s	sys 0.044s
W20260709 12:15:48.493536 11251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.005s	sys 0.010s
W20260709 12:15:48.502286 11322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.000s	sys 0.016s
I20260709 12:15:48.486860 19308 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:48.509445 11934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.026s	sys 0.044s
W20260709 12:15:48.522297 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.522617 11176 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 27: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:48.496557 18972 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:48.495373 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.002s	sys 0.009s
W20260709 12:15:48.525871 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.004s	sys 0.012s
W20260709 12:15:48.529461 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.004s	sys 0.012s
W20260709 12:15:48.505671 19287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.017s	sys 0.014s
W20260709 12:15:48.527887 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.002s	sys 0.010s
W20260709 12:15:48.542021 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 611323us
W20260709 12:15:48.550915 11255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.009s	sys 0.014s
I20260709 12:15:48.497356 19193 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.329489 11072 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 17 to 20, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 20 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:48.564620 11252 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.013s
I20260709 12:15:48.575716 11252 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:48.566951 11331 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:48.578987 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:48.579972 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:48.577466 11324 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
W20260709 12:15:48.559757 19287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.018s	sys 0.014s
I20260709 12:15:48.584905 11324 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:48.584969 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:48.585254 11251 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:15:48.592443 11325 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:48.599259 11324 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:48.582024 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:48.628485 11331 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:48.587847 11252 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
W20260709 12:15:48.588918 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.587280 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:48.633000 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45864.6us
I20260709 12:15:48.524298 19308 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:48.634433 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:48.669870 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:48.642969 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:48.571776 19177 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.645987 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:48.697518 11324 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.647814 11327 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:48.700497 11324 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:48.700918 11326 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:48.672999 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:48.701838 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55944.4us
I20260709 12:15:48.703933 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:48.624568 19320 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.004s	sys 0.011s
W20260709 12:15:48.704551 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.702400 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:48.592064 19274 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:48.724731 11324 raft_consensus.cc:1275] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:15:48.725370 11324 pending_rounds.cc:85] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 11
I20260709 12:15:48.545470 18972 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:48.743779 18972 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:48.744055 19123 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.049s	user 0.021s	sys 0.007s
I20260709 12:15:48.728012 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:48.734627 18805 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:48.734897 11324 pending_rounds.cc:107] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:15:48.745966 11324 raft_consensus.cc:2889] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:48.746025 11180 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:15:48.670956 19091 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:48.731411 11223 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:48.713924 19320 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.006s	sys 0.013s
I20260709 12:15:48.635845 19285 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.740312 11072 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 19 to 20, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:48.715615 19031 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:15:48.726459 19340 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:15:48.633708 19032 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.697791 19231 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:48.591683 19287 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:48.745388 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3124) took 1201 ms. Trace:
I20260709 12:15:48.801070 11322 rpcz_store.cc:276] 0709 12:15:47.544111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.680179 (+136068us) service_pool.cc:224] Handling call
0709 12:15:47.732666 (+ 52487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:47.733112 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:47.733164 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:47.733751 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:15:48.515712 (+781961us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:48.515758 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:48.515835 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:48.745129 (+229294us) raft_consensus.cc:1685] finished
0709 12:15:48.745152 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:48.745328 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2012,"raft.queue_time_us":1195,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":43919,"thread_start_us":632444,"threads_started":1,"wal-append.queue_time_us":924895}
W20260709 12:15:48.749209 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.009s	sys 0.021s
W20260709 12:15:48.804739 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.009s	sys 0.021s
W20260709 12:15:48.808488 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 818467us
I20260709 12:15:48.750546 19091 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:48.791069 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87208.9us
W20260709 12:15:48.779250 11253 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.008s	sys 0.033s
W20260709 12:15:48.820129 11253 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.009s	sys 0.033s
W20260709 12:15:48.794222 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.011s	sys 0.022s
I20260709 12:15:48.729413 19159 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:15:48.751137 19184 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.045s	sys 0.039s
W20260709 12:15:48.791134 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.015s	sys 0.018s
I20260709 12:15:48.830521 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:48.755371 19293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.006s	sys 0.019s
W20260709 12:15:48.832196 19293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.007s	sys 0.020s
W20260709 12:15:48.817458 11295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.006s	sys 0.062s
I20260709 12:15:48.818092 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1476) took 1118 ms. Trace:
I20260709 12:15:48.833091 11926 rpcz_store.cc:276] 0709 12:15:47.699701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.699851 (+   150us) service_pool.cc:224] Handling call
0709 12:15:47.700223 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.495033 (+794810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:48.814573 (+319540us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400555354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:48.818000 (+  3427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773053184}
W20260709 12:15:48.834621 11295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.007s	sys 0.062s
W20260709 12:15:48.820982 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.141s	user 0.011s	sys 0.023s
I20260709 12:15:48.729871 19286 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:15:48.837026 19286 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:48.830986 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.015s	sys 0.018s
I20260709 12:15:48.675124 19046 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:15:48.840319 11933 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 28. Current term is 30. Ops: []
I20260709 12:15:48.840745 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:48.842124 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:48.842949 11935 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:48.844861 11901 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:48.846122 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:48.847010 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:48.846131 18972 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:48.848063 18972 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:48.848553 18972 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:48.812920 19265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.010s	sys 0.020s
I20260709 12:15:48.709832 19177 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:48.841571 11926 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
W20260709 12:15:48.854525 11898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.014s	sys 0.002s
W20260709 12:15:48.890736 11898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.014s	sys 0.003s
W20260709 12:15:48.829239 19343 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.344s	user 0.001s	sys 0.000s
I20260709 12:15:48.799388 19262 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:48.886682 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:48.816702 19320 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.747354 19267 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:15:48.796496 18990 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:48.892480 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.010s	sys 0.031s
I20260709 12:15:48.811903 19277 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
W20260709 12:15:48.829777 19184 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.046s	sys 0.039s
I20260709 12:15:48.826423 19091 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:48.849982 19265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.010s	sys 0.020s
I20260709 12:15:48.816941 19362 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:48.849200 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:48.900383 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:48.791015 19285 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:48.902894 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.010s	sys 0.031s
I20260709 12:15:48.800210 19032 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:48.923635 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 1.1837e+06us
I20260709 12:15:48.753151 19274 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.756242 19123 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.891844 19293 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:48.897689 19345 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 12:15:48.893133 18652 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:48.934904 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 85778.3us
I20260709 12:15:48.888103 19153 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:48.947502 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1273) took 1598 ms. Trace:
W20260709 12:15:48.948254 18265 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.019s
W20260709 12:15:48.951086 18265 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.007s	sys 0.020s
W20260709 12:15:48.911159 19006 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.230s	user 0.007s	sys 0.018s
I20260709 12:15:48.854481 19286 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:48.919487 18990 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:48.987597 11181 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:48.949781 11255 rpcz_store.cc:276] 0709 12:15:47.348781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.348935 (+   154us) service_pool.cc:224] Handling call
0709 12:15:47.349345 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:47.349770 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:15:47.350105 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 12:15:47.402619 (+ 52514us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b18004e3340. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:48.551430 (+1148811us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:48.574893 (+ 23463us) log.cc:844] Serialized 52 byte log entry
0709 12:15:48.575125 (+   232us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:48.575173 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:48.575258 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:48.938545 (+363287us) raft_consensus.cc:1685] finished
0709 12:15:48.947200 (+  8655us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:48.947419 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142796,"prepare.queue_time_us":3004,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":143730,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":7069,"spinlock_wait_cycles":3586432,"thread_start_us":1057902,"threads_started":2,"wal-append.queue_time_us":896919}
I20260709 12:15:48.905599 19355 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:48.939175 19293 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:48.939638 19348 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.999341 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:48.997097 18265 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:15:48.962641 19266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.014s	sys 0.021s
I20260709 12:15:48.908869 19320 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.020572 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:48.989174 19352 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.004051 19369 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.039990 19369 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:48.986874 19359 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:15:49.026626 19266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.015s	sys 0.021s
I20260709 12:15:49.002938 19277 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:49.023996 18652 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.022799 19293 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:49.048282 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 316989us
I20260709 12:15:49.026692 19360 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.052366 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:49.056794 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:49.030361 19370 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.064422 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:49.027011 18990 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:49.041688 19098 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.869s	user 0.013s	sys 0.000s
I20260709 12:15:49.066910 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:48.986937 19153 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.045305 19369 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:49.075554 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:49.050086 19193 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:49.063421 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:49.052891 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.050297 19277 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:49.073680 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39356 (request call id 3130) took 1191 ms. Trace:
I20260709 12:15:48.948597 19287 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:49.062758 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.083455 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:49.095499 11180 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 30.
I20260709 12:15:49.095871 11324 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
W20260709 12:15:49.068631 19350 log.cc:927] Time spent T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 12:15:49.080741 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:49.084956 11329 rpcz_store.cc:276] 0709 12:15:47.882050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.882433 (+   383us) service_pool.cc:224] Handling call
0709 12:15:49.073512 (+1191079us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54002f76d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:49.073611 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673953152}
W20260709 12:15:49.064024 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 394272us
I20260709 12:15:49.099814 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:49.100090 11296 leader_election.cc:400] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:49.100394 11296 leader_election.cc:403] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:49.026265 19355 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:49.088930 11327 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:49.065804 19266 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:49.096992 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:49.097937 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:49.069427 19308 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.102764 11251 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:49.069681 19265 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.029532 19348 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.104545 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:49.087046 18642 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:49.111851 11322 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:49.103117 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:49.115006 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:49.120416 11329 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:49.114144 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:49.109948 11327 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:48.931243 11072 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 20 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 23 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:49.047610 19352 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.064200 18652 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:49.125098 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:49.135041 19177 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:49.135880 19369 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:49.109124 11180 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:49.146756 19348 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:49.150719 18652 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.153121 19277 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:49.114511 19362 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21.
I20260709 12:15:49.154719 19286 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:49.114118 19373 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:15:49.077775 19098 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:49.116191 19349 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.326s	user 0.001s	sys 0.001s
I20260709 12:15:49.234473 19031 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:49.068490 19360 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:49.239967 19031 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.146605 19320 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:49.164098 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3503) took 1277 ms. Trace:
I20260709 12:15:49.107159 19266 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:49.119125 19265 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:49.191763 19361 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 12:15:49.145699 19268 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:49.149554 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3474) took 1447 ms. Trace:
I20260709 12:15:49.108283 19366 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:49.170742 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.148422 19153 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:49.077943 19370 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:49.183223 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58217.1us
I20260709 12:15:49.315805 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:49.191633 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:49.146651 19359 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:15:49.181715 19342 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:15:49.242398 11326 rpcz_store.cc:276] 0709 12:15:47.886627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.355558 (+468931us) service_pool.cc:224] Handling call
0709 12:15:48.700752 (+345194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.701441 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.163806 (+462365us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54001c5cd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:49.164002 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793055488}
I20260709 12:15:49.131747 18816 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:49.166064 19231 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:49.190932 19090 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:15:49.285948 11934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.020s	sys 0.057s
W20260709 12:15:49.291200 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 480636us
W20260709 12:15:49.292060 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 347829us
I20260709 12:15:49.190812 19177 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:49.272684 11181 rpcz_store.cc:276] 0709 12:15:47.702231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.741723 (+ 39492us) service_pool.cc:224] Handling call
0709 12:15:49.149305 (+1407582us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54000da054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:49.149469 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2862883072}
W20260709 12:15:49.309938 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 277789us
I20260709 12:15:49.264488 19286 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.236375 18990 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.267639 19355 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:49.268018 19266 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:49.533286 11251 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:49.635649 11901 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:49.495653 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:49.511539 11182 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.045s	user 0.017s	sys 0.024s
W20260709 12:15:49.569108 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:49.384267 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68524.5us
I20260709 12:15:49.562704 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:49.541409 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3269) took 1817 ms. Trace:
I20260709 12:15:49.269383 19277 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:49.647228 11934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.032s	sys 0.078s
I20260709 12:15:49.419199 11173 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:15:49.418536 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3482) took 1396 ms. Trace:
I20260709 12:15:49.511969 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1555) took 1970 ms. Trace:
I20260709 12:15:49.269805 19123 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.697429 11253 rpcz_store.cc:276] 0709 12:15:47.724166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.772908 (+ 48742us) service_pool.cc:224] Handling call
0709 12:15:47.773364 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:47.773672 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:48.826576 (+1052904us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.518504 (+691928us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:49.518584 (+    80us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:49.518651 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.518781 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.541315 (+ 22534us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36462,"prepare.queue_time_us":207125,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"thread_start_us":1701476,"threads_started":3}
W20260709 12:15:49.700575 11252 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.003s	sys 0.011s
I20260709 12:15:49.697436 11323 rpcz_store.cc:276] 0709 12:15:47.541138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.541252 (+   114us) service_pool.cc:224] Handling call
0709 12:15:47.623847 (+ 82595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:47.624257 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:15:47.677905 (+ 53648us) log.cc:844] Serialized 14 byte log entry
0709 12:15:48.358115 (+680210us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:48.358675 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:15:48.358813 (+   138us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:48.358849 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:48.358931 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:49.511696 (+1152765us) raft_consensus.cc:1685] finished
0709 12:15:49.511721 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.511891 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":54610,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":191864,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":26645,"thread_start_us":679778,"threads_started":1,"wal-append.queue_time_us":1020717}
W20260709 12:15:49.632725 11175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.008s	sys 0.012s
W20260709 12:15:49.702690 11175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.008s	sys 0.012s
W20260709 12:15:49.701006 11252 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.003s	sys 0.011s
W20260709 12:15:49.706655 11182 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.027s	sys 0.029s
I20260709 12:15:49.699398 11176 rpcz_store.cc:276] 0709 12:15:48.022519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.022804 (+   285us) service_pool.cc:224] Handling call
0709 12:15:49.418302 (+1395498us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54000c0b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:49.418472 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980476288}
I20260709 12:15:49.712848 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:49.508571 19368 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.445s	user 0.001s	sys 0.001s
W20260709 12:15:49.398042 19361 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:15:49.516659 18972 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.000s
I20260709 12:15:49.697916 18652 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.718825 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:49.745532 11251 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:15:49.719216 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.011s
I20260709 12:15:49.720721 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:49.746317 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.003s	sys 0.011s
I20260709 12:15:49.746686 11256 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:49.266911 19159 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:49.744732 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:49.727632 11249 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:49.407938 19369 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.391088 19362 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:49.398222 19274 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:49.426257 19372 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:15:49.406535 19389 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:49.734663 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:49.395521 19391 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:49.741705 19268 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:49.407101 19285 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.645780 19177 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:49.716972 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 81411.3us
I20260709 12:15:49.732295 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:49.518832 19205 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:49.529700 19174 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.018s	sys 0.023s
I20260709 12:15:49.743696 19304 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 12:15:49.391559 19360 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.424401 19201 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:49.710731 19282 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:49.769960 18265 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.887610 18265 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:49.730031 19393 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.158s
I20260709 12:15:49.823228 11933 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:49.727859 19262 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:49.828809 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:49.829924 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:49.318804 19366 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.699617 19286 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:49.912700 11930 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:15:49.831591 11180 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
I20260709 12:15:49.913512 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1367) took 1517 ms. Trace:
W20260709 12:15:49.833928 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.914196 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:49.836089 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 189937us
I20260709 12:15:49.731029 19352 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.831914 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:49.834782 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:49.874349 19177 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:15:49.793964 18642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.004s	sys 0.026s
I20260709 12:15:49.858150 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1364) took 1482 ms. Trace:
I20260709 12:15:49.764820 18652 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.921464 11934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1475) took 2214 ms. Trace:
I20260709 12:15:49.923411 11182 rpcz_store.cc:276] 0709 12:15:48.375427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.377485 (+  2058us) service_pool.cc:224] Handling call
0709 12:15:48.412978 (+ 35493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.413240 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:49.707258 (+1294018us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.857280 (+150022us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:49.857418 (+   138us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:49.857556 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.857764 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.858049 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67491,"prepare.queue_time_us":633548,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2868,"raft.queue_time_us":26243,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":14837,"thread_start_us":1372380,"threads_started":2}
I20260709 12:15:49.854504 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:49.831209 19361 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:15:49.746768 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 630037us
I20260709 12:15:49.928781 11933 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:15:49.884593 19375 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.483s	user 0.000s	sys 0.001s
W20260709 12:15:49.884943 11220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.010s	sys 0.007s
W20260709 12:15:49.931480 11220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.010s	sys 0.007s
W20260709 12:15:49.931937 11331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.009s	sys 0.017s
I20260709 12:15:49.888662 11253 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
W20260709 12:15:49.812882 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.838591 19098 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:49.886453 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:49.871234 19174 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.347s	user 0.028s	sys 0.029s
W20260709 12:15:49.895140 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 162947us
I20260709 12:15:49.845104 19389 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:49.907392 19282 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.908186 18971 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:49.908847 19366 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:49.890288 19231 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:49.404511 11072 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 22 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 25 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:49.910362 19367 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.470s	user 0.000s	sys 0.001s
I20260709 12:15:49.885293 19403 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:49.911955 19382 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:15:49.749963 18972 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.844316 19391 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:49.904379 11251 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:15:49.917140 11176 raft_consensus.cc:2368] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26.
I20260709 12:15:49.842238 11249 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 27 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:15:50.019294 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:49.752166 19153 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.913942 11179 rpcz_store.cc:276] 0709 12:15:48.396113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.396346 (+   233us) service_pool.cc:224] Handling call
0709 12:15:48.396822 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:48.397359 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:48.911730 (+514371us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.394906 (+483176us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:49.413046 (+ 18140us) log.cc:844] Serialized 26 byte log entry
0709 12:15:49.413229 (+   183us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:49.413269 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.413344 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:49.913310 (+499966us) raft_consensus.cc:1685] finished
0709 12:15:49.913332 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.913451 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195170,"prepare.queue_time_us":649065,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":597,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":27480,"thread_start_us":866028,"threads_started":2,"wal-append.queue_time_us":781310}
I20260709 12:15:49.866515 19265 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:49.921464 11926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1479) took 1018 ms. Trace:
I20260709 12:15:49.914839 11180 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
W20260709 12:15:49.924817 18642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.004s	sys 0.026s
I20260709 12:15:49.924784 11934 rpcz_store.cc:276] 0709 12:15:47.699300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.700259 (+   959us) service_pool.cc:224] Handling call
0709 12:15:47.865683 (+165424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:48.523998 (+658315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:48.524036 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:48.545062 (+ 21026us) log.cc:844] Serialized 26 byte log entry
0709 12:15:49.698095 (+1153033us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:15:49.698153 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.698768 (+   615us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:49.913991 (+215223us) raft_consensus.cc:1685] finished
0709 12:15:49.914009 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.914124 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":583,"raft.queue_time_us":311362,"raft.run_cpu_time_us":6757,"raft.run_wall_time_us":40658,"thread_start_us":1759652,"threads_started":2,"wal-append.queue_time_us":1342191}
I20260709 12:15:49.824594 19369 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.883339 19159 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:49.928391 19379 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.295s
I20260709 12:15:49.963472 18971 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:49.956863 18265 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.916538 19123 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:49.917199 19201 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:49.966444 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:50.054523 11900 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:15:49.974274 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 87929.2us
I20260709 12:15:49.909595 19388 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:15:50.056149 19201 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.057104 19032 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:49.973531 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:50.007212 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.058048 11256 raft_consensus.cc:2368] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26.
I20260709 12:15:50.058317 19201 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.058900 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.024200 18642 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:50.009503 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:49.894471 19205 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.060228 18642 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.063210 19393 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:15:50.019271 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.057183 19388 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 12:15:49.917645 11181 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:50.006744 19362 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:50.034920 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:49.933040 11331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.009s	sys 0.018s
I20260709 12:15:49.941373 11248 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:15:50.021255 11926 rpcz_store.cc:276] 0709 12:15:48.894905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.895051 (+   146us) service_pool.cc:224] Handling call
0709 12:15:48.903418 (+  8367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:49.292969 (+389551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:49.293017 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:49.337354 (+ 44337us) log.cc:844] Serialized 26 byte log entry
0709 12:15:49.718978 (+381624us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:49.719020 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.719105 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:49.912963 (+193858us) raft_consensus.cc:1685] finished
0709 12:15:49.912984 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.913111 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145152,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4880,"thread_start_us":675953,"threads_started":2,"wal-append.queue_time_us":516127}
I20260709 12:15:50.047531 11935 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:49.991890 19348 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.964452 19389 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:50.022933 19352 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:50.056201 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:50.058931 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:50.022969 11251 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:50.055040 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3127) took 2320 ms. Trace:
I20260709 12:15:50.065289 11249 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: [24.12-26.13]
I20260709 12:15:50.064843 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:50.024780 19391 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:50.006237 18805 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:15:50.033092 19098 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:50.061391 19405 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:50.036484 19153 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.025506 19403 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.286695 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.287554 11324 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 20.
W20260709 12:15:50.287730 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 222979us
I20260709 12:15:50.187623 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1365) took 1811 ms. Trace:
I20260709 12:15:50.288780 11173 rpcz_store.cc:276] 0709 12:15:48.375773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.377588 (+  1815us) service_pool.cc:224] Handling call
0709 12:15:49.418934 (+1041346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:49.697578 (+278644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.187327 (+489749us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000c0b50. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:50.187534 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2314352896}
I20260709 12:15:50.214615 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.249513 11329 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:15:50.291605 18265 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.286716 11256 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:50.170246 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 229305us
I20260709 12:15:50.049618 19188 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.299643 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.301440 11330 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:15:50.302565 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:50.193161 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1370) took 1135 ms. Trace:
I20260709 12:15:50.306061 11177 rpcz_store.cc:276] 0709 12:15:49.057559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.057774 (+   215us) service_pool.cc:224] Handling call
0709 12:15:49.058139 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:49.106778 (+ 48639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:49.106834 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:49.106880 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:49.106909 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.192900 (+1085991us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b54002e4ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:50.192948 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.193096 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111301376}
I20260709 12:15:50.306629 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.307686 11324 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:50.307730 11330 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:15:50.310259 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 254127us
I20260709 12:15:50.032811 19274 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:50.071889 19385 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.645s	user 0.001s	sys 0.000s
W20260709 12:15:50.318343 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 283471us
I20260709 12:15:50.318773 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:50.320963 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:50.292187 18265 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.322642 11179 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:15:50.054638 19123 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.065199 19392 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:15:50.205935 19358 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:15:50.221553 19397 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.470s	user 0.001s	sys 0.000s
W20260709 12:15:50.069113 19091 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.139s	user 0.007s	sys 0.005s
I20260709 12:15:50.327142 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:50.291129 11328 rpcz_store.cc:276] 0709 12:15:47.734925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.735063 (+   138us) service_pool.cc:224] Handling call
0709 12:15:47.863731 (+128668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:47.864083 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:15:47.954504 (+ 90421us) log.cc:844] Serialized 14 byte log entry
0709 12:15:48.527047 (+572543us) log.cc:844] Serialized 14 byte log entry
0709 12:15:48.527237 (+   190us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:48.537791 (+ 10554us) log.cc:844] Serialized 26 byte log entry
0709 12:15:48.538021 (+   230us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:48.538070 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:48.538174 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:50.054811 (+1516637us) raft_consensus.cc:1685] finished
0709 12:15:50.054838 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.054978 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6279,"prepare.queue_time_us":149157,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":115,"raft.queue_time_us":215942,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":61117,"thread_start_us":449742,"threads_started":1,"wal-append.queue_time_us":736294}
I20260709 12:15:50.326525 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:50.265924 19404 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.378s	user 0.002s	sys 0.006s
I20260709 12:15:50.304944 19370 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.376250 11249 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:50.325902 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.070344 19286 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:50.284628 19357 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.374s
I20260709 12:15:50.063951 19205 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.346304 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:50.318491 19046 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:50.401958 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:50.371073 11326 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:50.371407 11329 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:50.336184 11935 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:50.372545 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:50.405666 11934 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:50.378564 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:50.373693 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:50.377605 11322 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:50.331780 19265 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:50.284845 19409 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.306s
I20260709 12:15:50.337687 19123 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:50.418761 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:50.310369 19098 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.419713 11927 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 30 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 12:15:50.384783 11256 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
W20260709 12:15:50.386281 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92249us
W20260709 12:15:50.423034 11257 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.040s
I20260709 12:15:50.312002 19348 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.402868 11932 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:50.204878 19337 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:50.347785 18816 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:50.412482 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:50.374595 19309 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:50.337947 19366 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 12:15:50.424043 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:50.369935 19268 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:50.323880 19188 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:50.426875 11257 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.006s	sys 0.040s
I20260709 12:15:50.375918 19355 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:50.372633 19287 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:50.425077 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:50.401790 19385 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:50.338121 19388 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:15:50.327603 19274 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.347532 19430 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:50.452646 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:50.310618 19405 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.347939 19308 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 12:15:50.407886 19282 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:50.413795 19006 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 12:15:50.444329 19337 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:50.385334 18805 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.071s
W20260709 12:15:50.452273 19397 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:15:50.545469 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 120477us
I20260709 12:15:50.547101 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1363) took 2172 ms. Trace:
I20260709 12:15:50.547439 11174 rpcz_store.cc:276] 0709 12:15:48.374965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.376366 (+  1401us) service_pool.cc:224] Handling call
0709 12:15:48.376817 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:48.377340 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:48.916517 (+539177us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.184292 (+267775us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.184573 (+   281us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:49.185145 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:15:49.185316 (+   171us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:49.185365 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.185474 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:49.418533 (+233059us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b18000c5020. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:49.602319 (+183786us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b48000030d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:15:50.546877 (+944558us) raft_consensus.cc:1685] finished
0709 12:15:50.546901 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.547025 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":197371,"prepare.queue_time_us":652990,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":6438,"raft.queue_time_us":13491,"raft.run_cpu_time_us":3781,"raft.run_wall_time_us":523685,"spinlock_wait_cycles":117823744,"thread_start_us":647570,"threads_started":2,"wal-append.queue_time_us":644102}
I20260709 12:15:50.381242 19091 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.551359 19090 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:50.551906 19090 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:50.552618 18972 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.524039 19268 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:50.554410 19415 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:15:50.554472 19403 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.512257 11931 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
W20260709 12:15:50.522341 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 121387us
I20260709 12:15:50.557662 11934 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:50.525333 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:50.539361 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:50.507566 19285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.011s	sys 0.070s
I20260709 12:15:50.369462 18971 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.523177 11932 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:15:50.453424 18816 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.516420 19287 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:50.516650 19355 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:50.543826 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:50.548635 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:50.538201 11927 raft_consensus.cc:2410] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:15:50.573050 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.461751 19392 raft_consensus.cc:697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:50.347867 19362 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.508664 19433 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.528301 19369 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.559737 19006 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:50.455873 19286 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:50.580049 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.423885 19412 consensus_queue.cc:1048] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.147s
I20260709 12:15:50.583958 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3513) took 2023 ms. Trace:
I20260709 12:15:50.607249 11322 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:50.607388 11331 rpcz_store.cc:276] 0709 12:15:48.560745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.672093 (+111348us) service_pool.cc:224] Handling call
0709 12:15:48.672409 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.672607 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:48.996869 (+324262us) log.cc:844] Serialized 14 byte log entry
0709 12:15:50.583464 (+1586595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:50.583544 (+    80us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:50.583610 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.583704 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.583880 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64569,"prepare.queue_time_us":72495,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":115699,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5854,"thread_start_us":1833968,"threads_started":3,"wal-append.queue_time_us":1188220}
I20260709 12:15:50.609709 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:50.610512 11331 raft_consensus.cc:2368] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26.
W20260709 12:15:50.561069 19285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.011s	sys 0.070s
I20260709 12:15:50.569988 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:50.596802 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:50.560420 19430 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:50.574942 19355 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:50.568912 18816 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:50.617650 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1481) took 1697 ms. Trace:
I20260709 12:15:50.612703 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:50.587061 18971 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:50.616349 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:50.622449 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 97219.2us
I20260709 12:15:50.585868 19403 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:50.585983 19379 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:50.633780 19448 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:50.634204 19379 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.634845 19188 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.635927 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3511) took 2339 ms. Trace:
I20260709 12:15:50.604519 19006 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:50.629050 19428 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:15:50.603528 19433 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:50.629778 19425 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.420s	user 0.000s	sys 0.001s
I20260709 12:15:50.629993 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:50.630760 19098 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.631923 19392 consensus_queue.cc:237] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:50.623649 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53741.2us
I20260709 12:15:50.686666 11322 raft_consensus.cc:2376] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22. Re-sending same reply.
I20260709 12:15:50.628636 19386 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:15:50.618137 11929 rpcz_store.cc:276] 0709 12:15:48.920467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.920657 (+   190us) service_pool.cc:224] Handling call
0709 12:15:48.952901 (+ 32244us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:49.312589 (+359688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:49.313049 (+   460us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:49.382069 (+ 69020us) log.cc:844] Serialized 52 byte log entry
0709 12:15:49.821873 (+439804us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:50.325590 (+503717us) log.cc:844] Serialized 14 byte log entry
0709 12:15:50.344551 (+ 18961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.344629 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:50.617317 (+272688us) raft_consensus.cc:1685] finished
0709 12:15:50.617347 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.617551 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10983,"prepare.queue_time_us":93848,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":203350,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":44808,"thread_start_us":1179074,"threads_started":3,"wal-append.queue_time_us":894035}
I20260709 12:15:50.380789 11072 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 9 to 21, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 21 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:50.616523 19293 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.547s	user 0.015s	sys 0.000s
I20260709 12:15:50.620729 19409 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:15:50.634951 19406 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 12:15:50.694378 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:50.619951 19337 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.695940 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1371) took 1637 ms. Trace:
I20260709 12:15:50.696372 19337 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:50.696333 11175 rpcz_store.cc:276] 0709 12:15:49.059049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.059233 (+   184us) service_pool.cc:224] Handling call
0709 12:15:49.059637 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:49.067974 (+  8337us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:49.098471 (+ 30497us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.718092 (+619621us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:49.728830 (+ 10738us) log.cc:844] Serialized 26 byte log entry
0709 12:15:49.729030 (+   200us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:49.729068 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.729134 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:50.348216 (+619082us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b1800493780. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:50.695618 (+347402us) raft_consensus.cc:1685] finished
0709 12:15:50.695720 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.695877 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49227,"prepare.queue_time_us":35525,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":49359,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":18534,"spinlock_wait_cycles":6366464,"thread_start_us":575012,"threads_started":1,"wal-append.queue_time_us":1104809}
I20260709 12:15:50.693784 19293 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:50.623823 19423 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.346s	user 0.000s	sys 0.001s
W20260709 12:15:50.629060 19431 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:15:50.662204 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:50.633852 19403 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:50.700059 19293 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.700256 19403 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:50.700974 19426 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:50.701751 19440 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:50.702320 19440 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.703984 19091 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.633775 19434 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:50.655601 19285 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:15:50.677374 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:50.709424 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:50.654847 19268 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.617136 19286 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.725597 11326 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:15:50.561192 19193 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
W20260709 12:15:50.684653 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 71933.2us
I20260709 12:15:50.663329 18861 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:50.689237 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:50.730777 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:50.685122 11325 rpcz_store.cc:276] 0709 12:15:48.296213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.593173 (+296960us) service_pool.cc:224] Handling call
0709 12:15:48.628712 (+ 35539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:48.671409 (+ 42697us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:48.727099 (+ 55690us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.073293 (+346194us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:49.073795 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:15:49.073928 (+   133us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:49.073963 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.074029 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:50.071473 (+997444us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b240025b248. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363e09d 00007f3ec627010e 00007f3ec628788c 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221
0709 12:15:50.619752 (+548279us) raft_consensus.cc:1685] finished
0709 12:15:50.635649 (+ 15897us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.635841 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61686,"prepare.queue_time_us":73419,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":227956,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":23563,"spinlock_wait_cycles":11748096,"thread_start_us":261252,"threads_started":1,"wal-append.queue_time_us":588133}
W20260709 12:15:50.694442 19446 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:15:50.693749 19419 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.302s	user 0.002s	sys 0.000s
I20260709 12:15:50.697898 19430 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:50.714568 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52451us
I20260709 12:15:50.715319 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:50.692718 19006 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.679633 19153 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:50.706241 19177 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.768s	user 0.003s	sys 0.009s
I20260709 12:15:50.731909 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:50.652357 19287 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.707933 19348 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:50.690696 19427 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.750223 19401 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:50.704656 19440 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:50.732862 18861 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:50.747189 19277 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.029s	user 0.010s	sys 0.003s
I20260709 12:15:50.753582 19091 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:15:50.754587 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:50.726939 19266 consensus_meta.cc:287] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.003s	user 0.006s	sys 0.007s
I20260709 12:15:50.705690 19396 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.705582 19090 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.798686 19396 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.799432 19389 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.800272 19090 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.800819 19396 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:50.800567 19123 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.705780 19395 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:50.711885 19434 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.706616 19274 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:50.771181 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 55962.7us
I20260709 12:15:50.790058 19405 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.724539 19265 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.778910 11329 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:50.733580 18805 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:50.780813 19441 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:50.780645 19403 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:50.781865 19437 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:15:50.778264 19268 consensus_queue.cc:1059] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 12:15:50.784209 19433 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.816715 19353 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:15:50.788938 19387 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:15:50.789750 19287 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:50.791249 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 59440.5us
I20260709 12:15:50.836958 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:50.723520 19285 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:50.751492 19439 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:15:50.795531 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:50.844990 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49550.7us
I20260709 12:15:50.797492 19293 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:15:50.794145 19153 consensus_queue.cc:1175] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 17 status { last_received { term: 17 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 17 index: 10 } }
I20260709 12:15:50.798048 19422 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:50.746699 19386 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:50.865463 19422 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.866086 19231 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:15:50.865958 19386 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.867895 19159 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:50.868494 19159 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.808985 11325 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:15:50.869861 11325 pending_rounds.cc:85] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 12
I20260709 12:15:50.870252 11325 pending_rounds.cc:107] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:15:50.870559 11325 raft_consensus.cc:2889] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:50.870893 19159 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.810747 11901 leader_election.cc:400] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26.
I20260709 12:15:50.871412 11901 leader_election.cc:403] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:15:50.871652 19403 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:50.871887 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63514.3us
I20260709 12:15:50.872290 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:50.826493 19441 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.874475 19441 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:50.875039 19447 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26.
I20260709 12:15:50.876592 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.877856 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.878458 11249 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:15:50.811425 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 267676us
I20260709 12:15:50.664673 19448 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.815752 19389 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.815918 18861 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.816479 19286 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:50.811434 19006 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:50.807637 19434 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:50.804409 19395 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:50.818251 19265 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.792862 19401 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:50.804919 19440 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:50.708668 19391 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.756888 19188 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 12:15:50.770592 19177 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.804672 19266 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:50.787159 11072 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:50.824105 18805 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.840335 18265 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:50.794342 19427 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:50.835487 19353 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.858108 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:50.858860 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.859908 11256 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:15:50.772868 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:50.879271 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.028334 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3272) took 1469 ms. Trace:
W20260709 12:15:51.209510 11935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1480) took 2309 ms (client timeout 2918 ms). Trace:
W20260709 12:15:51.216223 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.217785 11251 raft_consensus.cc:2368] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24.
I20260709 12:15:51.013267 19458 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.218793 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.003291 11254 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:51.218719 19458 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.220142 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.958225 18816 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.222102 11934 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:51.216370 11257 rpcz_store.cc:276] 0709 12:15:49.559238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.559416 (+   178us) service_pool.cc:224] Handling call
0709 12:15:49.559911 (+   495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:49.835586 (+275675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:49.835617 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:49.843794 (+  8177us) log.cc:844] Serialized 52 byte log entry
0709 12:15:50.508677 (+664883us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:50.508745 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.508838 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:51.027998 (+519160us) raft_consensus.cc:1685] finished
0709 12:15:51.028030 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.028247 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":206315,"raft.queue_time_us":468305,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":32332,"thread_start_us":871195,"threads_started":2,"wal-append.queue_time_us":684283}
I20260709 12:15:51.224121 18805 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:50.846746 19098 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:51.216370 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:51.229254 18265 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.989621 19268 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.263484 19453 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.959743 18652 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:51.264674 18652 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:51.264554 19453 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.265377 18652 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.266215 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.266913 11326 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:51.267882 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.268690 11326 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:51.269243 19287 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.253733 19389 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.269874 19287 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.225304 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.270170 19389 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.271085 11257 raft_consensus.cc:2393] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:51.272083 19287 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.272385 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.272662 19287 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:50.995010 19352 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.000s
I20260709 12:15:51.274176 19177 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:51.274590 19352 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:51.226024 19362 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:50.994422 19046 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.003s
I20260709 12:15:50.944674 19187 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.292913 19187 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:51.050962 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.294845 19453 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.295188 19187 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.295533 11251 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:51.227770 19440 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:51.228202 19401 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:50.880410 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:51.223609 19266 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.003099 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3280) took 1268 ms. Trace:
I20260709 12:15:51.228571 19353 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:51.215999 11935 rpcz_store.cc:269] 0709 12:15:48.899748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.900064 (+   316us) service_pool.cc:224] Handling call
0709 12:15:51.209306 (+2309242us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400673ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:51.209419 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2592872704}
I20260709 12:15:50.946115 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:51.349889 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1366) took 2973 ms. Trace:
I20260709 12:15:51.253875 19090 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:51.300881 11253 rpcz_store.cc:276] 0709 12:15:49.734284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.734412 (+   128us) service_pool.cc:224] Handling call
0709 12:15:51.002803 (+1268391us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b5400109854. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:51.003011 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115392256}
I20260709 12:15:51.355584 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.356345 11253 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:51.357064 11251 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:51.361200 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2807) took 1426 ms. Trace:
I20260709 12:15:51.361955 11254 rpcz_store.cc:276] 0709 12:15:49.934888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.059806 (+124918us) service_pool.cc:224] Handling call
0709 12:15:51.003021 (+943215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:51.220146 (+217125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.360911 (+140765us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b5400109850. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:51.361115 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2168416000}
I20260709 12:15:50.877864 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:51.364599 11927 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:51.218900 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:50.879065 19392 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:50.938481 11222 connection.cc:256] RPC call timeout handler was delayed by -1.7727548589998605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 12:15:51.368419 11222 connection.cc:256] RPC call timeout handler was delayed by -1.1265548589997252s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
I20260709 12:15:51.268675 18652 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:51.272768 19389 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.272965 19304 raft_consensus.cc:3055] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:15:51.260946 19265 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.275581 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.011178 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:50.948762 19355 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.262024 18265 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.284757 11250 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:51.281368 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 65107.3us
I20260709 12:15:51.288498 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1277) took 2204 ms. Trace:
W20260709 12:15:51.290280 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 517479us
W20260709 12:15:50.995950 19456 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.149s	user 0.003s	sys 0.000s
I20260709 12:15:51.293124 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:51.227113 11325 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:15:51.226778 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.422724 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:50.855777 19405 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:51.297510 11322 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:51.382457 19304 raft_consensus.cc:740] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:15:51.300051 11331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.007s	sys 0.004s
I20260709 12:15:51.317310 19440 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:51.016891 19262 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:50.949790 19231 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:51.177795 19379 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:51.366114 11256 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:50.866012 19422 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:51.368235 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:51.370203 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.263504 19098 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.224705 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:51.450911 11935 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:51.451318 11248 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:51.302878 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2834) took 1467 ms. Trace:
I20260709 12:15:51.224454 19427 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:51.352662 11178 rpcz_store.cc:276] 0709 12:15:48.376167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.377699 (+  1532us) service_pool.cc:224] Handling call
0709 12:15:48.562893 (+185194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:48.588967 (+ 26074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:48.589021 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:48.593959 (+  4938us) log.cc:844] Serialized 26 byte log entry
0709 12:15:49.186084 (+592125us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:49.186147 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.186241 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:51.295028 (+2108787us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b240012ad48. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363dfb0 00007f3ec62b4488 00007f3ec62b4be8 00007f3ec628c398 00007f3ec628776a 00007f3ec6286bb6
0709 12:15:51.295703 (+   675us) raft_consensus.cc:1685] finished
0709 12:15:51.295731 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.349801 (+ 54070us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25545,"raft.queue_time_us":26410,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":80109,"spinlock_wait_cycles":72089856,"thread_start_us":444181,"threads_started":1,"wal-append.queue_time_us":818415}
I20260709 12:15:51.452720 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.452149 11179 rpcz_store.cc:276] 0709 12:15:49.835673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.020376 (+184703us) service_pool.cc:224] Handling call
0709 12:15:50.020763 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:50.323367 (+302604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.302458 (+979091us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540016f8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:51.302783 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693469440}
I20260709 12:15:51.296037 19159 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.383219 11257 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:51.376602 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.021690 19285 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.352722 19358 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:51.296217 19362 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.350916 19174 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:51.350466 19458 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:51.475973 18265 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.401892 19391 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.403016 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:51.379103 19392 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.382555 19395 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.404474 18861 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:51.406191 19355 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:51.409941 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:51.284232 19287 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.411238 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:51.504992 11930 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
W20260709 12:15:51.415462 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.027s	sys 0.072s
W20260709 12:15:51.505834 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.027s	sys 0.072s
I20260709 12:15:51.419608 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:51.507483 19458 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:15:51.507685 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 88210.1us
I20260709 12:15:51.416929 11255 rpcz_store.cc:276] 0709 12:15:49.071728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.072158 (+   430us) service_pool.cc:224] Handling call
0709 12:15:49.072615 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:49.750862 (+678247us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:49.820168 (+ 69306us) log.cc:844] Serialized 14 byte log entry
0709 12:15:50.386529 (+566361us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:50.387163 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:15:50.408268 (+ 21105us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:50.408308 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.008908 (+600600us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400277c54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:51.008993 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:51.217759 (+208766us) raft_consensus.cc:1685] finished
0709 12:15:51.275540 (+ 57781us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.275705 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173611,"prepare.queue_time_us":69306,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":6285,"raft.queue_time_us":683748,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":43987,"spinlock_wait_cycles":741200768,"thread_start_us":739053,"threads_started":2,"wal-append.queue_time_us":503058}
I20260709 12:15:51.510461 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.512079 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:51.512945 11255 raft_consensus.cc:2368] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26.
I20260709 12:15:51.422216 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3273) took 1861 ms. Trace:
I20260709 12:15:51.298523 11931 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:51.513805 11250 rpcz_store.cc:276] 0709 12:15:49.561274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.561424 (+   150us) service_pool.cc:224] Handling call
0709 12:15:49.601719 (+ 40295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:51.415931 (+1814212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.422005 (+  6074us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54001cfcd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:51.422159 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3870041344}
I20260709 12:15:51.298400 19046 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:15:51.298673 19360 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.954854 19123 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.437486 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.438851 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.439143 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:51.440377 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.442960 11929 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:15:51.446453 11327 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.020s	sys 0.010s
I20260709 12:15:51.286386 19268 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.449657 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.372982 19090 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:51.451071 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.290520 19388 consensus_queue.cc:1059] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.140s
I20260709 12:15:51.453491 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.261264 18816 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.461860 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:51.465174 11176 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:51.434559 19293 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.466420 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.431264 19359 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.471626 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.473500 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.300896 19393 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 12:15:51.478719 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.451895 19262 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:51.466960 19032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.011s	sys 0.018s
I20260709 12:15:51.449175 19453 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:51.469445 19358 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:51.473348 19174 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.507913 19458 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.434075 11249 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:51.514814 11931 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:51.558120 11329 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:51.558892 19458 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:51.559166 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.559954 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.560736 19174 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:51.561005 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:51.539485 19463 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:15:51.562203 11182 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:51.562420 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:51.562469 19268 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:51.569288 19032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.013s	sys 0.019s
I20260709 12:15:51.563201 11179 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:15:51.557925 19393 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:51.563468 19360 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.563525 11257 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:51.568995 19405 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.559469 19453 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:51.218917 11072 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:51.569620 11325 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:51.474610 19159 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.600828 18816 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:51.620640 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.539808 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:51.571291 11901 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:51.623106 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51892us
I20260709 12:15:51.623764 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1574) took 1060 ms. Trace:
I20260709 12:15:51.492578 18805 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.600559 11252 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:51.626880 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.601861 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1573) took 1038 ms. Trace:
I20260709 12:15:51.461001 19098 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.602645 11933 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 16. (term mismatch)
I20260709 12:15:51.631047 11933 pending_rounds.cc:85] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 15
I20260709 12:15:51.631296 11933 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 26.16
I20260709 12:15:51.631554 11933 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:51.453009 19231 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.557782 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.647135 11255 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:51.538645 19427 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 12:15:51.652432 19453 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.630507 19266 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:51.655805 19266 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:51.656244 19266 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:51.658994 19465 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 12:15:51.660773 11181 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:51.612357 19393 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:51.664867 19393 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.666551 19393 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:51.678354 19466 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:15:51.680641 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1577) took 1086 ms. Trace:
W20260709 12:15:51.680979 19464 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:15:51.681155 11328 rpcz_store.cc:276] 0709 12:15:50.594652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.708302 (+113650us) service_pool.cc:224] Handling call
0709 12:15:50.708652 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:50.807796 (+ 99144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:50.807833 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:50.823584 (+ 15751us) log.cc:844] Serialized 26 byte log entry
0709 12:15:51.083919 (+260335us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:51.206643 (+122724us) log.cc:844] Serialized 14 byte log entry
0709 12:15:51.207052 (+   409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.207176 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:51.680261 (+473085us) raft_consensus.cc:1685] finished
0709 12:15:51.680298 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.680546 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":116801,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":107207,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":168,"thread_start_us":344545,"threads_started":2,"wal-append.queue_time_us":613410}
I20260709 12:15:51.606367 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.606552 11329 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:51.557613 11327 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.021s	sys 0.012s
I20260709 12:15:51.684002 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.684711 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3139) took 1511 ms. Trace:
I20260709 12:15:51.684826 11328 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:51.685817 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:51.570297 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 160465us
I20260709 12:15:51.686952 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.601521 19184 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:51.624740 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.718346 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.467231 19430 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:15:51.601351 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.685086 11327 rpcz_store.cc:276] 0709 12:15:50.173526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.192153 (+ 18627us) service_pool.cc:224] Handling call
0709 12:15:50.192552 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:50.192807 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:50.593735 (+400928us) log.cc:844] Serialized 14 byte log entry
0709 12:15:51.684360 (+1090625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:51.684418 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:51.684463 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.684525 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.684658 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35549,"prepare.queue_time_us":416096,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":74507,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":1398918,"threads_started":3,"wal-append.queue_time_us":396107}
I20260709 12:15:51.722172 11176 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:15:51.722327 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.633471 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:51.634168 11927 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:51.600914 19359 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.605933 19447 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:15:51.624387 18816 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.491384 19304 consensus_queue.cc:260] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.617580 19405 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:51.627516 11324 rpcz_store.cc:276] 0709 12:15:50.563472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.563602 (+   130us) service_pool.cc:224] Handling call
0709 12:15:50.563964 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:50.612737 (+ 48773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:50.612880 (+   143us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:50.656006 (+ 43126us) log.cc:844] Serialized 26 byte log entry
0709 12:15:50.749859 (+ 93853us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:15:50.755633 (+  5774us) log.cc:844] Serialized 14 byte log entry
0709 12:15:50.813533 (+ 57900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.814380 (+   847us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:50.995390 (+181010us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:51.271260 (+275870us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b180048c220. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:15:51.601643 (+330383us) raft_consensus.cc:1685] finished
0709 12:15:51.601666 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.601805 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37359,"prepare.queue_time_us":5300,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":289143,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":12268,"spinlock_wait_cycles":94427904,"thread_start_us":128511,"threads_started":3,"wal-append.queue_time_us":137736}
I20260709 12:15:51.684007 11178 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:51.719904 11328 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:51.624167 11331 rpcz_store.cc:276] 0709 12:15:50.563819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.620486 (+ 56667us) service_pool.cc:224] Handling call
0709 12:15:50.620850 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:50.662992 (+ 42142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:50.663597 (+   605us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:50.680820 (+ 17223us) log.cc:844] Serialized 26 byte log entry
0709 12:15:51.431944 (+751124us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:51.432003 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.432114 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:51.623530 (+191416us) raft_consensus.cc:1685] finished
0709 12:15:51.623555 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.623702 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":194683,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":676265,"threads_started":2,"wal-append.queue_time_us":834042}
I20260709 12:15:51.735797 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.736760 11331 raft_consensus.cc:2393] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:51.604424 19293 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.732913 11174 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:51.608310 19445 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.112s
I20260709 12:15:51.734791 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.764935 11325 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:51.735888 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.765959 11328 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:51.767055 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:51.767654 11328 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:51.725998 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.740209 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:51.742055 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.768967 11331 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:51.769025 11324 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:51.770110 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:51.770663 11324 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:51.773453 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 1296) took 1427 ms. Trace:
I20260709 12:15:51.752799 11329 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:51.775846 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.776247 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:51.735711 18816 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:51.773901 11930 rpcz_store.cc:276] 0709 12:15:50.345624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.381567 (+ 35943us) service_pool.cc:224] Handling call
0709 12:15:51.773270 (+1391703us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b540069e454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:51.773391 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1947314432}
I20260709 12:15:51.784061 11331 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:51.726702 19184 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:51.742862 19448 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:51.794651 11327 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:51.793505 11325 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:51.796736 11255 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:51.793463 11324 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
W20260709 12:15:51.799391 11326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.016s	sys 0.038s
W20260709 12:15:51.801412 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.016s	sys 0.064s
I20260709 12:15:51.768543 19304 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:15:51.802850 11326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.016s	sys 0.038s
W20260709 12:15:51.803149 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.016s	sys 0.064s
I20260709 12:15:51.803810 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:51.809754 11182 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:51.810652 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.811307 11329 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:51.812302 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:51.813681 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:51.782902 19123 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:51.789445 19098 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.821363 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:51.822064 11329 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:51.824267 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:51.643890 11072 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 14 to 20, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:15:51.783335 19362 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:51.836472 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3279) took 2110 ms. Trace:
I20260709 12:15:51.855960 11252 rpcz_store.cc:276] 0709 12:15:49.726447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.747364 (+ 20917us) service_pool.cc:224] Handling call
0709 12:15:49.750715 (+  3351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:51.627206 (+1876491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.836205 (+208999us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540007e754. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:51.836398 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4253721472}
I20260709 12:15:51.825606 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:51.839900 11257 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 26. Current term is 27. Ops: []
W20260709 12:15:51.874432 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48887.2us
I20260709 12:15:51.875423 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:51.884761 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.888347 11174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:51.890100 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:51.891366 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:51.813910 19231 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:51.783030 19262 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:51.821569 19123 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.821004 19392 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:51.827797 18652 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:15:51.911273 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.914058 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.913530 11329 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:51.925098 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:51.926230 11329 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:51.914076 11145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.012s	sys 0.038s
I20260709 12:15:51.846243 19430 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:51.926923 11145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.013s	sys 0.039s
I20260709 12:15:51.886478 19188 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:51.928169 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.928104 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.946300 11257 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:51.946432 11252 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:51.947592 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:51.886713 19153 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:15:51.930871 11147 connection.cc:256] RPC call timeout handler was delayed by -1.007702978999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 12:15:51.926578 11182 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:51.940143 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:51.942155 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:51.951195 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.948380 11257 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:51.951445 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:51.952389 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:51.952988 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.953052 11329 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:51.892967 18642 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.909740 19123 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:51.900741 19360 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:51.952522 18972 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.026s	sys 0.031s
W20260709 12:15:51.984211 11251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.027s	sys 0.048s
W20260709 12:15:52.004218 11251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.028s	sys 0.048s
I20260709 12:15:51.930637 18652 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.990046 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:51.951442 11147 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 2.918s (SENT) [suppressed 3 similar messages]
I20260709 12:15:51.952451 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:52.009027 11182 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:52.005856 11248 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.553s	user 0.005s	sys 0.008s
I20260709 12:15:52.009451 11248 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:52.010044 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3295) took 1132 ms. Trace:
I20260709 12:15:52.010377 11248 rpcz_store.cc:276] 0709 12:15:50.877296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.224451 (+347155us) service_pool.cc:224] Handling call
0709 12:15:52.009976 (+785525us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:52.010712 11175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.007s	sys 0.024s
W20260709 12:15:52.012331 11175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.007s	sys 0.024s
W20260709 12:15:52.004103 18972 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.027s	sys 0.032s
W20260709 12:15:52.017052 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.978957 19231 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:15:52.022193 11935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.024s	sys 0.021s
W20260709 12:15:52.027415 11935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.024s	sys 0.021s
I20260709 12:15:52.030074 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3156) took 1371 ms. Trace:
I20260709 12:15:52.030113 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3290) took 1154 ms. Trace:
I20260709 12:15:52.030582 11323 rpcz_store.cc:276] 0709 12:15:50.659152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.993293 (+334141us) service_pool.cc:224] Handling call
0709 12:15:50.993880 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:50.994451 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:50.994841 (+   390us) log.cc:844] Serialized 14 byte log entry
0709 12:15:51.468510 (+473669us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:51.472243 (+  3733us) log.cc:844] Serialized 26 byte log entry
0709 12:15:51.472420 (+   177us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:51.472469 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.472553 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.029816 (+557263us) raft_consensus.cc:1685] finished
0709 12:15:52.029839 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.030009 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2754,"prepare.queue_time_us":570,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":23018,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":156,"spinlock_wait_cycles":7363456,"thread_start_us":457591,"threads_started":1,"wal-append.queue_time_us":898219}
I20260709 12:15:52.032227 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:52.033138 11331 raft_consensus.cc:2410] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 26 because replica has last-logged OpId of term: 24 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 10.
I20260709 12:15:52.033059 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:52.034068 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:52.034343 11323 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:52.035761 11324 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:15:52.037324 11324 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:52.039686 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:52.034126 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:52.044668 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.030792 11249 rpcz_store.cc:276] 0709 12:15:50.875993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.879034 (+  3041us) service_pool.cc:224] Handling call
0709 12:15:52.029948 (+1150914us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400068954. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:52.030064 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501636864}
I20260709 12:15:52.045617 11331 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:52.048031 11222 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.007s	sys 0.023s
W20260709 12:15:52.048434 11222 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.008s	sys 0.024s
W20260709 12:15:52.048717 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 677519us
I20260709 12:15:51.980149 19287 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:52.054442 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 97032.1us
I20260709 12:15:52.062559 11223 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:52.074460 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:52.011116 18652 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:51.955714 19430 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:52.087906 19174 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.008s
I20260709 12:15:52.092792 11331 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:52.094866 19405 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.098513 11328 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
W20260709 12:15:52.097075 19353 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.719s	user 0.009s	sys 0.002s
I20260709 12:15:52.007383 19153 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:52.157136 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 469860us
I20260709 12:15:52.078226 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.009781 19188 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:52.175236 19309 raft_consensus.cc:3055] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:15:52.175935 19309 raft_consensus.cc:740] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:52.181278 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3292) took 1304 ms. Trace:
I20260709 12:15:52.181481 19184 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:52.182381 19153 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.181278 19262 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.137903 11329 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:52.187928 11329 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:15:52.189284 11329 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:15:52.026113 19231 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.190165 19395 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:52.190301 19231 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:52.055871 18805 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.192111 11324 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:52.106789 19410 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:15:52.105118 19006 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.846s	user 0.010s	sys 0.001s
I20260709 12:15:52.186013 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:52.199390 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:52.181788 11251 rpcz_store.cc:276] 0709 12:15:50.876527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.358904 (+482377us) service_pool.cc:224] Handling call
0709 12:15:51.359336 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:51.359623 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:51.359661 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:51.376706 (+ 17045us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.004707 (+628001us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:52.004750 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.004821 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.181045 (+176224us) raft_consensus.cc:1685] finished
0709 12:15:52.181066 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.181208 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":583967,"threads_started":1,"wal-append.queue_time_us":792103}
W20260709 12:15:52.104876 11322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.005s	sys 0.024s
I20260709 12:15:52.005528 18642 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.106571 19360 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:52.166363 11900 leader_election.cc:400] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26.
I20260709 12:15:52.186088 19098 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:52.109004 19434 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.846s	user 0.011s	sys 0.000s
W20260709 12:15:52.204785 11322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.005s	sys 0.025s
I20260709 12:15:52.076129 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:52.222102 11926 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:15:51.938171 11072 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:52.213810 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47601.5us
I20260709 12:15:52.157721 19359 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:52.249009 19473 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:15:52.226213 19467 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.284s	user 0.004s	sys 0.000s
I20260709 12:15:52.223358 19006 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:15:52.226550 19358 consensus_meta.cc:287] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.668s	user 0.005s	sys 0.009s
I20260709 12:15:52.150991 19174 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.223330 18805 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:52.177138 19309 consensus_queue.cc:260] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.261302 18805 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:52.267086 19309 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:15:52.267494 19393 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.600s	user 0.003s	sys 0.012s
I20260709 12:15:52.266597 11072 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:52.268229 19393 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:52.268702 19393 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:52.269071 19393 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:52.259006 19006 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:52.270489 19006 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:52.269909 11173 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 18.
I20260709 12:15:52.275074 11328 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:15:52.276542 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.277257 11178 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:52.278216 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.278746 11178 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:52.279532 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.280602 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:52.266683 18805 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.283105 11328 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:15:52.281534 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.160715 19353 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.285605 11328 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:52.238063 19293 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:52.301870 11254 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.011s	sys 0.022s
I20260709 12:15:52.302536 11177 raft_consensus.cc:1240] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: [22.12-22.12]
W20260709 12:15:52.303992 11254 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.011s	sys 0.022s
I20260709 12:15:52.239416 19098 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:52.306787 11901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.007s	sys 0.023s
I20260709 12:15:52.308969 11932 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
W20260709 12:15:52.309811 11901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.007s	sys 0.023s
W20260709 12:15:52.310110 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 545627us
W20260709 12:15:52.312019 18816 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.003s
I20260709 12:15:52.312594 18816 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.315075 11901 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:52.315986 19262 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:52.316638 19262 raft_consensus.cc:697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:52.317595 19262 consensus_queue.cc:237] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.329573 18816 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:52.334112 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:52.290555 19159 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:52.339556 11899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.014s	sys 0.018s
W20260709 12:15:52.340034 11899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.014s	sys 0.019s
I20260709 12:15:52.239557 19434 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.345669 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.347812 11181 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:52.349205 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:52.352414 19353 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.353207 19353 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:52.358119 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 818352us
I20260709 12:15:52.285955 19188 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.358999 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:52.360154 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:52.362047 19427 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:52.362748 19427 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:52.362893 19098 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.363457 19427 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:52.369520 19304 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:52.370123 19304 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.370494 18642 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.372211 19304 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:52.373371 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1578) took 1707 ms. Trace:
I20260709 12:15:52.373371 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3155) took 1735 ms. Trace:
I20260709 12:15:52.373592 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:52.374691 11181 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:15:52.378535 11898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.005s	sys 0.038s
I20260709 12:15:52.374027 11326 rpcz_store.cc:276] 0709 12:15:50.665966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.269269 (+603303us) service_pool.cc:224] Handling call
0709 12:15:51.269593 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:51.269818 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:51.270421 (+   603us) log.cc:844] Serialized 14 byte log entry
0709 12:15:52.365472 (+1095051us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:52.365528 (+    56us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:52.365572 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.373178 (+  7606us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.373309 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":556060,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":556458,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":538555,"threads_started":1,"wal-append.queue_time_us":639565}
I20260709 12:15:52.291025 19358 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.393702 11250 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:52.397347 11898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.005s	sys 0.039s
I20260709 12:15:52.374146 11330 rpcz_store.cc:276] 0709 12:15:50.637790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.771656 (+133866us) service_pool.cc:224] Handling call
0709 12:15:50.772157 (+   501us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:50.772627 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:50.772669 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:50.795140 (+ 22471us) log.cc:844] Serialized 52 byte log entry
0709 12:15:51.506326 (+711186us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:51.609276 (+102950us) log.cc:844] Serialized 14 byte log entry
0709 12:15:52.183113 (+573837us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:52.353355 (+170242us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b18001db060. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3ec3601a40 00007f3ec35fc4e4 00007f3ec35fc455 00007f3ec66e4060 00007f3ec66f6716 00007f3ec6707287 00007f3ec6707208
0709 12:15:52.353558 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.353659 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.373159 (+ 19500us) raft_consensus.cc:1685] finished
0709 12:15:52.373179 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.373308 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3526,"prepare.queue_time_us":106367,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":3368,"raft.queue_time_us":796002,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"spinlock_wait_cycles":35240192,"thread_start_us":681225,"threads_started":1,"wal-append.queue_time_us":994407}
W20260709 12:15:52.358184 19401 consensus_meta.cc:287] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.985s	user 0.003s	sys 0.011s
W20260709 12:15:52.379822 19440 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.922s	user 0.009s	sys 0.007s
I20260709 12:15:52.353601 19353 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.406448 18652 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:52.401021 19266 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.744s	user 0.009s	sys 0.005s
I20260709 12:15:52.379479 19177 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:15:52.384142 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69180.1us
I20260709 12:15:52.415473 11327 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:15:52.416391 11326 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:52.417670 11327 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:52.420162 11181 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:52.425215 19434 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:52.425936 19393 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.429612 18652 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.385577 19098 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:15:52.346624 18816 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:52.434001 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:52.414659 19353 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:52.415488 19309 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:52.442009 11901 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:52.406136 19401 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.444095 11178 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:52.439065 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:52.445016 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.446022 11174 raft_consensus.cc:606] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Received request to transfer leadership to TS fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:52.448213 11174 raft_consensus.cc:614] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:52.436756 19458 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.877s	user 0.001s	sys 0.014s
W20260709 12:15:52.417687 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.454237 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 502897us
I20260709 12:15:52.457023 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:52.435943 18816 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:52.458881 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.449259 19430 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:52.460960 19490 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:52.429924 19403 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:15:52.415243 19440 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.431911 19293 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 12:15:52.495083 19293 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:52.495532 19293 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:52.496601 19393 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.496508 19293 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.499271 19440 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:52.499163 19006 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:52.500609 19481 log.cc:927] Time spent T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:15:52.428448 19266 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:52.455910 18861 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.951s	user 0.004s	sys 0.016s
I20260709 12:15:52.452229 19434 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
W20260709 12:15:52.473742 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.461158 19046 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:15:52.478264 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.506625 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.504611 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.481370 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.473343 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:52.489840 19358 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:52.451910 19309 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.458132 18816 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:52.453763 19458 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.501875 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3522) took 1069 ms. Trace:
I20260709 12:15:52.454680 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.480118 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:52.508817 11327 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
W20260709 12:15:52.509749 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.522276 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.525786 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:52.573989 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.512202 11322 rpcz_store.cc:276] 0709 12:15:51.432608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.434726 (+  2118us) service_pool.cc:224] Handling call
0709 12:15:51.460752 (+ 26026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:51.462403 (+  1651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:51.462452 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:51.463107 (+   655us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.229993 (+766886us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:52.231527 (+  1534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.231643 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.501656 (+270013us) raft_consensus.cc:1685] finished
0709 12:15:52.501681 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.501816 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1175,"raft.queue_time_us":5079,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7423,"thread_start_us":766502,"threads_started":1,"wal-append.queue_time_us":935243}
I20260709 12:15:52.503625 19386 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:15:52.582980 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 122229us
I20260709 12:15:52.589848 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:52.590618 19458 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:52.590792 19486 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:52.591243 19486 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.591588 19486 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:15:52.586069 19483 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:15:52.512364 19293 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:52.599416 19358 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:52.599964 19358 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.600243 11324 raft_consensus.cc:1240] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:15:52.600376 19358 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:52.602113 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:52.602504 19353 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.603431 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:52.603696 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39342 (request call id 3529) took 1143 ms. Trace:
I20260709 12:15:52.604224 11324 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:52.586037 19386 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:52.605739 19090 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.502210 18861 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.104s
I20260709 12:15:52.503463 19266 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.606602 19386 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.607924 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3485) took 1172 ms. Trace:
I20260709 12:15:52.510407 19174 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.646237 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:52.647343 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.648401 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:52.610661 11147 leader_election.cc:400] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24.
I20260709 12:15:52.651012 11147 leader_election.cc:403] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:15:52.584807 11181 raft_consensus.cc:2376] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25. Re-sending same reply.
I20260709 12:15:52.612859 11933 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:52.604175 11325 rpcz_store.cc:276] 0709 12:15:51.460162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.627818 (+167656us) service_pool.cc:224] Handling call
0709 12:15:52.603520 (+975702us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b54000c7bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:52.603632 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96108288}
I20260709 12:15:52.654924 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:52.501609 19006 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.502085 19265 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.656086 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:52.656416 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:52.658623 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 48165.6us
I20260709 12:15:52.566300 19046 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:52.566619 19098 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.698518 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:52.657598 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:52.510792 19490 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:52.734532 11147 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:52.744881 19231 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.001s
I20260709 12:15:52.656580 19403 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:15:52.746160 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:52.748013 19403 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:52.748698 19403 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.749217 19403 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:52.608578 19358 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.752931 19358 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:52.698145 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 49831.7us
W20260709 12:15:52.702507 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 45002.5us
I20260709 12:15:52.699472 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:52.699481 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.657713 19090 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:52.756698 11327 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:52.757262 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:52.757798 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:52.758225 11181 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:52.758430 11327 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:52.696484 19430 consensus_queue.cc:1059] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.062s
I20260709 12:15:52.759341 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.760511 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.511637 19393 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:52.731386 19098 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.707600 19265 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.761948 19393 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.763397 19309 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:52.763615 19098 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:52.764151 19098 raft_consensus.cc:3055] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:52.764323 19393 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:52.764566 19098 raft_consensus.cc:740] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:52.764330 19046 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.765897 19231 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.758291 19490 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.756948 19090 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.729455 19266 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:15:52.709913 18972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.006s	sys 0.035s
I20260709 12:15:52.768316 19231 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:52.737159 19285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.017s	sys 0.027s
I20260709 12:15:52.696794 19476 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24.
I20260709 12:15:52.599997 19486 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.769068 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.790376 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:52.608330 11175 rpcz_store.cc:276] 0709 12:15:51.435820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.435967 (+   147us) service_pool.cc:224] Handling call
0709 12:15:51.436393 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:51.453527 (+ 17134us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:51.466122 (+ 12595us) log.cc:844] Serialized 14 byte log entry
0709 12:15:52.014813 (+548691us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:52.022612 (+  7799us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.023142 (+   530us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:52.023232 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.023496 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.607691 (+584195us) raft_consensus.cc:1685] finished
0709 12:15:52.607714 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.607863 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10809,"prepare.queue_time_us":15765,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":36328,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":26784,"thread_start_us":525515,"threads_started":1,"wal-append.queue_time_us":913883}
I20260709 12:15:52.800902 11327 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:52.699182 19488 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:52.805572 11181 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:52.771198 19006 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:52.808069 11322 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:52.808799 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:52.835847 11175 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:52.803004 19285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.017s	sys 0.028s
W20260709 12:15:52.794029 18972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.007s	sys 0.035s
W20260709 12:15:52.792425 19422 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.320s	user 0.007s	sys 0.009s
W20260709 12:15:52.793881 18805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.018s	sys 0.019s
I20260709 12:15:52.759363 19430 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:52.837502 11175 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 26. Current term is 27. Ops: []
W20260709 12:15:52.851363 19123 consensus_meta.cc:287] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.003s
I20260709 12:15:52.854993 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:52.856653 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:52.856850 11927 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
W20260709 12:15:52.803782 19410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.011s	sys 0.037s
W20260709 12:15:52.853818 11328 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.018s	sys 0.027s
W20260709 12:15:52.864600 11176 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.004s	sys 0.031s
W20260709 12:15:52.851590 19360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.017s	sys 0.018s
I20260709 12:15:52.879010 11175 raft_consensus.cc:1275] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
W20260709 12:15:52.883181 11176 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.004s	sys 0.031s
I20260709 12:15:52.883105 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1286) took 1502 ms. Trace:
I20260709 12:15:52.883744 11255 rpcz_store.cc:276] 0709 12:15:51.381050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.805298 (+424248us) service_pool.cc:224] Handling call
0709 12:15:51.805760 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:51.806265 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:51.806310 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:51.806896 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.304619 (+497723us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:52.304688 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.304775 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.882834 (+578059us) raft_consensus.cc:1685] finished
0709 12:15:52.882866 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.883044 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6671,"thread_start_us":422932,"threads_started":1,"wal-append.queue_time_us":882284}
I20260709 12:15:52.880678 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:52.886245 11322 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:52.883071 11328 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.018s	sys 0.027s
I20260709 12:15:52.888383 11175 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
W20260709 12:15:52.860397 18805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.019s	sys 0.019s
W20260709 12:15:52.880859 19262 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.012s	sys 0.020s
W20260709 12:15:52.882553 19410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.012s	sys 0.037s
I20260709 12:15:52.893957 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:52.918546 11327 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:52.894785 11175 raft_consensus.cc:1240] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:15:52.765517 19098 consensus_queue.cc:260] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:52.857753 19153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.024s	sys 0.022s
W20260709 12:15:52.904529 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.887192 19360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.017s	sys 0.018s
I20260709 12:15:52.898940 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3500) took 1104 ms. Trace:
I20260709 12:15:52.939978 11175 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 27. Current term is 28. Ops: []
W20260709 12:15:52.951552 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.896628 19262 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.014s	sys 0.022s
W20260709 12:15:52.899452 19388 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.007s	sys 0.030s
W20260709 12:15:52.892712 19188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.012s
W20260709 12:15:52.941602 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.012s	sys 0.007s
W20260709 12:15:52.957945 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.013s	sys 0.008s
I20260709 12:15:52.949985 11176 rpcz_store.cc:276] 0709 12:15:51.794259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.794489 (+   230us) service_pool.cc:224] Handling call
0709 12:15:51.794927 (+   438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:51.795377 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:51.795420 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:51.802043 (+  6623us) log.cc:844] Serialized 52 byte log entry
0709 12:15:52.309518 (+507475us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:52.883822 (+574304us) log.cc:844] Serialized 14 byte log entry
0709 12:15:52.888954 (+  5132us) log.cc:844] Serialized 14 byte log entry
0709 12:15:52.898432 (+  9478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.898551 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.898619 (+    68us) raft_consensus.cc:1685] finished
0709 12:15:52.898641 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:52.898859 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5283,"raft.queue_time_us":55687,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":11130,"thread_start_us":1061511,"threads_started":2,"wal-append.queue_time_us":783652}
W20260709 12:15:52.948069 19153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.025s	sys 0.023s
I20260709 12:15:52.878582 19422 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:52.940577 19410 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:52.885998 19123 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:52.882598 19430 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:52.967283 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.978735 11928 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:52.965961 19098 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:52.969221 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:52.976728 19482 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.720s	user 0.005s	sys 0.000s
W20260709 12:15:52.958801 19188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.007s	sys 0.013s
W20260709 12:15:52.985327 11326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.019s	sys 0.031s
W20260709 12:15:53.005597 11326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.019s	sys 0.032s
W20260709 12:15:52.960371 19388 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.007s	sys 0.030s
I20260709 12:15:52.992358 11327 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
W20260709 12:15:52.993135 11331 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.004s	sys 0.016s
I20260709 12:15:52.983430 11322 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:15:53.008981 11929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.014s	sys 0.039s
W20260709 12:15:52.995307 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.004s	sys 0.016s
W20260709 12:15:53.009390 11929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.014s	sys 0.039s
W20260709 12:15:53.009712 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.004s	sys 0.016s
W20260709 12:15:52.977952 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 617817us
W20260709 12:15:52.960942 19489 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:15:52.955120 19360 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:52.961156 19474 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.011s	sys 0.006s
W20260709 12:15:53.002681 11254 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.020s	sys 0.031s
W20260709 12:15:53.038189 11254 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.021s	sys 0.031s
I20260709 12:15:53.003466 11253 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:52.985162 11329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.030s
W20260709 12:15:53.039705 11329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.012s	sys 0.031s
I20260709 12:15:53.040577 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3163) took 1420 ms. Trace:
I20260709 12:15:53.041018 11329 rpcz_store.cc:276] 0709 12:15:51.620651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.197879 (+577228us) service_pool.cc:224] Handling call
0709 12:15:52.198160 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.198365 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:52.472508 (+274143us) log.cc:844] Serialized 14 byte log entry
0709 12:15:53.040188 (+567680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:53.040241 (+    53us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:53.040300 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.040370 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.040505 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":660958,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":841129,"threads_started":2}
I20260709 12:15:53.010277 11182 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 26: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:53.011240 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.043249 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:53.010174 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 637628us
I20260709 12:15:53.027371 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:53.045296 11322 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 31 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:15:53.012290 11181 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:53.011548 11072 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.010s	sys 0.043s
I20260709 12:15:53.034955 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.046762 11327 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:15:53.012063 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:53.009411 11331 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.004s	sys 0.016s
I20260709 12:15:53.049893 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3501) took 1229 ms. Trace:
I20260709 12:15:53.050417 11177 rpcz_store.cc:276] 0709 12:15:51.820640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.839641 (+ 19001us) service_pool.cc:224] Handling call
0709 12:15:51.840051 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.303379 (+463328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.049529 (+746150us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54002b6454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:53.049815 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062878080}
I20260709 12:15:53.055097 18652 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:53.055675 18652 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:53.058480 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:53.046566 11072 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.010s	sys 0.043s
W20260709 12:15:53.037954 19474 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.011s	sys 0.007s
I20260709 12:15:53.060127 19410 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.060993 19090 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.062281 19486 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:53.062599 19422 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.063112 19386 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:53.064204 19123 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:53.070287 19493 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:15:53.061156 19496 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:15:53.071547 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 59552.3us
I20260709 12:15:53.056668 18652 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:53.096881 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.077307 11327 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
W20260709 12:15:53.111878 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53516.7us
I20260709 12:15:53.119925 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:53.120326 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:53.116074 18652 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:53.069419 19403 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.126143 19403 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:52.984877 19442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.006s	sys 0.017s
W20260709 12:15:53.134189 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.148105 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.014s	sys 0.011s
I20260709 12:15:53.148509 11935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1309) took 1776 ms. Trace:
W20260709 12:15:53.148761 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.015s	sys 0.011s
W20260709 12:15:53.149441 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.07331e+06us
I20260709 12:15:53.149065 11935 rpcz_store.cc:276] 0709 12:15:51.371939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.462399 (+ 90460us) service_pool.cc:224] Handling call
0709 12:15:51.462746 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:51.463130 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:51.463172 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:51.463716 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.027852 (+564136us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:52.027897 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.032521 (+  4624us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:52.795767 (+763246us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:53.145399 (+349632us) raft_consensus.cc:1685] finished
0709 12:15:53.145419 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.148425 (+  3006us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3241,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7443,"spinlock_wait_cycles":5919616,"thread_start_us":540654,"threads_started":1,"wal-append.queue_time_us":779899}
I20260709 12:15:53.118456 19177 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:15:53.160388 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 425936us
W20260709 12:15:53.174995 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.175716 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.175876 11250 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 1294) took 1768 ms (client timeout 2212 ms). Trace:
W20260709 12:15:53.176456 11250 rpcz_store.cc:269] 0709 12:15:51.407510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.514774 (+107264us) service_pool.cc:224] Handling call
0709 12:15:53.175610 (+1660836us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b5400068e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:53.175800 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915196800}
I20260709 12:15:53.176896 11148 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:53.178010 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.129055 19422 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:53.178903 11180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3504) took 1339 ms. Trace:
I20260709 12:15:53.196555 11322 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:53.179633 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:53.138336 19442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.007s	sys 0.018s
I20260709 12:15:53.181280 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3509) took 1169 ms. Trace:
I20260709 12:15:53.202070 11182 rpcz_store.cc:276] 0709 12:15:52.011697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.021520 (+  9823us) service_pool.cc:224] Handling call
0709 12:15:53.181026 (+1159506us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54002323d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:53.181209 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1949936000}
I20260709 12:15:53.181363 18642 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:53.184015 19401 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.183868 19159 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:53.191929 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:53.180470 19393 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:15:53.199617 11173 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3488) took 1760 ms (client timeout 1925 ms). Trace:
I20260709 12:15:53.195979 11180 rpcz_store.cc:276] 0709 12:15:51.839386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.841288 (+  1902us) service_pool.cc:224] Handling call
0709 12:15:51.841695 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:51.861171 (+ 19476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:51.861216 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:51.861845 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.445826 (+583981us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:52.445886 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.445999 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:53.178668 (+732669us) raft_consensus.cc:1685] finished
0709 12:15:53.178692 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.178826 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18946,"raft.queue_time_us":19104,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":15257,"thread_start_us":495850,"threads_started":1,"wal-append.queue_time_us":947464}
W20260709 12:15:53.204998 11173 rpcz_store.cc:269] 0709 12:15:51.438741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.438873 (+   132us) service_pool.cc:224] Handling call
0709 12:15:53.199442 (+1760569us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54002e45d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:53.199559 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502308608}
I20260709 12:15:53.200304 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:51810 (request call id 71) took 1287 ms. Trace:
I20260709 12:15:53.206364 11174 rpcz_store.cc:276] 0709 12:15:51.887855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.888159 (+   304us) service_pool.cc:224] Handling call
0709 12:15:53.174904 (+1286745us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54000d9b54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e6853 00007f3ecb6b460a 00007f3ec4c3a7d4 00007f3ec4c3a762 00007f3ec4c3a691 00007f3ec4c3a60c 00007f3ec4c398c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:53.175038 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278369024}
W20260709 12:15:53.160005 19492 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:15:53.220199 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:53.143949 19458 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:53.204594 11253 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 1284) took 1835 ms (client timeout 2425 ms). Trace:
W20260709 12:15:53.224040 11253 rpcz_store.cc:269] 0709 12:15:51.368807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.375661 (+  6854us) service_pool.cc:224] Handling call
0709 12:15:53.204398 (+1828737us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400087ad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:53.204516 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3544463104}
W20260709 12:15:53.210944 11930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.018s	sys 0.018s
I20260709 12:15:53.204573 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1487) took 2345 ms. Trace:
I20260709 12:15:53.217187 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:53.216267 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:53.224712 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3507) took 1251 ms. Trace:
W20260709 12:15:53.226763 11930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.018s	sys 0.018s
I20260709 12:15:53.226397 11181 rpcz_store.cc:276] 0709 12:15:51.973337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.839952 (+866615us) service_pool.cc:224] Handling call
0709 12:15:52.852104 (+ 12152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.047461 (+195357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.224470 (+177009us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54000d9654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:53.224656 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367869696}
W20260709 12:15:53.227566 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.230441 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.233266 11173 raft_consensus.cc:3055] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:15:53.233852 11926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 1294) took 2889 ms (client timeout 1528 ms). Trace:
I20260709 12:15:53.234197 11322 raft_consensus.cc:1240] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:53.233866 11173 raft_consensus.cc:740] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:53.234326 11926 rpcz_store.cc:269] 0709 12:15:50.344777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.345937 (+  1160us) service_pool.cc:224] Handling call
0709 12:15:53.233659 (+2887722us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540053a7d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:53.233782 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3802519808}
W20260709 12:15:53.236222 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.225263 11932 rpcz_store.cc:276] 0709 12:15:50.859230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.859399 (+   169us) service_pool.cc:224] Handling call
0709 12:15:50.871695 (+ 12296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.309907 (+1438212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.204378 (+894471us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400667bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:53.204519 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3302012672}
I20260709 12:15:53.207507 19090 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:53.240589 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:53.241432 11148 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:53.247372 19293 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:53.247879 19293 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:53.195950 19422 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.197897 19231 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:53.195942 19508 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:53.221208 19123 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:53.243647 11173 consensus_queue.cc:260] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.159657 19498 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 12:15:53.263018 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:53.257540 11173 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:53.266397 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:53.269557 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:53.200697 19501 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:15:53.250264 19285 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:53.249230 19153 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:15:53.273504 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.253453 19422 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:53.261369 19090 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:15:53.268590 19495 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:15:53.280851 11147 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:53.256801 19123 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.255236 19442 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.266124 19262 consensus_queue.cc:1059] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.040s
W20260709 12:15:53.286190 19502 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.173s	user 0.001s	sys 0.001s
I20260709 12:15:53.306182 19090 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.308703 19153 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:53.309419 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3523) took 1876 ms. Trace:
I20260709 12:15:53.309736 19153 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.309854 11323 rpcz_store.cc:276] 0709 12:15:51.432944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.035252 (+602308us) service_pool.cc:224] Handling call
0709 12:15:52.035597 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.035824 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.035849 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:52.036402 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.446239 (+409837us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:52.446290 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.446376 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:53.309197 (+862821us) raft_consensus.cc:1685] finished
0709 12:15:53.309218 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.309361 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26752,"thread_start_us":409461,"threads_started":1,"wal-append.queue_time_us":845882}
I20260709 12:15:53.265897 19508 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:53.311617 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.312218 11323 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:15:53.275707 19503 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 12:15:53.287937 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:53.313310 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
W20260709 12:15:53.314131 19497 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 12:15:53.314340 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.314889 11322 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:15:53.260948 19353 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:53.315979 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.316963 11327 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:53.319387 19188 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:53.320595 19188 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:53.276127 19231 consensus_queue.cc:1059] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:15:53.249015 19360 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:53.323065 19509 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:15:53.240897 19458 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.324440 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.301719 19123 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:53.301167 19430 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.327060 11327 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:53.256186 19358 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:15:53.279804 19403 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:53.279824 19285 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.265065 19266 consensus_queue.cc:1048] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.064s
W20260709 12:15:53.346175 11931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.011s	sys 0.019s
I20260709 12:15:53.289357 19434 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:53.289491 18805 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:15:53.402992 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.281212 19046 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.109s
W20260709 12:15:53.376128 11931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.012s	sys 0.019s
W20260709 12:15:53.394227 11324 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.010s	sys 0.040s
W20260709 12:15:53.472162 11324 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.014s	sys 0.040s
I20260709 12:15:53.340765 19393 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:53.389729 11328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.005s	sys 0.029s
W20260709 12:15:53.506445 11328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.006s	sys 0.030s
I20260709 12:15:53.434095 11323 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:15:53.348097 18642 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:15:53.375284 19285 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:53.393977 19032 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.003s	sys 0.034s
I20260709 12:15:53.524582 19486 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.535952 19486 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:53.533500 19032 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.003s	sys 0.034s
I20260709 12:15:53.523880 18642 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:53.538379 19486 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:53.539103 18642 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.540843 18642 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:53.283809 19487 consensus_queue.cc:1059] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.247s
I20260709 12:15:53.524523 19410 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:53.550493 19410 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:53.551574 19410 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.553129 19410 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:53.555635 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3174) took 1730 ms. Trace:
W20260709 12:15:53.558271 19520 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:15:53.559697 11930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1495) took 1088 ms. Trace:
I20260709 12:15:53.560094 11930 rpcz_store.cc:276] 0709 12:15:52.471822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.576884 (+105062us) service_pool.cc:224] Handling call
0709 12:15:52.598061 (+ 21177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.598452 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.598483 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:52.599009 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:15:53.227505 (+628496us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:53.227546 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.227619 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:53.559412 (+331793us) raft_consensus.cc:1685] finished
0709 12:15:53.559437 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.559628 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5752,"raft.queue_time_us":8981,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":628193,"threads_started":1,"wal-append.queue_time_us":685822}
I20260709 12:15:53.312317 19508 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.576417 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1492) took 1120 ms. Trace:
I20260709 12:15:53.305190 19442 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.579365 19353 raft_consensus.cc:3055] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:15:53.579916 19442 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:53.582501 19265 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:53.556196 11326 rpcz_store.cc:276] 0709 12:15:51.825726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.417254 (+591528us) service_pool.cc:224] Handling call
0709 12:15:52.433875 (+ 16621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.434408 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.434451 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:52.435470 (+  1019us) log.cc:844] Serialized 26 byte log entry
0709 12:15:53.006257 (+570787us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:53.006306 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.006382 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:53.555345 (+548963us) raft_consensus.cc:1685] finished
0709 12:15:53.555375 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.555560 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":3885,"thread_start_us":554781,"threads_started":1,"wal-append.queue_time_us":937908}
I20260709 12:15:53.530539 19405 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.349457 19403 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:53.391676 18805 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:53.479802 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.520813 11901 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:53.581514 19515 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:15:53.521811 18652 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.288734 19293 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.437163 19231 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.441948 19434 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:53.526247 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3539) took 1787 ms. Trace:
W20260709 12:15:53.261004 19476 consensus_queue.cc:1175] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 25 status { last_received { term: 24 index: 20 } last_committed_idx: 20 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:15:53.543514 19388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.013s	sys 0.031s
I20260709 12:15:53.440972 11327 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:15:53.343555 19498 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.099s
W20260709 12:15:53.595019 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.594969 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 74253.5us
I20260709 12:15:53.597430 18652 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.594810 19403 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:53.596628 18805 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:53.526351 19393 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.577029 11931 rpcz_store.cc:276] 0709 12:15:52.455996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.456163 (+   167us) service_pool.cc:224] Handling call
0709 12:15:52.461255 (+  5092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.566949 (+105694us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:52.567253 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 12:15:53.472262 (+905009us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:53.472341 (+    79us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:53.472408 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.576080 (+103672us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54005d0054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:53.576153 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.576341 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33765,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":473,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3277,"spinlock_wait_cycles":242249984,"thread_start_us":904589,"threads_started":1}
I20260709 12:15:53.579967 19353 raft_consensus.cc:740] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:53.607079 11328 rpcz_store.cc:276] 0709 12:15:51.738857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.293465 (+554608us) service_pool.cc:224] Handling call
0709 12:15:52.293781 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.296638 (+  2857us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:52.887890 (+591252us) log.cc:844] Serialized 14 byte log entry
0709 12:15:53.525747 (+637857us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:53.525850 (+   103us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:53.525916 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.526025 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.526180 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":601084,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1462,"raft.queue_time_us":5321,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3300,"thread_start_us":1209536,"threads_started":2}
I20260709 12:15:53.426925 11322 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
W20260709 12:15:53.618305 19386 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.002s
I20260709 12:15:53.620931 18652 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.622392 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:53.638193 19434 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.639158 11328 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 18.
I20260709 12:15:53.640264 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:53.613502 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:53.607070 19266 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:15:53.645192 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:53.645687 19266 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:53.634368 19006 consensus_meta.cc:287] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.796s	user 0.011s	sys 0.002s
I20260709 12:15:53.643651 19177 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.646961 19266 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.647730 19177 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.607554 19458 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.649971 19177 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:53.641516 11328 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:15:53.609721 19231 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.650879 11328 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:53.651929 11328 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.652655 19231 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:53.652989 11328 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:53.644218 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:53.653205 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.654140 18972 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:53.654554 18972 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.654937 18972 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:53.619827 19490 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.658550 19490 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:53.659276 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.661129 19490 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:53.615237 19388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.013s	sys 0.031s
I20260709 12:15:53.664529 18652 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:53.664650 18972 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.706552 18652 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:53.707790 18652 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 24.11, Last appended by leader: 11, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.709045 18972 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:53.668798 19006 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:53.675207 19393 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:53.675786 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.680279 19518 log.cc:927] Time spent T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:15:53.680699 18816 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.098s	user 0.007s	sys 0.007s
W20260709 12:15:53.681718 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.689175 19353 consensus_queue.cc:260] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:53.691952 19511 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.250s	user 0.002s	sys 0.001s
I20260709 12:15:53.713351 18816 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.716295 18816 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:53.717861 19353 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:15:53.718310 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.720942 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.699589 19422 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.700851 11322 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:15:53.701243 19516 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:15:53.705602 19525 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:15:53.590549 19474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.011s	sys 0.022s
W20260709 12:15:53.756999 19098 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.752s	user 0.010s	sys 0.003s
W20260709 12:15:53.754451 19445 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.498s	user 0.020s	sys 0.029s
W20260709 12:15:53.760110 19474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.011s	sys 0.023s
W20260709 12:15:53.740557 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.741206 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.742885 19174 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:53.746132 19486 raft_consensus.cc:3055] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:53.747184 19488 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:15:53.748970 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.753957 11329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.019s	sys 0.024s
W20260709 12:15:53.757325 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.764890 19159 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.000s	sys 0.016s
W20260709 12:15:53.767437 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.015s	sys 0.001s
I20260709 12:15:53.736420 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.761294 19285 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:53.766060 19445 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.511s	user 0.021s	sys 0.029s
I20260709 12:15:53.817750 19123 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.810652 19403 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.820179 19123 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:53.820346 19285 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:53.782198 11331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.018s	sys 0.026s
W20260709 12:15:53.830476 11331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.019s	sys 0.027s
I20260709 12:15:53.830549 19287 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:15:53.763856 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 491198us
W20260709 12:15:53.805714 11329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.021s	sys 0.024s
I20260709 12:15:53.834329 11327 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:15:53.834703 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3181) took 1214 ms. Trace:
I20260709 12:15:53.835964 11328 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:53.769801 19453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.998s	user 0.032s	sys 0.021s
I20260709 12:15:53.776314 19174 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:53.805267 19488 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:53.778209 19486 raft_consensus.cc:740] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:53.821962 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.016s	sys 0.001s
I20260709 12:15:53.822506 18805 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.763650 19525 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:53.827348 19177 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.828759 19422 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:53.841907 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.013s	sys 0.005s
W20260709 12:15:53.897598 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.013s	sys 0.006s
W20260709 12:15:53.898487 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 617668us
I20260709 12:15:53.900717 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:53.833542 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:53.902194 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68720.7us
W20260709 12:15:53.776661 11221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.008s
W20260709 12:15:53.905130 11221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.011s	sys 0.008s
W20260709 12:15:53.906311 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.889215 19476 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:15:53.908599 11173 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:53.909478 11181 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:15:53.820629 19403 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:53.821048 19159 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.000s	sys 0.017s
W20260709 12:15:53.842904 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.016s	sys 0.034s
I20260709 12:15:53.850808 11328 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 19.
W20260709 12:15:53.926164 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.017s	sys 0.034s
W20260709 12:15:53.927016 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 807091us
I20260709 12:15:53.927176 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.928133 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:53.928968 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:53.887070 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:53.928154 11328 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:53.842792 19453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.071s	user 0.033s	sys 0.022s
I20260709 12:15:53.935827 11179 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:53.938992 19490 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:15:53.940168 11254 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.023s	sys 0.038s
W20260709 12:15:53.940605 11254 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.024s	sys 0.038s
I20260709 12:15:53.941527 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:53.942277 11181 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
W20260709 12:15:53.946671 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:53.889447 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 703345us
W20260709 12:15:53.911990 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.020s	sys 0.044s
W20260709 12:15:53.955549 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.020s	sys 0.045s
I20260709 12:15:53.959954 11296 leader_election.cc:400] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26.
I20260709 12:15:53.960398 11296 leader_election.cc:403] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:15:53.961194 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:53.973214 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72594.9us
W20260709 12:15:53.974714 11222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.010s	sys 0.041s
W20260709 12:15:53.975184 11222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.010s	sys 0.041s
I20260709 12:15:53.974407 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:53.975559 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 768820us
I20260709 12:15:53.915840 19388 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:53.986013 19525 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.994737 19353 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.995417 19353 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.941908 19458 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:53.998076 19458 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.999325 19353 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:54.001127 19458 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:53.915515 19177 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:53.976536 19174 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:53.939604 19490 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:54.008312 11328 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:53.939494 19403 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.978514 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:54.016216 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:54.017179 11327 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:53.983676 19476 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:15:54.028093 11931 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:54.032249 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:54.033479 11926 raft_consensus.cc:2376] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20. Re-sending same reply.
W20260709 12:15:54.037191 11324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.015s	sys 0.074s
W20260709 12:15:54.038153 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.835089 11329 rpcz_store.cc:276] 0709 12:15:52.620305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.187598 (+567293us) service_pool.cc:224] Handling call
0709 12:15:53.188055 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.188329 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:53.188651 (+   322us) log.cc:844] Serialized 14 byte log entry
0709 12:15:53.834359 (+645708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:53.834413 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:53.834457 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.834520 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.834648 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2954,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":83,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6476,"thread_start_us":645248,"threads_started":1}
W20260709 12:15:54.037519 11324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.015s	sys 0.074s
W20260709 12:15:54.047034 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.067507 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1587) took 1591 ms. Trace:
I20260709 12:15:54.067989 11324 rpcz_store.cc:276] 0709 12:15:52.476078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.605076 (+128998us) service_pool.cc:224] Handling call
0709 12:15:52.605397 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.605607 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:15:53.472689 (+867082us) log.cc:844] Serialized 14 byte log entry
0709 12:15:54.067093 (+594404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:54.067150 (+    57us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:15:54.067209 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.067294 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.067437 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25910,"prepare.queue_time_us":405206,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":1344082,"threads_started":2}
I20260709 12:15:54.074647 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:54.076896 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:54.078449 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:54.025341 19422 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:15:54.037047 19153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.034s	sys 0.055s
I20260709 12:15:54.021401 19458 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:54.079262 11932 raft_consensus.cc:2376] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27. Re-sending same reply.
I20260709 12:15:54.024183 19403 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:54.023901 19488 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.032202 19174 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:54.085775 19517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.004s	sys 0.015s
I20260709 12:15:54.123442 19564 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:54.146622 19501 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:15:54.173646 19517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.004s	sys 0.015s
I20260709 12:15:54.148070 19422 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:54.187654 19508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.006s	sys 0.032s
W20260709 12:15:54.193135 19508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.006s	sys 0.032s
I20260709 12:15:54.185720 19501 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:54.191658 19488 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:54.150130 19490 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:54.213835 19401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.034s	sys 0.038s
W20260709 12:15:54.218966 19401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.034s	sys 0.039s
I20260709 12:15:54.222065 19486 consensus_queue.cc:260] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:54.230010 19538 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
W20260709 12:15:54.229503 19553 log.cc:927] Time spent T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:15:54.236626 19486 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:54.219821 19422 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:54.222500 19533 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:15:54.219986 19501 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:54.247140 19501 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:54.295042 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.298035 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:54.300230 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.307207 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:54.308449 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 334059us
I20260709 12:15:54.310119 11174 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:15:54.310281 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.311493 11176 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:54.311659 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:54.313218 11175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.005s	sys 0.009s
W20260709 12:15:54.317170 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.317109 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1283) took 2966 ms. Trace:
W20260709 12:15:54.318111 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.317845 11254 rpcz_store.cc:276] 0709 12:15:51.350148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.377024 (+ 26876us) service_pool.cc:224] Handling call
0709 12:15:51.377652 (+   628us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:51.723721 (+346069us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:52.306403 (+582682us) log.cc:844] Serialized 14 byte log entry
0709 12:15:53.038650 (+732247us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:53.129846 (+ 91196us) log.cc:844] Serialized 52 byte log entry
0709 12:15:53.130054 (+   208us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:53.941021 (+810967us) log.cc:844] Serialized 14 byte log entry
0709 12:15:53.945109 (+  4088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.033329 (+ 88220us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b5400086bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:54.033395 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.316257 (+282862us) raft_consensus.cc:1685] finished
0709 12:15:54.316335 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.316776 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1614908,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":8540,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":37449,"spinlock_wait_cycles":794662784,"thread_start_us":2016188,"threads_started":3,"wal-append.queue_time_us":1044011}
W20260709 12:15:54.321872 11175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.005s	sys 0.010s
I20260709 12:15:54.315744 11179 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3496) took 2701 ms. Trace:
W20260709 12:15:54.323419 11323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.007s	sys 0.011s
W20260709 12:15:54.082289 11325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.004s	sys 0.015s
W20260709 12:15:54.323973 11323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.007s	sys 0.012s
W20260709 12:15:54.324389 11325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.004s	sys 0.015s
W20260709 12:15:54.325908 11326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.016s	sys 0.006s
W20260709 12:15:54.326439 11326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.016s	sys 0.006s
I20260709 12:15:54.326308 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:54.327697 11329 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 27: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:54.206308 19293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.022s	sys 0.049s
I20260709 12:15:54.327950 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3189) took 1596 ms. Trace:
W20260709 12:15:54.328596 19293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.023s	sys 0.049s
I20260709 12:15:54.327574 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3176) took 1708 ms. Trace:
W20260709 12:15:54.081980 19032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.008s	sys 0.009s
I20260709 12:15:54.307771 11173 raft_consensus.cc:1240] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 26. Current term is 27. Ops: []
I20260709 12:15:54.312039 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:54.329574 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:54.335355 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.328526 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1579) took 2763 ms. Trace:
I20260709 12:15:54.347187 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3185) took 1687 ms. Trace:
I20260709 12:15:54.322510 11179 rpcz_store.cc:276] 0709 12:15:51.614323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.615296 (+   973us) service_pool.cc:224] Handling call
0709 12:15:51.615592 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.936342 (+2320750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.315121 (+378779us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54000d9154. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:54.315530 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5335401344}
I20260709 12:15:54.378692 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.379333 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:54.380349 11176 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:15:54.380757 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:54.383212 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:54.384279 11173 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:54.328519 11326 rpcz_store.cc:276] 0709 12:15:52.731405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.602385 (+870980us) service_pool.cc:224] Handling call
0709 12:15:53.612525 (+ 10140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:53.613005 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:53.613048 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:53.613619 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 12:15:54.327313 (+713694us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:54.327442 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.327557 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.327627 (+    70us) raft_consensus.cc:1685] finished
0709 12:15:54.327639 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.327867 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1166,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5841,"thread_start_us":712853,"threads_started":1,"wal-append.queue_time_us":611048}
I20260709 12:15:54.372144 11323 rpcz_store.cc:276] 0709 12:15:52.659635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.523290 (+863655us) service_pool.cc:224] Handling call
0709 12:15:53.535810 (+ 12520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.536017 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:53.536303 (+   286us) log.cc:844] Serialized 14 byte log entry
0709 12:15:54.325071 (+788768us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:54.325203 (+   132us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:54.325310 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.325440 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.347092 (+ 21652us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9233,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":22617,"thread_start_us":787968,"threads_started":1,"wal-append.queue_time_us":690049}
I20260709 12:15:54.387089 11174 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:15:54.389099 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.389441 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:54.390614 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.390661 11255 raft_consensus.cc:2393] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:54.339907 19266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.005s	sys 0.034s
I20260709 12:15:54.391381 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:54.067529 11327 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:54.330168 19293 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:15:54.393205 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.369282 19032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.009s	sys 0.009s
W20260709 12:15:54.404405 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.333712 19445 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:54.172711 19153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.038s	sys 0.059s
W20260709 12:15:54.412544 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.392336 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.424492 11256 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:54.381515 19403 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:54.410315 19266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.005s	sys 0.037s
I20260709 12:15:54.339740 11330 rpcz_store.cc:276] 0709 12:15:52.618793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.069099 (+450306us) service_pool.cc:224] Handling call
0709 12:15:53.069452 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:53.069855 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:53.069889 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:53.070393 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:15:53.956216 (+885823us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:53.956279 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.956388 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.326787 (+370399us) raft_consensus.cc:1685] finished
0709 12:15:54.326867 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.327262 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1213,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":15603,"thread_start_us":885379,"threads_started":1,"wal-append.queue_time_us":1111124}
I20260709 12:15:54.452476 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:54.455765 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:54.458181 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:54.458725 19177 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:54.459689 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.462131 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.474382 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:54.476673 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:54.478199 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 499315us
I20260709 12:15:54.478484 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:54.480681 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:54.481859 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.454735 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.497416 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:54.498286 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1392) took 2157 ms. Trace:
I20260709 12:15:54.498749 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.369567 11331 rpcz_store.cc:276] 0709 12:15:51.565073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.138849 (+573776us) service_pool.cc:224] Handling call
0709 12:15:52.139213 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.366435 (+227222us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:53.048683 (+682248us) log.cc:844] Serialized 14 byte log entry
0709 12:15:53.831447 (+782764us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:53.832093 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 12:15:53.832262 (+   169us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:53.832309 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.832414 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.327980 (+495566us) raft_consensus.cc:1685] finished
0709 12:15:54.328018 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.328299 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":226748,"prepare.queue_time_us":597780,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":227880,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":195,"thread_start_us":1385768,"threads_started":2,"wal-append.queue_time_us":949188}
W20260709 12:15:54.503429 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 542128us
W20260709 12:15:54.505699 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.508811 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.509959 18642 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.511068 18642 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.514699 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1591) took 1894 ms. Trace:
I20260709 12:15:54.516043 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.517112 19498 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:54.482163 19231 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.517617 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:54.523833 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.498519 11253 raft_consensus.cc:3060] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:54.515308 11325 rpcz_store.cc:276] 0709 12:15:52.620674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.190706 (+570032us) service_pool.cc:224] Handling call
0709 12:15:53.191191 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:53.325424 (+134233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:53.325473 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:53.326024 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:15:54.325442 (+999418us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:54.325556 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.514069 (+188513us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54000e5e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:54.514149 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.514243 (+    94us) raft_consensus.cc:1685] finished
0709 12:15:54.514267 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.514637 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1155,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6055,"spinlock_wait_cycles":307597184,"thread_start_us":998613,"threads_started":1,"wal-append.queue_time_us":797140}
I20260709 12:15:54.526073 11179 raft_consensus.cc:1240] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:54.518141 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.505126 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:54.528168 18805 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:54.528370 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:54.530138 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.530386 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:54.530862 19486 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.531186 11324 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 17.
I20260709 12:15:54.531833 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.531669 19486 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.533774 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.534890 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:54.535559 11322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.005s	sys 0.013s
I20260709 12:15:54.535653 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:54.536036 11322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.005s	sys 0.013s
I20260709 12:15:54.537717 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:54.520008 19544 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.538161 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.538980 19537 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26.
W20260709 12:15:54.539647 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 227885us
I20260709 12:15:54.531205 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:54.539069 19544 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:54.530628 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.529567 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.525430 11182 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 12:15:54.498917 11175 rpcz_store.cc:276] 0709 12:15:52.340525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.954234 (+613709us) service_pool.cc:224] Handling call
0709 12:15:52.954564 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.009163 (+ 54599us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:53.527045 (+517882us) log.cc:844] Serialized 14 byte log entry
0709 12:15:54.322860 (+795815us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:54.322964 (+   104us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:54.323065 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.449804 (+126739us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b54002e4fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:54.449874 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.498173 (+ 48299us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":278247,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":15198,"raft.run_cpu_time_us":462,"raft.run_wall_time_us":462,"spinlock_wait_cycles":125047040,"thread_start_us":1291865,"threads_started":2,"wal-append.queue_time_us":677814}
I20260709 12:15:54.542927 11182 pending_rounds.cc:85] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 12
I20260709 12:15:54.546964 11256 raft_consensus.cc:2393] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:54.547225 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:54.549885 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.519093 11222 connection.cc:256] RPC call timeout handler was delayed by -1.7617168960000527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 12:15:54.551792 11222 connection.cc:256] RPC call timeout handler was delayed by -1.754516896000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 12:15:54.552137 11222 connection.cc:256] RPC call timeout handler was delayed by -1.6573168959998839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
I20260709 12:15:54.558773 11176 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
W20260709 12:15:54.562106 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.546093 11182 pending_rounds.cc:107] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 12:15:54.562891 11257 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
W20260709 12:15:54.566082 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5646168959999613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
I20260709 12:15:54.566085 11182 raft_consensus.cc:2889] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:54.518440 19512 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.571648 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1597) took 1317 ms. Trace:
I20260709 12:15:54.572868 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.520814 19231 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.573887 19090 raft_consensus.cc:2764] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:15:54.541762 19293 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:54.574038 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:54.575085 19567 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:15:54.576069 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.518069 19498 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:54.580219 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.580618 11328 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:15:54.584306 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5277168959999017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 12:15:54.557350 19544 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:54.590359 11173 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
W20260709 12:15:54.592835 11222 connection.cc:256] RPC call timeout handler was delayed by -1.5043168960000912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1806s for a response
W20260709 12:15:54.595717 11222 connection.cc:256] RPC call timeout handler was delayed by -1.3972168959999181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
I20260709 12:15:54.596804 11250 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:54.596097 11222 connection.cc:256] RPC call timeout handler was delayed by -1.2649168959999315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 12:15:54.604748 11222 connection.cc:256] RPC call timeout handler was delayed by -1.1713168960000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
I20260709 12:15:54.605170 11256 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:15:54.609330 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.600514 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:54.613075 18642 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:54.617851 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:54.618331 19486 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:54.619779 11325 pending_rounds.cc:85] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 12
I20260709 12:15:54.620213 11325 pending_rounds.cc:107] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 26.13
I20260709 12:15:54.620491 11325 raft_consensus.cc:2889] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:54.620494 18642 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:54.621031 18642 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:54.621408 18642 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:54.621994 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:54.623129 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:54.624020 11331 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:15:54.626436 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:54.626905 11254 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:54.627452 11254 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:54.583952 11322 rpcz_store.cc:276] 0709 12:15:53.254399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.763274 (+508875us) service_pool.cc:224] Handling call
0709 12:15:53.779645 (+ 16371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:53.835900 (+ 56255us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:53.887817 (+ 51917us) log.cc:844] Serialized 14 byte log entry
0709 12:15:54.536781 (+648964us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:54.537663 (+   882us) log.cc:844] Serialized 26 byte log entry
0709 12:15:54.551356 (+ 13693us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:54.551439 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.563404 (+ 11965us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.571364 (+  7960us) raft_consensus.cc:1685] finished
0709 12:15:54.571387 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.571574 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4417,"prepare.queue_time_us":52997,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":58976,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":36137,"thread_start_us":596428,"threads_started":1,"wal-append.queue_time_us":644612}
I20260709 12:15:54.635777 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.636906 11251 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:54.637701 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:54.639400 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:54.643105 11324 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:15:54.645025 11324 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:15:54.646965 19445 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:54.647478 19445 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:54.647881 19445 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:54.628793 11254 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.649212 11254 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:54.625353 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:54.650822 19564 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:54.651607 19564 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:54.666572 11330 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 27. Current term is 28. Ops: []
I20260709 12:15:54.678412 11253 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:54.681759 18816 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:54.685900 11254 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:15:54.686581 11254 pending_rounds.cc:85] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 14
I20260709 12:15:54.686941 11254 pending_rounds.cc:107] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:15:54.687243 11254 raft_consensus.cc:2889] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:54.707454 11180 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:54.700450 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1601) took 1071 ms. Trace:
I20260709 12:15:54.715955 11330 rpcz_store.cc:276] 0709 12:15:53.619785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.553367 (+933582us) service_pool.cc:224] Handling call
0709 12:15:54.653736 (+100369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:54.690935 (+ 37199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.691200 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:54.717971 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.719381 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.681842 19543 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.728093 11327 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:54.654788 19512 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:54.736790 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53437.1us
I20260709 12:15:54.714047 19403 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:54.744395 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.759908 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 255042us
W20260709 12:15:54.760170 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46307.6us
I20260709 12:15:54.766279 11324 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.14-25.14]
W20260709 12:15:54.770749 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 218545us
I20260709 12:15:54.772755 11255 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:54.740662 19403 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:54.781805 19575 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.004s
I20260709 12:15:54.782749 11178 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:15:54.784083 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:54.728765 19293 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:54.781967 11932 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:15:54.794355 19512 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:54.796330 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:54.798978 18816 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:54.800750 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:54.802989 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 185322us
I20260709 12:15:54.803592 11930 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:54.804032 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:54.806106 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:54.807265 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:54.807533 19153 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:54.808784 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:54.808398 19537 raft_consensus.cc:2764] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:15:54.809644 11931 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
I20260709 12:15:54.807265 19546 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:54.810379 19546 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:54.810492 19285 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.811264 19285 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:54.809029 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:54.822192 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51813.8us
I20260709 12:15:54.781847 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:54.831351 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.852885 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44035.1us
I20260709 12:15:54.824040 19512 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:54.857950 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61773.7us
I20260709 12:15:54.855980 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:54.759971 19543 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.870757 11926 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:15:54.862619 19486 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.851477 19266 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.848517 19564 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:54.871356 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 65410.2us
W20260709 12:15:54.875391 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.878535 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.851012 19287 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:54.876716 11222 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:54.881582 11930 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:15:54.811609 19546 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.886986 11327 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 12:15:54.871884 19567 raft_consensus.cc:2764] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election decision vote started in defunct term 25: won
I20260709 12:15:54.886983 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3200) took 1243 ms. Trace:
I20260709 12:15:54.888882 11324 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:15:54.894353 11931 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:54.893059 19403 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:54.901160 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:54.901661 19551 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:15:54.902089 11933 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:54.907918 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3202) took 1161 ms. Trace:
I20260709 12:15:54.908352 11325 rpcz_store.cc:276] 0709 12:15:53.747043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.594129 (+847086us) service_pool.cc:224] Handling call
0709 12:15:54.619320 (+ 25191us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:54.621298 (+  1978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:54.621332 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:54.644492 (+ 23160us) log.cc:844] Serialized 26 byte log entry
0709 12:15:54.789795 (+145303us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:54.789854 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.789924 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.907656 (+117732us) raft_consensus.cc:1685] finished
0709 12:15:54.907681 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.907847 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":143479,"threads_started":1,"wal-append.queue_time_us":207288}
I20260709 12:15:54.913256 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:54.914240 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.913744 19266 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.887823 19231 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:54.890452 19486 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:54.920920 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3208) took 1006 ms. Trace:
I20260709 12:15:54.920521 19266 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:54.924605 19486 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:54.923763 11322 rpcz_store.cc:276] 0709 12:15:53.914322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.634153 (+719831us) service_pool.cc:224] Handling call
0709 12:15:54.634616 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:54.635243 (+   627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:54.635274 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:54.636052 (+   778us) log.cc:844] Serialized 26 byte log entry
0709 12:15:54.815689 (+179637us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:54.815893 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.816023 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.920718 (+104695us) raft_consensus.cc:1685] finished
0709 12:15:54.920736 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.920859 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3134,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":148,"thread_start_us":145182,"threads_started":1,"wal-append.queue_time_us":251567}
I20260709 12:15:54.920550 11330 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:15:54.917052 18642 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:54.939507 18642 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:54.941926 19498 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:54.945597 11933 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
W20260709 12:15:54.946519 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 71543.5us
W20260709 12:15:54.948614 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60552.4us
I20260709 12:15:54.950142 19512 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:15:54.955633 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 93545.5us
I20260709 12:15:54.964093 11255 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:54.965790 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.966661 11254 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:54.967680 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:54.968593 11249 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:54.973237 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.974159 11181 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:54.975041 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.975281 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:54.976709 11182 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 27. Ops: []
I20260709 12:15:54.977959 11173 raft_consensus.cc:1217] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Deduplicated request from leader. Original: 27.17->[28.18-28.18]   Dedup: 28.18->[]
I20260709 12:15:54.978518 11173 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 28. Current term is 30. Ops: [28.18-28.18]
I20260709 12:15:54.976606 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.001375 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.009887 11931 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:15:54.889524 11329 rpcz_store.cc:276] 0709 12:15:53.644081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.562374 (+918293us) service_pool.cc:224] Handling call
0709 12:15:54.607976 (+ 45602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:54.647935 (+ 39959us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:54.780849 (+132914us) log.cc:844] Serialized 14 byte log entry
0709 12:15:54.885918 (+105069us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:54.885996 (+    78us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:54.886045 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.886786 (+   741us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.886927 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23159,"prepare.queue_time_us":128847,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":26,"raft.queue_time_us":15352,"thread_start_us":180876,"threads_started":2}
I20260709 12:15:55.016989 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:55.023658 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.027459 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.028565 11926 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 28 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
W20260709 12:15:55.036430 19174 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.003s
I20260709 12:15:55.038983 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.039868 11251 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:55.043186 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.044076 11328 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:55.044317 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:55.048045 19501 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.796s	user 0.000s	sys 0.016s
I20260709 12:15:55.052590 19490 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:55.042080 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.055790 19285 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:55.062734 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.000660 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:55.064352 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:55.064862 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 113812us
W20260709 12:15:55.069350 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.068836 18816 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:55.062317 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.067901 11223 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:55.074731 19588 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:55.075335 19588 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:55.075726 19588 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:55.077929 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:55.079304 19543 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:55.080039 19543 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:55.080821 19522 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:55.082898 11253 raft_consensus.cc:2410] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 12:15:55.083240 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.018339 19564 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.085127 19285 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:55.085448 19564 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:55.086220 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.086988 11322 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:15:55.087882 19564 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:55.087976 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.089210 11322 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
W20260709 12:15:55.089675 19594 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:15:55.097940 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:55.099193 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 214678us
I20260709 12:15:55.100317 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:55.102859 19266 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:55.103359 19266 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:55.103794 19266 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:55.104276 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:55.105341 11926 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:55.105808 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.106033 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.106371 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.106760 19488 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:55.107095 11930 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:15:55.107168 11254 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:15:55.079205 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53287.4us
I20260709 12:15:55.108500 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:55.109441 11932 raft_consensus.cc:2368] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27.
I20260709 12:15:55.109706 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:55.110404 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.110672 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.112041 11329 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:55.112278 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:55.113185 11927 raft_consensus.cc:2368] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23.
I20260709 12:15:55.110666 18642 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:55.114985 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3201) took 1446 ms. Trace:
I20260709 12:15:55.115540 11328 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:15:55.115417 11323 rpcz_store.cc:276] 0709 12:15:53.668500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.582009 (+913509us) service_pool.cc:224] Handling call
0709 12:15:54.582409 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:54.582933 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:54.582975 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:54.591160 (+  8185us) log.cc:844] Serialized 26 byte log entry
0709 12:15:54.781205 (+190045us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:54.781243 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.781360 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:55.114782 (+333422us) raft_consensus.cc:1685] finished
0709 12:15:55.114803 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:55.114931 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":184507,"threads_started":1,"wal-append.queue_time_us":516123}
I20260709 12:15:55.118903 11251 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:15:55.120846 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.122330 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.123071 11251 raft_consensus.cc:2368] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23.
I20260709 12:15:55.105960 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.125248 11933 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:15:55.120846 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:55.129137 19522 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.129575 18816 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:55.130062 18816 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:55.130477 18816 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:55.130257 19486 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:55.130072 19522 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:55.127671 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59855.9us
I20260709 12:15:55.133513 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.135099 19522 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:55.134919 19512 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:55.136024 19512 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:15:55.136471 19512 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:55.136847 19498 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:55.137742 19512 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.138712 11253 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:15:55.139252 19512 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:55.142464 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:55.143059 19231 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:55.143699 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:55.144246 19501 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:55.144645 19159 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:55.144860 19544 raft_consensus.cc:2764] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:15:55.145159 19159 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:55.059422 19445 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:55.146253 19525 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:55.146369 19159 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.144680 19501 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:55.148429 19501 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:55.147873 19445 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:55.153932 11931 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27. Ops: []
I20260709 12:15:55.140859 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.155946 11324 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:55.156792 19159 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:55.158008 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.158799 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.158919 19498 consensus_queue.cc:1059] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:55.159641 11254 raft_consensus.cc:2410] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:15:55.160101 11256 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:15:55.160204 19498 raft_consensus.cc:3055] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:55.160590 19498 raft_consensus.cc:740] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:55.161433 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.162022 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.162197 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:55.162657 11257 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:55.158907 11252 raft_consensus.cc:2393] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:55.163997 19498 consensus_queue.cc:260] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:55.160624 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.143890 19403 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:55.165748 11253 raft_consensus.cc:2393] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:55.165850 19403 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:55.166376 19403 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:55.171877 11928 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:15:55.196928 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.197484 19498 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:55.198060 11926 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:55.199290 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:55.200927 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.201784 11252 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:55.196820 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.178403 18816 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.204993 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:55.206571 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.210449 11248 raft_consensus.cc:1240] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27. Ops: []
I20260709 12:15:55.212241 11250 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:15:55.215194 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.215977 11252 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:55.221011 19445 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:15:55.222080 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 76774.9us
I20260709 12:15:55.222328 19445 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:55.224756 19445 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:55.228896 18816 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:55.248742 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.252851 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.253719 11252 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:55.254472 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.255266 11175 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:55.252971 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:55.256362 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.257499 11182 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:55.256474 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.262202 11177 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 18.
W20260709 12:15:55.262369 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54373us
I20260709 12:15:55.262660 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.264602 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:55.266839 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.267624 11933 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:15:55.268070 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.268411 19422 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:55.262193 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.268924 19422 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:55.269104 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.270049 11935 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
W20260709 12:15:55.270133 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 97551.6us
I20260709 12:15:55.271010 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.271767 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:55.272078 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:55.273509 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.278141 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.279218 11931 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:55.253408 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.269392 19422 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:55.281313 18816 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:55.282859 11176 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:15:55.285351 11252 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:55.286057 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:55.289359 11257 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:15:55.290733 11253 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:15:55.292136 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.291636 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.292523 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.205214 19490 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.295498 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.297612 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.298859 11257 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:15:55.299798 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:55.301491 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 59361.1us
I20260709 12:15:55.302356 11254 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:15:55.314596 11253 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:55.314663 11254 pending_rounds.cc:85] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 11
I20260709 12:15:55.315171 11254 pending_rounds.cc:107] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:15:55.315462 11254 raft_consensus.cc:2889] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:55.302983 19403 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:55.303035 19490 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.304077 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.307155 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:55.304111 11177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:55.309747 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.309979 19006 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:15:55.310540 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.310950 11926 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:15:55.311609 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.311672 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.330677 11176 raft_consensus.cc:2368] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:15:55.330785 11257 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:15:55.324946 11182 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
W20260709 12:15:55.331822 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.324334 11177 raft_consensus.cc:606] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:55.335310 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:55.334583 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:55.346890 11331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.021s
I20260709 12:15:55.347609 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:55.357863 11329 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 28
W20260709 12:15:55.366107 11331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.000s	sys 0.021s
I20260709 12:15:55.391691 11329 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:15:55.397912 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:55.398247 11901 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:55.403234 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.404620 18972 raft_consensus.cc:2804] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:15:55.405362 18972 raft_consensus.cc:697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:55.408466 11329 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:15:55.326296 19588 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.414029 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.423915 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:55.425498 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.426729 11176 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:55.417586 18816 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:55.428437 18816 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:55.343681 19522 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:55.428905 18816 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:55.430125 11257 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 12:15:55.322650 19501 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.432704 11222 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:55.442741 19231 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:55.443853 19231 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.433987 11257 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:15:55.453869 19231 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:55.410868 19445 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:55.480384 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.511035 11252 raft_consensus.cc:2410] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 12:15:55.483963 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.485507 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:55.485855 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 77625.3us
I20260709 12:15:55.485502 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.487932 19543 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:55.489748 19545 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:15:55.490309 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 163718us
I20260709 12:15:55.492246 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3554) took 1418 ms. Trace:
I20260709 12:15:55.516469 11331 rpcz_store.cc:276] 0709 12:15:54.082260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.651567 (+569307us) service_pool.cc:224] Handling call
0709 12:15:54.652158 (+   591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:54.660882 (+  8724us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:15:54.737319 (+ 76437us) log.cc:844] Serialized 14 byte log entry
0709 12:15:55.368722 (+631403us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:55.369430 (+   708us) log.cc:844] Serialized 26 byte log entry
0709 12:15:55.369598 (+   168us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:55.369665 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.369758 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:55.491949 (+122191us) raft_consensus.cc:1685] finished
0709 12:15:55.491977 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:55.492182 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":128984,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":43,"raft.queue_time_us":1361,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":330,"thread_start_us":611818,"threads_started":2,"wal-append.queue_time_us":715596}
I20260709 12:15:55.502724 19501 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:55.406608 18972 consensus_queue.cc:237] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.505820 19266 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.506886 11932 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:55.507203 11178 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:15:55.508630 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:55.509025 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:55.508628 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:55.515239 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.526012 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.517575 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.499115 19490 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:55.412075 19498 consensus_queue.cc:1059] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 12:15:55.530192 11256 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 18.
I20260709 12:15:55.531286 11931 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:55.432404 19522 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:55.539098 11248 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 12:15:55.539707 11248 pending_rounds.cc:85] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 13
I20260709 12:15:55.540103 11248 pending_rounds.cc:107] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 12:15:55.539776 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.544947 19498 raft_consensus.cc:3055] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:55.545476 19498 raft_consensus.cc:740] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:55.546470 19498 consensus_queue.cc:260] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:55.548117 19498 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:55.550501 11147 leader_election.cc:400] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23.
I20260709 12:15:55.550869 11147 leader_election.cc:403] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:15:55.554061 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3206) took 1717 ms. Trace:
I20260709 12:15:55.554499 11326 rpcz_store.cc:276] 0709 12:15:53.836891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.607778 (+770887us) service_pool.cc:224] Handling call
0709 12:15:54.731882 (+124104us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:54.828039 (+ 96157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:54.828089 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:54.890550 (+ 62461us) log.cc:844] Serialized 52 byte log entry
0709 12:15:55.343420 (+452870us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:55.416805 (+ 73385us) log.cc:844] Serialized 14 byte log entry
0709 12:15:55.417038 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.417109 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:55.553830 (+136721us) raft_consensus.cc:1685] finished
0709 12:15:55.553852 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:55.553997 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":73902,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":21462,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"spinlock_wait_cycles":8583040,"thread_start_us":432264,"threads_started":1,"wal-append.queue_time_us":649017}
I20260709 12:15:55.556589 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:55.558162 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.558921 11327 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:55.559966 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.560817 11327 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:15:55.564931 19445 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23.
I20260709 12:15:55.568535 19266 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:55.571235 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.571540 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:55.572235 11296 connection.cc:256] RPC call timeout handler was delayed by -3.684401785000091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 12:15:55.572681 11296 connection.cc:256] RPC call timeout handler was delayed by -3.6817017850000866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
W20260709 12:15:55.572984 11296 connection.cc:256] RPC call timeout handler was delayed by -3.5899017849999382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 12:15:55.573266 11296 connection.cc:256] RPC call timeout handler was delayed by -2.947301785000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24700000000000003s for a response
I20260709 12:15:55.573802 19567 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:55.573875 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.574611 19567 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:55.575945 11256 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:55.577442 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.579011 11256 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
W20260709 12:15:55.580858 19603 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:15:55.536386 19522 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:55.588999 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:55.541819 11331 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:55.591344 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.591821 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:15:55.591295 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.598201 11928 raft_consensus.cc:1240] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 28. Current term is 30. Ops: []
I20260709 12:15:55.598387 11182 raft_consensus.cc:2410] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 31 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:15:55.598387 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:55.600410 11248 raft_consensus.cc:2889] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:55.577307 19567 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:55.606043 11173 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:15:55.608057 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:55.612040 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:55.614684 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.616885 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.619943 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.628760 11329 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:55.635449 11327 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:55.638025 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:55.638913 11331 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:55.641515 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:55.642463 11257 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:55.646347 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:55.646514 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:55.649876 11256 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 12:15:55.659230 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.618669 11330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.009s	sys 0.012s
I20260709 12:15:55.606592 19588 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:55.664408 11177 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:55.669152 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.669564 18816 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:55.674078 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.674244 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47315.3us
W20260709 12:15:55.644119 19174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.018s	sys 0.008s
W20260709 12:15:55.675735 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.675938 11176 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
W20260709 12:15:55.678622 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.683300 11323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.018s	sys 0.017s
W20260709 12:15:55.683823 11323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.018s	sys 0.017s
W20260709 12:15:55.661513 11330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.010s	sys 0.012s
W20260709 12:15:55.690281 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.678120 19174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.018s	sys 0.009s
I20260709 12:15:55.695099 18816 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:55.692826 19403 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:55.728852 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 125587us
W20260709 12:15:55.732399 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.733538 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.769366 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.704146 19422 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:55.799922 11325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.020s	sys 0.046s
W20260709 12:15:55.803406 11325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.020s	sys 0.046s
I20260709 12:15:55.804363 11253 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:55.809000 11296 leader_election.cc:400] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26.
I20260709 12:15:55.821327 11257 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 19.
I20260709 12:15:55.828712 11178 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:55.829820 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:55.855574 11900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.020s
I20260709 12:15:55.834426 19611 raft_consensus.cc:2764] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:15:55.837872 19231 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:55.865059 19498 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:15:55.866845 11078 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:55.869331 19610 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:15:55.879163 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.887127 11900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.008s	sys 0.021s
W20260709 12:15:55.906453 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 356122us
I20260709 12:15:55.911175 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:55.913249 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 356441us
I20260709 12:15:55.865710 19501 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:55.866842 19595 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.198s	user 0.003s	sys 0.000s
I20260709 12:15:55.869555 11078 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:55.929751 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:55.943898 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 152356us
I20260709 12:15:55.961494 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:55.943713 19546 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.963297 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:55.972608 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.972904 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:55.955037 18642 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:56.012102 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 738471us
I20260709 12:15:55.971383 19231 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:55.949833 19501 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:55.972519 19445 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:56.015214 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 155748us
I20260709 12:15:55.974324 19403 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:55.993981 19174 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:56.032661 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:56.040561 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 114073us
I20260709 12:15:55.997438 18642 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.033821 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:56.046257 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:56.051200 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:56.033191 19422 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:56.026925 19403 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.040997 19174 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:56.024991 19445 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:56.015933 19546 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:56.049891 19285 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.062093 19614 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:56.073999 19614 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.074487 19614 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:56.078738 19602 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 12:15:56.080201 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1617) took 1019 ms. Trace:
I20260709 12:15:56.080513 19174 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:56.080657 11322 rpcz_store.cc:276] 0709 12:15:55.061261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:55.090026 (+ 28765us) service_pool.cc:224] Handling call
0709 12:15:55.090646 (+   620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:55.091271 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:55.091317 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:55.091898 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:15:55.517932 (+426034us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:55.517994 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.518083 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:56.078089 (+560006us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:56.079851 (+  1762us) raft_consensus.cc:1685] finished
0709 12:15:56.079872 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:56.080128 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6663,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":103,"spinlock_wait_cycles":12062080,"thread_start_us":425555,"threads_started":1,"wal-append.queue_time_us":613539}
I20260709 12:15:56.056330 19567 raft_consensus.cc:2764] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:15:56.063634 19403 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:56.064510 19159 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:56.090170 11900 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:56.071153 19445 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:56.090648 11078 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:56.066987 19090 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
W20260709 12:15:56.112443 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 84362.5us
I20260709 12:15:56.108307 19490 raft_consensus.cc:2804] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:15:56.115096 19620 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:15:56.116410 19621 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:15:56.117264 19546 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:56.119148 19607 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
W20260709 12:15:56.120062 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:56.120579 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:56.122318 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 56513.7us
I20260709 12:15:56.124049 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:56.126646 19606 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:15:56.133208 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:56.164371 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 122929us
I20260709 12:15:56.090133 19285 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:56.169162 11331 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.004s
I20260709 12:15:56.171471 18816 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:56.187683 18816 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.188246 18816 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:56.189168 11331 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:56.195808 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:56.198443 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64469.4us
I20260709 12:15:56.213908 19490 raft_consensus.cc:697] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:15:56.210850 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 85878.2us
W20260709 12:15:56.220253 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 87122.6us
I20260709 12:15:56.224068 11326 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:56.224700 11326 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:56.228317 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:56.229800 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63654.2us
I20260709 12:15:56.169926 19546 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:15:56.178290 19588 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:15:56.219309 19611 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:56.243683 19231 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:15:56.244230 19612 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 12:15:56.246327 19624 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.118s	user 0.002s	sys 0.000s
I20260709 12:15:56.263898 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3227) took 1092 ms. Trace:
I20260709 12:15:56.264458 11327 rpcz_store.cc:276] 0709 12:15:55.171907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:55.663176 (+491269us) service_pool.cc:224] Handling call
0709 12:15:55.678725 (+ 15549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:55.694069 (+ 15344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:55.694117 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:55.720256 (+ 26139us) log.cc:844] Serialized 26 byte log entry
0709 12:15:55.994573 (+274317us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:55.994613 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.994688 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:56.245734 (+251046us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:56.263516 (+ 17782us) raft_consensus.cc:1685] finished
0709 12:15:56.263550 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:56.263814 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7133,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6566,"spinlock_wait_cycles":6736128,"thread_start_us":255900,"threads_started":1,"wal-append.queue_time_us":330082}
W20260709 12:15:56.248164 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.253720 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1620) took 1089 ms. Trace:
I20260709 12:15:56.268792 11323 rpcz_store.cc:276] 0709 12:15:55.164195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:55.164339 (+   144us) service_pool.cc:224] Handling call
0709 12:15:55.164697 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:55.165143 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:55.165173 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:55.165814 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 12:15:55.684331 (+518517us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:55.684392 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.684486 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:56.253460 (+568974us) raft_consensus.cc:1685] finished
0709 12:15:56.253481 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:56.253617 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1310,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":80,"thread_start_us":518033,"threads_started":1,"wal-append.queue_time_us":690596}
W20260709 12:15:56.253924 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:56.270934 19615 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.342s	user 0.000s	sys 0.003s
I20260709 12:15:56.258683 19285 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.272204 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3222) took 1101 ms. Trace:
I20260709 12:15:56.272823 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:56.272610 11328 rpcz_store.cc:276] 0709 12:15:55.170490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:55.170641 (+   151us) service_pool.cc:224] Handling call
0709 12:15:55.207060 (+ 36419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:55.290148 (+ 83088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:55.290198 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:55.300363 (+ 10165us) log.cc:844] Serialized 26 byte log entry
0709 12:15:55.618830 (+318467us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:55.618891 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.618985 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:56.270637 (+651652us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b240025b248. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363e09d 00007f3ec627010e 00007f3ec628788c 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221
0709 12:15:56.271923 (+  1286us) raft_consensus.cc:1685] finished
0709 12:15:56.271945 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:56.272147 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":175,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":5119,"spinlock_wait_cycles":12294016,"thread_start_us":285664,"threads_started":1,"wal-append.queue_time_us":509040}
W20260709 12:15:56.275672 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:56.278644 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47828.9us
I20260709 12:15:56.280031 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:56.285766 19285 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:56.285454 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:56.292940 19285 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:56.293926 19490 consensus_queue.cc:237] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.293939 19285 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:56.295466 19285 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:56.299749 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.300789 19632 consensus_queue.cc:1059] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:56.302436 19618 raft_consensus.cc:3055] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:15:56.302987 19618 raft_consensus.cc:740] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:56.304121 19501 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:56.304263 19618 consensus_queue.cc:260] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.300805 11901 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:56.306420 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.234838 19174 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:56.309931 19174 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:56.310166 19486 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:56.322855 19618 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:56.323271 18642 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.324391 19632 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:56.326288 19006 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:56.326702 19006 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:56.327556 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:56.327678 19006 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:56.331766 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 217629us
I20260709 12:15:56.336862 18816 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.332685 11326 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:56.347729 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.350142 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:56.357738 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 122532us
I20260709 12:15:56.310380 19174 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:56.311607 19486 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:56.364109 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:56.359989 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:56.371937 11329 raft_consensus.cc:2393] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:56.352908 19522 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.766s	user 0.006s	sys 0.008s
I20260709 12:15:56.365854 18816 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:56.347215 19632 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:56.317222 19614 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:56.387127 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 80221.1us
I20260709 12:15:56.359596 19546 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:56.391631 11326 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:56.381065 19512 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:56.394586 11323 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:56.395569 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:56.397555 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:56.400336 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.402712 19046 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
W20260709 12:15:56.405025 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.384404 19632 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:56.435729 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 100205us
W20260709 12:15:56.438671 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 189448us
W20260709 12:15:56.439492 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63074.6us
I20260709 12:15:56.449172 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:56.394182 19006 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:56.382524 19522 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:56.461336 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 65871.3us
I20260709 12:15:56.462320 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:56.437294 19632 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:56.465934 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:56.439097 19611 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:56.476694 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:56.451840 19422 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.406883 19546 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.483299 11182 raft_consensus.cc:2393] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:56.480867 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 106977us
I20260709 12:15:56.490423 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:56.405745 19637 consensus_queue.cc:1059] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.234s
I20260709 12:15:56.472541 19564 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:56.502178 19631 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:56.502899 19639 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:15:56.502836 19631 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.480615 19512 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:15:56.480842 19501 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:56.504945 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:56.506877 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:56.510722 19567 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:56.511315 19567 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:56.511744 19567 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:56.495498 19564 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:56.516224 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:56.488018 19422 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.483829 19486 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.531857 19564 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:56.546476 19501 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:56.527431 19543 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:56.553328 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46565.8us
I20260709 12:15:56.540614 19512 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:56.524361 19403 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.558789 19650 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:15:56.548635 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 62484.6us
I20260709 12:15:56.578672 19631 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.579358 19445 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:56.582302 19636 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:15:56.587301 11296 leader_election.cc:400] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:15:56.588778 11296 leader_election.cc:403] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:56.621331 19422 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.622190 19285 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.627432 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:56.628202 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.593686 19614 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:56.629361 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:56.590065 19512 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:56.639669 19512 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:56.641850 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 97478.8us
I20260709 12:15:56.642309 19567 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:56.643394 19501 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.643913 19174 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.644076 19501 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.645195 19567 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:56.645313 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:56.652477 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 147615us
I20260709 12:15:56.592387 19445 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:56.655721 11326 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:15:56.662472 11326 pending_rounds.cc:85] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 14
I20260709 12:15:56.662950 11326 pending_rounds.cc:107] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:15:56.663431 11326 raft_consensus.cc:2889] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:56.660630 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:56.656365 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 81330.3us
W20260709 12:15:56.679281 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.590632 19543 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:56.675261 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.702165 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2858) took 1084 ms. Trace:
W20260709 12:15:56.737130 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 91901us
W20260709 12:15:56.755553 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.741474 11323 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:56.770987 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:56.736313 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:56.771797 11328 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:56.655403 19522 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:56.767180 11182 rpcz_store.cc:276] 0709 12:15:55.614218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:55.614427 (+   209us) service_pool.cc:224] Handling call
0709 12:15:56.698307 (+1083880us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b54001f3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:56.698482 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1992661888}
I20260709 12:15:56.674367 19614 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:56.799317 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 133748us
W20260709 12:15:56.713617 19630 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.584s	user 0.001s	sys 0.001s
I20260709 12:15:56.801574 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:56.672346 19654 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:15:56.712165 18642 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:56.805547 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:56.805541 11325 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:56.696585 19525 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:56.684214 19006 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:56.702082 19546 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:56.822780 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.716342 19285 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.782696 19588 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.693737 19567 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:15:56.660286 19231 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.139s
W20260709 12:15:56.819955 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.720294 19564 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:56.821883 19006 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.837386 19006 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:56.838088 19640 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:15:56.840919 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:56.702082 19445 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.117s
I20260709 12:15:56.849848 11324 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:15:56.850999 19658 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:15:56.851197 19646 raft_consensus.cc:3055] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:56.851681 19646 raft_consensus.cc:740] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:56.832983 19403 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.704727 19619 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 12:15:56.829059 19545 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:56.854496 19545 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:56.855974 19174 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.703480 19649 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.118s
I20260709 12:15:56.796583 19498 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:15:56.858676 11078 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 18 to 20, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 20 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:56.860213 19498 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:56.694527 19632 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:56.842839 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 106669us
I20260709 12:15:56.860886 11078 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 21 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 25 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:56.823149 19501 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.855576 19545 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.808873 19614 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.861037 19498 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:56.874262 11330 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
W20260709 12:15:56.878517 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.883239 19006 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:56.869678 18972 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:56.830207 19522 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:15:56.898378 19006 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:56.887961 18972 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.852833 19646 consensus_queue.cc:260] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:56.901016 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:56.835457 19588 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:56.908293 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 102000us
I20260709 12:15:56.909389 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:56.910068 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 212810us
I20260709 12:15:56.910778 18972 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:56.913152 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.916734 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:56.921957 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:56.924247 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.932487 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:56.933519 11250 raft_consensus.cc:2410] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 12:15:56.935930 19646 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:56.937302 19614 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:56.938083 19648 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:56.919389 19090 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:15:56.940158 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:56.942024 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:56.942493 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:56.942708 11176 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:56.947492 19498 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:56.947875 19618 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.000s
I20260709 12:15:56.941578 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:56.948949 11178 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:56.950629 19564 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:56.952363 11324 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:56.952597 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57071.1us
I20260709 12:15:56.953310 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:56.953388 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:56.953778 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:56.954264 11249 raft_consensus.cc:2368] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22.
I20260709 12:15:56.954696 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:56.952422 19543 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:56.955322 19670 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:56.955780 19670 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.956116 19670 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:56.958308 11253 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 16. (term mismatch)
I20260709 12:15:56.958734 11253 pending_rounds.cc:85] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 15
I20260709 12:15:56.958950 11253 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:15:56.959215 11253 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:56.958946 11257 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:15:56.959800 11257 pending_rounds.cc:85] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 13
I20260709 12:15:56.960032 11257 pending_rounds.cc:107] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:15:56.960261 11257 raft_consensus.cc:2889] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:56.889796 19654 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:56.964138 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:56.974915 19642 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:56.976141 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:56.976951 11177 raft_consensus.cc:2393] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:56.976892 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:56.974323 19543 consensus_queue.cc:1059] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:15:56.977741 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:56.983587 11182 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:15:56.978012 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:15:56.985212 11072 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 173) took 26352 ms (client timeout 15000 ms). Trace:
I20260709 12:15:56.983150 19498 raft_consensus.cc:3055] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:56.983070 19588 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:56.977061 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:56.998432 11176 raft_consensus.cc:3060] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:56.998741 11328 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:56.991820 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:56.977114 19635 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.917866 19006 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 12:15:56.981769 19664 consensus_queue.cc:1059] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
W20260709 12:15:56.982007 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 72737.7us
I20260709 12:15:56.975831 19642 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:56.922065 19512 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:15:56.996841 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:56.981828 19564 raft_consensus.cc:2764] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election decision vote started in defunct term 24: won
I20260709 12:15:56.982074 11078 catalog_manager.cc:5697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 14 to 19, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 19 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.005451 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:56.999758 11256 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:57.000761 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.001682 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.008782 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:57.011364 11928 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:57.012704 19632 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:57.013126 11251 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 30 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:15:57.013873 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:57.009568 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:56.997884 19498 raft_consensus.cc:740] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:57.024089 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:57.024255 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:57.024914 11175 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:57.024542 11078 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.025938 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:57.006896 19543 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:15:57.028388 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:57.027237 11147 leader_election.cc:400] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27.
I20260709 12:15:57.026513 11078 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.031883 11325 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:15:57.004664 19619 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:57.040126 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:57.041232 11178 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:57.021317 19659 raft_consensus.cc:3055] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:57.042766 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:57.021767 19090 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:15:57.019011 19512 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:57.027136 19640 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:57.027401 19648 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:57.038532 11078 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 16 to 20, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.051893 11222 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:57.028635 19543 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:15:57.082491 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.081048 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.084692 11935 raft_consensus.cc:2393] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 28: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:57.084833 11928 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:57.063153 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:57.063278 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:57.061118 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:57.086422 11248 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:57.069190 11176 raft_consensus.cc:2468] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:15:57.073848 19046 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.074111 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:57.049289 19512 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:15:57.004639 19635 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.088977 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.089769 11177 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:57.027928 19498 consensus_queue.cc:260] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.090083 11255 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:57.090245 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:57.076467 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.077889 19611 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:15:57.078874 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 51785.3us
I20260709 12:15:57.092283 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.079876 11932 raft_consensus.cc:1217] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Deduplicated request from leader. Original: 24.15->[27.16-27.16]   Dedup: 27.16->[]
I20260709 12:15:57.080124 11078 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 14 to 18, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 18 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:57.080806 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.093789 11933 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:57.093609 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.081718 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.082592 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:57.094853 11252 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:57.095206 11179 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:57.095741 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.095880 19642 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.096475 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.088275 19654 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.097489 11932 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:15:57.097546 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:57.098857 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:57.098966 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:57.045076 19659 raft_consensus.cc:740] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:57.100831 19659 consensus_queue.cc:260] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.101948 19611 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:57.102183 19659 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:57.102720 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.102955 19611 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.104617 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:57.105203 11322 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:57.106351 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:57.108398 11927 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:57.086130 11254 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:57.087401 11256 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:57.088047 11181 raft_consensus.cc:1240] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:57.088394 11173 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:15:57.068332 11175 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:15:57.109460 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.089480 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.075572 19422 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:57.081023 11931 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:15:57.110625 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.111500 11176 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:57.110622 11177 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
W20260709 12:15:57.115617 11222 connection.cc:256] RPC call timeout handler was delayed by -2.8225184969999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:15:57.116317 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:57.129060 11252 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:15:57.130188 11248 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:57.049108 19231 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:57.132820 11250 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:15:57.134073 11177 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:57.134851 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.124478 11078 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 18 to 22, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 22 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.139010 11182 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:57.142313 11900 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:57.140378 11178 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:57.144676 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.146085 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:57.147364 11078 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 16 to 20, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.149310 11181 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:15:57.141072 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:57.156471 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.156636 11177 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:57.040557 19619 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.159492 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:57.161482 11325 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:15:57.163178 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:57.165699 11928 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:15:57.166887 11178 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:57.155524 19631 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.168280 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.169169 11175 raft_consensus.cc:2410] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:15:57.173178 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:57.177446 11181 raft_consensus.cc:1240] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:15:57.148914 19650 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.206322 19650 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.108419 19046 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.212423 19046 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:57.167853 19486 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:57.214267 18972 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:57.214898 18972 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.215370 18972 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:57.218003 19631 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.221215 19631 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.183820 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:57.187127 19635 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:57.187721 19498 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:57.161156 19090 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:57.243458 19090 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:57.243932 19501 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:57.244376 19501 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:57.233301 19672 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.245249 19501 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.246213 19672 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.149971 19614 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.194078 11179 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:57.246814 19501 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:57.250209 19614 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.254717 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.255594 11327 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
W20260709 12:15:57.248920 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.259037 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:57.248700 11178 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:57.260344 19543 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:57.260753 19543 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:57.260869 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.261670 19543 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.263303 19543 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:56.992174 11072 rpcz_store.cc:269] 0709 12:15:30.632299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.637847 (+  5548us) service_pool.cc:224] Handling call
0709 12:15:56.984929 (+26347082us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:53.523083 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:53.532153 (+  9070us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:53.532219 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:15:53.532233 (+    14us) tablet.cc:662] Decoding operations
0709 12:15:53.536084 (+  3851us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:53.536182 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:53.536212 (+    30us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:15:53.537781 (+  1569us) tablet.cc:701] Row locks acquired
0709 12:15:53.537832 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:15:53.538036 (+   204us) write_op.cc:270] Start()
0709 12:15:53.538189 (+   153us) write_op.cc:276] Timestamp: P: 1783599353537956 usec, L: 0
0709 12:15:53.538209 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:15:53.538940 (+   731us) log.cc:844] Serialized 7557 byte log entry
0709 12:15:54.593596 (+1054656us) op_driver.cc:464] REPLICATION: finished
0709 12:15:54.893398 (+299802us) write_op.cc:301] APPLY: starting
0709 12:15:54.924242 (+ 30844us) spinlock_profiling.cc:243] Waited 985 us on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66e932a 00007f3ec666ef14 00007f3ec666eec3 00007f3ec666ee38 00007f3ec666edf3 00007f3ec666e0dc 00007f3ec6697e16 00007f3ec667fcbb 00007f3ec666a39d 00007f3ec666dd75
0709 12:15:54.952107 (+ 27865us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:55.056032 (+103925us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:55.056077 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:55.224034 (+167957us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:55.224506 (+   472us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 12:15:55.224581 (+    75us) write_op.cc:312] APPLY: finished
0709 12:15:55.225281 (+   700us) log.cc:844] Serialized 172 byte log entry
0709 12:15:55.225513 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:55.236113 (+ 10600us) write_op.cc:454] Released schema lock
0709 12:15:55.236193 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:55.236706 (+   513us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:55.236740 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1143158,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":268561,"apply.run_cpu_time_us":35923,"apply.run_wall_time_us":352965,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83699,"num_ops":19,"prepare.queue_time_us":928946,"prepare.run_cpu_time_us":24688,"prepare.run_wall_time_us":747396,"raft.queue_time_us":346029,"raft.run_cpu_time_us":14387,"raft.run_wall_time_us":337953,"replication_time_us":1052572,"spinlock_wait_cycles":2264448,"thread_start_us":1734771,"threads_started":4,"wal-append.queue_time_us":681831,"wal-append.run_cpu_time_us":29711,"wal-append.run_wall_time_us":1998260}]]}
I20260709 12:15:57.233552 19640 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.268453 19640 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.270702 19640 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.244822 19090 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.273587 19614 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.274611 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.195660 19619 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:57.197853 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 136783us
I20260709 12:15:57.279655 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:57.199901 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:57.203074 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 110074us
I20260709 12:15:57.281011 11147 leader_election.cc:400] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23.
I20260709 12:15:57.221736 19632 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:57.172452 19266 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:15:57.181571 11078 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 14 to 17, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 17 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.233237 11181 raft_consensus.cc:1240] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:57.229460 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50428.8us
I20260709 12:15:57.174114 19564 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:57.209385 19650 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:57.309715 11078 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:57.300884 11178 raft_consensus.cc:1240] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 26. Current term is 27. Ops: [26.13-26.13]
W20260709 12:15:57.304137 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 45226.1us
I20260709 12:15:57.308578 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.309132 11331 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:57.284394 11254 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:57.316071 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:57.319594 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.319844 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.320236 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.320607 11181 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:57.309291 19544 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.321825 19544 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.324921 19490 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:15:57.325987 19642 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.326304 19635 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:57.326570 19642 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.301884 19266 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.328070 19266 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:57.328732 19642 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.326152 19544 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.331435 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.333480 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.334175 11331 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
W20260709 12:15:57.335103 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.336853 19564 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:57.338480 19564 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:57.341980 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.342876 19490 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.321451 19632 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:57.344723 19632 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.345772 11256 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:15:57.345968 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:57.346268 19422 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:57.347239 19422 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:57.347291 18642 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:57.347679 19422 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:57.347874 18642 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.348333 18642 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:57.343441 19490 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.351240 19490 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.351810 19670 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:57.354035 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 170322us
I20260709 12:15:57.354892 11900 leader_election.cc:400] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22.
W20260709 12:15:57.355800 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.356581 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.357224 11181 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 21.
I20260709 12:15:57.357432 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.357618 19046 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:57.360813 11331 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
W20260709 12:15:57.361917 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 81046.4us
I20260709 12:15:57.363216 11180 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:15:57.364919 11325 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:15:57.321525 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:57.366699 11173 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 22: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:57.345499 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:57.377907 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:57.248528 19672 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:15:57.387046 11322 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:15:57.408382 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:57.409348 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46551.2us
W20260709 12:15:57.410113 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 47533.1us
W20260709 12:15:57.404034 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46379.4us
I20260709 12:15:57.357754 19641 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:57.417107 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:57.411355 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56906.7us
I20260709 12:15:57.308754 19159 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.415514 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:57.423558 18816 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:15:57.386080 11078 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 14 to 17, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 17 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.419008 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.404510 19544 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:57.423991 11296 leader_election.cc:400] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
W20260709 12:15:57.429330 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.412770 19642 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:15:57.430904 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.348625 19422 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.446959 11173 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:15:57.453804 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.454802 18642 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.457026 11148 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:57.422040 19641 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.406955 19498 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:15:57.465579 11078 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:15:57.427973 19501 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:57.476264 11180 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:15:57.477818 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:57.418382 19231 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:15:57.357774 19486 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.099s
I20260709 12:15:57.482303 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:57.482760 19672 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:57.482560 11078 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 22, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 22 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:57.483374 19654 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:15:57.483876 19544 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:57.414232 19512 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.425346 19159 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.468623 19656 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:57.465310 19641 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:15:57.519609 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 92317us
I20260709 12:15:57.430892 19670 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:57.504940 18642 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:15:57.513546 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 98141.4us
I20260709 12:15:57.530071 18642 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.524385 11078 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 18, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.530563 18642 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:57.513248 19422 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:57.537618 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 55395.5us
I20260709 12:15:57.540319 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:57.543684 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:57.545683 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:57.523762 19656 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:57.546248 19664 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:57.546682 19664 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:57.547408 11900 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:57.505280 19642 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:15:57.551120 18642 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.555670 11078 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:57.558696 18816 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:15:57.558852 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:57.559334 18816 raft_consensus.cc:697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:57.536837 19659 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:57.584147 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:57.555984 19614 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:57.602303 18816 consensus_queue.cc:237] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.556951 19670 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:57.592512 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:57.614241 11179 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:15:57.559178 19490 consensus_queue.cc:1059] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:15:57.554466 19266 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.606290 19654 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:15:57.628754 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 212049us
I20260709 12:15:57.610754 19486 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:57.632658 19659 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:15:57.612887 19614 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:57.634805 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:57.605772 19672 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:15:57.636790 11324 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:57.633283 19659 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:57.639653 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.639935 11175 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:57.635855 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:57.629782 19654 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:57.631031 19490 consensus_queue.cc:1243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:15:57.634833 19631 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.581648 19656 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:57.657181 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.641541 19659 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:57.641405 19486 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:57.647989 19611 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:57.668398 19614 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:57.670267 19681 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:15:57.652298 19654 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:57.679527 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 262497us
I20260709 12:15:57.689778 19611 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:57.671208 19682 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:57.650784 19672 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:57.714905 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:57.729501 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:57.733275 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.734551 11322 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:57.737525 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:57.717890 19544 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:57.658018 19631 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:15:57.772629 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:57.776074 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:57.778271 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49080.6us
W20260709 12:15:57.845377 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.727557 19646 raft_consensus.cc:3055] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:57.854257 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.742666 19159 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.855914 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:57.857013 19611 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:57.714810 19664 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.860348 19664 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:57.860716 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:57.846393 19231 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:57.857762 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:57.851410 19486 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.876654 11176 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:57.876654 19672 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:57.877753 19672 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.879166 19641 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.742115 19642 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:57.908571 11324 raft_consensus.cc:2468] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:15:57.719642 19564 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:57.846283 19543 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:57.754323 19544 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:57.906940 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 016411114c6d42039bd04025378dfc98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:15:57.923491 11065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 207416us
W20260709 12:15:57.926137 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 367367us
I20260709 12:15:57.851166 19646 raft_consensus.cc:740] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:57.926882 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:57.925954 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65888.7us
W20260709 12:15:57.929303 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.916555 19501 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:15:57.921499 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 376542us
I20260709 12:15:57.917781 19543 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.941895 19046 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:57.938817 19611 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.041656 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:57.946802 19543 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:58.048445 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:58.039326 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 194664us
I20260709 12:15:58.044696 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:57.947964 19266 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:57.946296 19501 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:58.038954 11898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:58.051996 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 416246us
I20260709 12:15:58.060452 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:58.066327 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:58.067093 11252 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:58.074362 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:58.075057 11257 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: []
I20260709 12:15:58.079525 11935 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:58.049178 19588 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.082360 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.083880 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:58.087845 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:58.091940 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:58.101734 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:58.096254 11928 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:15:58.100811 11935 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:15:58.101920 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:58.110285 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:58.068953 19646 consensus_queue.cc:260] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.083604 19672 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:58.118060 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.118760 11324 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:15:58.066138 19611 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.100776 19700 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:15:58.096026 19702 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:58.129530 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:58.129884 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:58.130492 11248 raft_consensus.cc:2393] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:15:58.093874 19588 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.075470 19544 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.127375 19672 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:58.098033 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 171256us
W20260709 12:15:58.138530 19684 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.296s	user 0.001s	sys 0.001s
W20260709 12:15:58.193449 19701 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 12:15:58.194456 19703 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:58.196368 19611 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.197696 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:58.143726 19641 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.198752 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:15:58.143503 19687 log.cc:927] Time spent T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:15:58.199718 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:58.149921 19631 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.203056 19543 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.098454 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:58.204458 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:58.214604 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:58.219553 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 70082.6us
W20260709 12:15:58.235236 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 136858us
I20260709 12:15:58.133330 19700 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:58.208287 11223 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:58.227147 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:58.226888 19670 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.616s	user 0.013s	sys 0.000s
I20260709 12:15:58.113121 19501 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.158739 19564 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:58.231841 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:15:58.219475 19422 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
W20260709 12:15:58.233435 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 139218us
I20260709 12:15:58.235373 11325 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:15:58.236295 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:58.230475 11927 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:15:58.238756 11182 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:58.197906 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:58.243603 19694 raft_consensus.cc:2804] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:58.246024 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:58.247360 11324 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:58.265144 11330 raft_consensus.cc:2410] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:15:58.248025 18642 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.249022 11323 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:58.249992 19611 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.262740 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.269951 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.271510 19498 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:58.271754 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:58.235472 19544 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:58.267464 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:58.276230 18642 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.283351 11181 raft_consensus.cc:2410] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:15:58.284615 11250 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:15:58.275972 11174 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:15:58.266355 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.279093 11331 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:15:58.280447 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:58.288802 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:58.239392 19696 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:58.298285 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:58.301992 11323 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:58.263418 19670 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:58.281936 19693 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:15:58.318003 11222 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:58.321309 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 92644.6us
I20260709 12:15:58.281036 19690 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:58.282855 19642 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:58.336791 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:58.336781 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:58.234144 19698 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 12:15:58.338932 18642 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:58.296069 19501 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:58.290948 19694 raft_consensus.cc:697] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:58.297446 19635 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:58.343571 18642 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:58.345310 11331 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:15:58.288283 19611 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.392913 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:15:58.393129 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44790.6us
I20260709 12:15:58.288538 19231 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:15:58.344097 19090 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.424705 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.419001 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:15:58.427860 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.429562 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.426690 19691 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:15:58.430294 19642 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:58.435619 19642 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:58.438468 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.439754 19642 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:15:58.440369 19642 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.440873 19642 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:15:58.442273 19716 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:15:58.445256 18972 consensus_meta.cc:287] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.229s	user 0.009s	sys 0.005s
I20260709 12:15:58.445864 18972 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.447837 19543 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.448483 19611 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.448616 19588 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.450609 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.426047 19690 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:15:58.454401 19717 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 12:15:58.457352 19715 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:15:58.416164 19700 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.348449 19696 consensus_queue.cc:1059] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 12:15:58.461778 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.430480 19159 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:58.457089 19543 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:58.462967 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51897.4us
I20260709 12:15:58.465158 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:58.468593 11331 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:15:58.468768 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43944.2us
I20260709 12:15:58.470901 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.471298 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:58.488404 19694 consensus_queue.cc:237] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.516543 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:58.517726 11324 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
W20260709 12:15:58.522471 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51689.2us
I20260709 12:15:58.499835 19159 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.536499 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:58.537405 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:58.422753 19266 consensus_queue.cc:1059] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.078s
I20260709 12:15:58.498832 19668 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:58.550171 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.560956 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:58.428817 19641 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 12:15:58.576408 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:58.571061 11331 raft_consensus.cc:2468] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:15:58.577875 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:15:58.572299 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57324.1us
W20260709 12:15:58.531311 19719 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:15:58.518801 19648 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.579783 11223 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:58.580101 11330 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:15:58.577775 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43595.7us
I20260709 12:15:58.538198 19159 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.426450 19090 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.605237 19642 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.608566 19642 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.587898 19648 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.610136 11324 raft_consensus.cc:1275] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:15:58.611819 19648 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.605827 18642 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:15:58.613440 18642 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:58.614494 18642 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.615796 18642 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:58.619465 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:58.648550 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:58.654814 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:58.655494 11251 raft_consensus.cc:2393] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 29: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:58.519510 19588 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:15:58.573724 19668 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.580842 19498 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:15:58.678035 19717 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:15:58.692638 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:15:58.744369 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 164674us
I20260709 12:15:58.602263 19231 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:58.759496 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:58.520947 19724 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.177s
I20260709 12:15:58.755661 19710 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:58.770602 11251 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:58.678524 19635 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.768591 11324 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:15:58.690236 19668 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:58.775538 18642 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:58.783236 18642 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.791780 18642 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:58.792536 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 216562us
W20260709 12:15:58.785923 19664 consensus_meta.cc:287] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.922s	user 0.014s	sys 0.001s
I20260709 12:15:58.807286 18642 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.808468 11330 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:58.783012 19231 raft_consensus.cc:2804] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:15:58.822311 11325 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:58.822798 11325 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:58.822877 11322 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:15:58.823386 11322 pending_rounds.cc:85] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 15
I20260709 12:15:58.823695 11322 pending_rounds.cc:107] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:15:58.823959 11322 raft_consensus.cc:2889] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:58.797513 18972 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.790565 19422 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:58.841940 19711 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:58.843173 19726 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 12:15:58.848893 11323 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.000s
I20260709 12:15:58.859366 11323 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:15:58.855382 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:58.860177 11324 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:58.816377 19670 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.832309 11325 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.837720 19266 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 12:15:58.867919 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 102565us
I20260709 12:15:58.864230 11901 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:58.785951 19635 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.853926 18642 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:58.878616 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:15:58.881367 11325 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:58.881688 11323 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:58.832351 19656 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:58.889117 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 427445us
W20260709 12:15:58.866917 19486 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.000s	sys 0.004s
I20260709 12:15:58.817019 19724 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:15:58.894615 18642 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.902838 18642 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.903658 11926 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:58.903676 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:58.905134 11928 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:15:58.898394 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:58.906592 11182 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:15:58.898814 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:58.907243 11178 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:15:58.857324 19711 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:58.839076 19498 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:58.899535 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:58.897759 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.020s	sys 0.039s
W20260709 12:15:58.917071 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.020s	sys 0.039s
I20260709 12:15:58.917105 11173 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:15:58.908782 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:58.917872 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 655167us
I20260709 12:15:58.918769 11296 leader_election.cc:400] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:15:58.912532 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:58.919844 11926 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:15:58.861416 19231 raft_consensus.cc:697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:58.904158 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:58.914824 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:58.870102 19664 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.917979 11178 raft_consensus.cc:3060] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:58.892391 11323 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:58.925429 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:58.925815 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:58.924480 11223 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:58.938043 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:58.938565 11932 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:15:58.940615 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.945022 11175 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:15:58.946599 11253 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 29. Current term is 30. Ops: []
I20260709 12:15:58.905685 19588 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:58.854794 19728 consensus_queue.cc:1048] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.081s
I20260709 12:15:58.948663 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:58.957252 18642 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:58.960006 18642 raft_consensus.cc:2804] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:58.960618 18642 raft_consensus.cc:697] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:15:58.963366 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 379104us
I20260709 12:15:58.964612 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:58.965773 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.969170 11325 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:15:58.969081 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.961731 18642 consensus_queue.cc:237] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.914016 19711 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:58.880460 19670 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:58.871024 19422 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.988845 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.996753 19422 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:58.998801 19670 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:59.004012 19670 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:59.004951 19670 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:59.005846 19670 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:59.002246 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.008311 19670 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:59.013525 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:58.992575 11324 raft_consensus.cc:2468] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:59.015300 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:58.906186 19656 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.018374 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:59.033488 11173 raft_consensus.cc:2468] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
W20260709 12:15:59.039297 19736 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:15:59.041553 19635 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.041716 19739 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.042829 19743 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.044853 19741 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:15:59.047161 19654 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.325s	user 0.014s	sys 0.000s
I20260709 12:15:59.047672 19654 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:59.048305 19672 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.840s	user 0.011s	sys 0.000s
I20260709 12:15:59.048846 19672 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:59.049847 11328 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:59.052920 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.058714 11928 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 30.
I20260709 12:15:59.063549 11178 raft_consensus.cc:2468] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:15:59.065119 11253 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 12:15:59.065550 11253 pending_rounds.cc:85] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 14
I20260709 12:15:59.065869 11253 pending_rounds.cc:107] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:15:59.066088 11932 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 29. Current term is 30. Ops: []
I20260709 12:15:59.066125 11253 raft_consensus.cc:2889] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:58.938567 19498 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:58.984381 19588 raft_consensus.cc:697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:59.078254 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.081996 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.083526 11173 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:58.944727 19664 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.088495 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.034324 19670 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:59.086969 19672 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:59.034464 19090 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:59.081516 19726 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:15:59.109581 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:59.108462 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 95019.3us
I20260709 12:15:59.076480 19724 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:15:59.110659 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.102273 19747 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.104285 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.080855 19422 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.087265 19691 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.149822 11147 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.049912 19231 consensus_queue.cc:237] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.148234 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:59.094761 19700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.016s	sys 0.006s
I20260709 12:15:59.161046 11253 raft_consensus.cc:1240] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:59.161453 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.168603 11257 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:59.162400 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.170184 11173 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:59.162686 11933 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:59.106040 19706 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:59.074924 19611 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 12:15:59.105911 19725 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.175091 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.043366 19743 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.179936 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.180248 11180 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:15:59.180588 11182 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
W20260709 12:15:59.182027 19749 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:15:59.182639 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.184595 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:59.184820 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.185467 11173 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:59.186569 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.186667 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.189622 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.190065 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:59.190730 11933 raft_consensus.cc:2368] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22.
I20260709 12:15:59.192131 11180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.192940 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.193849 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:59.183245 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 217585us
I20260709 12:15:59.199273 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.076659 19738 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.099s
W20260709 12:15:59.174726 19700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.018s	sys 0.006s
I20260709 12:15:59.203936 11182 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:59.203630 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.208642 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.042258 19739 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.177531 19706 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.189021 19744 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:59.045504 19741 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.210397 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.211229 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:59.215380 19659 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.501s	user 0.003s	sys 0.011s
I20260709 12:15:59.215516 11174 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:15:59.174458 19670 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:59.204981 11930 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:59.217161 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.256830 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:15:59.257732 11933 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.222617 11256 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:15:59.222611 11926 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: []
I20260709 12:15:59.177738 19654 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.221921 19755 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:59.226253 19759 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:15:59.229060 19753 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.200300 19672 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.157071 19422 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.245119 11257 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:59.206444 19635 raft_consensus.cc:2764] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:15:59.231782 19706 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.262137 11180 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:59.256086 11177 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:59.164552 19691 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.216239 19486 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:15:59.217754 11173 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
I20260709 12:15:59.284827 11173 raft_consensus.cc:3055] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:59.205925 19724 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:15:59.255604 19670 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:59.276702 19654 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:15:59.279438 19760 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 12:15:59.280215 19763 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:59.280849 19691 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.280965 19422 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.281371 19753 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.285316 11173 raft_consensus.cc:740] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:59.151469 19747 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.276701 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:15:59.295337 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44698.3us
I20260709 12:15:59.297000 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3363) took 1189 ms. Trace:
I20260709 12:15:59.297078 11177 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:59.264526 19755 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:15:59.285332 19702 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.145s	user 0.010s	sys 0.002s
I20260709 12:15:59.301988 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.257905 19659 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.190798 19725 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.300266 19664 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:59.318512 19712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.028s	sys 0.048s
I20260709 12:15:59.318724 19769 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:59.342470 19769 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:59.343544 19769 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.298539 19670 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:59.315202 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:59.350939 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.351384 11253 raft_consensus.cc:2393] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:15:59.315058 11252 rpcz_store.cc:276] 0709 12:15:58.107191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.111873 (+  4682us) service_pool.cc:224] Handling call
0709 12:15:58.112244 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:58.205421 (+ 93177us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:58.339757 (+134336us) log.cc:844] Serialized 14 byte log entry
0709 12:15:58.431033 (+ 91276us) log.cc:844] Serialized 14 byte log entry
0709 12:15:58.579643 (+148610us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:58.584415 (+  4772us) log.cc:844] Serialized 26 byte log entry
0709 12:15:58.584565 (+   150us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:58.584803 (+   238us) log.cc:844] Serialized 14 byte log entry
0709 12:15:58.758829 (+174026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:58.758920 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:59.296722 (+537802us) raft_consensus.cc:1685] finished
0709 12:15:59.296746 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:59.296909 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":221506,"prepare.queue_time_us":237515,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":93,"raft.queue_time_us":302938,"raft.run_cpu_time_us":590,"raft.run_wall_time_us":24429,"spinlock_wait_cycles":737152,"thread_start_us":504827,"threads_started":4,"wal-append.queue_time_us":177078}
W20260709 12:15:59.324118 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 149114us
I20260709 12:15:59.244231 19588 consensus_queue.cc:237] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.356096 11174 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:15:59.330497 19702 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:59.316174 11933 raft_consensus.cc:2468] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:59.329969 19775 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:59.358537 11323 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:59.358973 11256 raft_consensus.cc:1240] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:59.276789 19672 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.360456 11252 raft_consensus.cc:1275] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:15:59.358116 19773 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:15:59.361795 11252 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:15:59.361881 19672 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:59.362867 19501 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.971s	user 0.013s	sys 0.000s
I20260709 12:15:59.298084 19654 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:59.326474 19732 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
W20260709 12:15:59.340808 19767 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:15:59.341974 19712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.028s	sys 0.048s
I20260709 12:15:59.323428 19755 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:59.367954 19659 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:59.368166 19747 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.369047 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3360) took 1294 ms. Trace:
I20260709 12:15:59.353794 19771 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:59.369504 19706 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.354065 19770 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:15:59.369423 11255 rpcz_store.cc:276] 0709 12:15:58.074818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.074929 (+   111us) service_pool.cc:224] Handling call
0709 12:15:58.075210 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:58.085519 (+ 10309us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:15:58.262112 (+176593us) log.cc:844] Serialized 14 byte log entry
0709 12:15:58.771432 (+509320us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:58.774775 (+  3343us) log.cc:844] Serialized 26 byte log entry
0709 12:15:58.774936 (+   161us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:58.795635 (+ 20699us) log.cc:844] Serialized 14 byte log entry
0709 12:15:58.828663 (+ 33028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:58.828762 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:59.368849 (+540087us) raft_consensus.cc:1685] finished
0709 12:15:59.368870 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:59.368991 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":196164,"prepare.queue_time_us":348581,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":3739,"raft.queue_time_us":343008,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":8776,"spinlock_wait_cycles":13247232,"thread_start_us":592282,"threads_started":3,"wal-append.queue_time_us":619002}
I20260709 12:15:59.354808 19778 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:15:59.356081 19741 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.321820 11173 consensus_queue.cc:260] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.377759 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.378589 11174 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.378700 11327 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:59.372284 11173 raft_consensus.cc:3060] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.363351 11252 raft_consensus.cc:1240] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
W20260709 12:15:59.340018 19748 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:15:59.370025 19771 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.383034 11177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c"
dest_uuid: "016411114c6d42039bd04025378dfc98"
mode: GRACEFUL
new_leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
 from {username='slave'} at 127.0.0.1:51810
I20260709 12:15:59.383689 11177 raft_consensus.cc:606] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Received request to transfer leadership to TS bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.384052 11177 raft_consensus.cc:614] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:59.385862 19635 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:59.381937 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.383224 19771 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.354377 19656 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.371918 19706 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:15:59.361835 19690 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.908s	user 0.008s	sys 0.004s
I20260709 12:15:59.354802 19739 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.407867 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:59.324113 19700 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.086s
I20260709 12:15:59.402369 19659 raft_consensus.cc:3055] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:15:59.404592 19778 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:15:59.435496 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 85914.1us
I20260709 12:15:59.376330 19670 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.436389 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:59.438560 11296 leader_election.cc:400] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22.
I20260709 12:15:59.438675 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.438892 11296 leader_election.cc:403] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:15:59.427261 19690 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:59.430933 19648 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:15:59.443673 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.444587 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.445356 11174 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:15:59.444651 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.445703 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.447624 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:59.447862 19782 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:15:59.448807 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.448537 19711 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.449890 19773 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:59.451128 19702 consensus_queue.cc:1059] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:59.452217 19702 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:15:59.452628 19702 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.452971 19702 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:15:59.454505 19544 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.164s	user 0.012s	sys 0.001s
I20260709 12:15:59.454772 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.455518 11254 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:15:59.455608 11177 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:59.456214 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.456830 11252 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:15:59.442406 19755 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.442667 19672 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) with higher term. Message: Invalid argument: T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22.
I20260709 12:15:59.458853 11931 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:59.459573 19672 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:59.460129 19672 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:59.449936 19664 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:15:59.464365 19755 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:59.465786 19711 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:59.466660 11173 raft_consensus.cc:1275] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:15:59.468518 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.469369 19702 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.471073 19614 raft_consensus.cc:3055] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:59.471555 19614 raft_consensus.cc:740] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:59.471625 19702 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.466591 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.472644 19614 consensus_queue.cc:260] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.474143 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.476509 19512 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.477913 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.478592 11177 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:59.479548 19734 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.660s	user 0.002s	sys 0.000s
I20260709 12:15:59.480860 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3365) took 1339 ms. Trace:
I20260709 12:15:59.481278 11248 rpcz_store.cc:276] 0709 12:15:58.141802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.142831 (+  1029us) service_pool.cc:224] Handling call
0709 12:15:58.143216 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:58.143652 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:58.143693 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:58.144258 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:15:58.537808 (+393550us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:58.537858 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:58.537933 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:59.478845 (+940912us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:15:59.480493 (+  1648us) raft_consensus.cc:1685] finished
0709 12:15:59.480523 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:59.480789 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14173,"raft.queue_time_us":118,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":45298,"spinlock_wait_cycles":12091136,"thread_start_us":306877,"threads_started":1,"wal-append.queue_time_us":546727}
I20260709 12:15:59.483580 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.484548 11222 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.485056 11328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3290) took 1006 ms. Trace:
I20260709 12:15:59.461136 19672 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.485430 11328 rpcz_store.cc:276] 0709 12:15:58.478724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.479846 (+  1122us) service_pool.cc:224] Handling call
0709 12:15:58.521502 (+ 41656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:59.050486 (+528984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:59.484820 (+434334us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b54001c52d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:59.484994 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214572416}
I20260709 12:15:59.486294 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:15:59.490468 19544 raft_consensus.cc:2804] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:15:59.489746 19490 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:15:59.489406 19706 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:15:59.491225 19544 raft_consensus.cc:697] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:59.487427 19614 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:15:59.557730 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 84664.3us
I20260709 12:15:59.488858 19707 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 12:15:59.490231 19231 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:15:59.556505 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:59.441581 19708 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.066s
I20260709 12:15:59.455717 19670 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:59.562443 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.429746 19654 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.560562 19778 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:15:59.431403 19659 raft_consensus.cc:740] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:15:59.574717 19725 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.491343 19501 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.572765 19699 raft_consensus.cc:3055] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:15:59.436051 19739 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:15:59.574286 19498 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:15:59.500273 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.562443 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:59.497956 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3596) took 1723 ms. Trace:
I20260709 12:15:59.495880 19588 consensus_queue.cc:1048] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:59.588668 11177 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:15:59.605194 19498 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:59.594219 19659 consensus_queue.cc:260] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.607157 19659 raft_consensus.cc:3060] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:15:59.607349 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3289) took 1129 ms. Trace:
W20260709 12:15:59.607815 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.607788 11327 rpcz_store.cc:276] 0709 12:15:58.478312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.478968 (+   656us) service_pool.cc:224] Handling call
0709 12:15:58.479260 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:59.387417 (+908157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:59.607117 (+219700us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b540042a9d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:59.607288 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067895168}
I20260709 12:15:59.445858 19769 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:15:59.588163 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 82089.2us
I20260709 12:15:59.603003 11322 rpcz_store.cc:276] 0709 12:15:57.774185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:57.774377 (+   192us) service_pool.cc:224] Handling call
0709 12:15:57.774706 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:58.824371 (+1049665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:59.497659 (+673288us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001c4dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:15:59.497894 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2409653120}
W20260709 12:15:59.561867 19668 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.002s
I20260709 12:15:59.586457 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.602255 11252 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:15:59.584146 19753 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.612007 11323 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:15:59.619349 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:15:59.615442 11901 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:59.605043 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:59.623823 11181 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
W20260709 12:15:59.592839 19787 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:15:59.635748 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:15:59.636642 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.637658 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.595683 19544 consensus_queue.cc:237] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.597731 19739 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.646492 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.647109 11935 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:15:59.648950 11926 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:15:59.645357 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.667845 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.619732 19769 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.597477 19699 raft_consensus.cc:740] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:59.667474 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.584718 19771 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.609934 19588 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:15:59.682534 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:15:59.679394 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 230645us
I20260709 12:15:59.638093 19778 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.638599 19498 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:15:59.690011 19498 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 30 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:59.689893 19778 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.690989 19498 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.692239 19778 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.626410 19668 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.684808 19713 consensus_queue.cc:1059] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:59.670576 19501 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:59.668764 19739 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.634234 19753 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:15:59.694710 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1341) took 1442 ms. Trace:
I20260709 12:15:59.648355 19641 raft_consensus.cc:2804] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:59.704609 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.706893 11181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.714418 11181 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:59.707871 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.695973 11251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1348) took 1429 ms. Trace:
I20260709 12:15:59.687150 19588 consensus_queue.cc:1243] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:15:59.674567 19769 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:59.719847 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:15:59.718466 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:59.683068 19774 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.720683 11253 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:59.722015 11176 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:15:59.610009 19711 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:59.716063 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:59.651094 11222 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:59.697350 19512 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:15:59.742939 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 91972us
I20260709 12:15:59.648924 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:15:59.719004 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:59.743376 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 107773us
I20260709 12:15:59.743857 11323 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:15:59.715837 11251 rpcz_store.cc:276] 0709 12:15:58.266158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.786603 (+520445us) service_pool.cc:224] Handling call
0709 12:15:58.801786 (+ 15183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:58.829034 (+ 27248us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:15:58.893310 (+ 64276us) log.cc:844] Serialized 14 byte log entry
0709 12:15:59.695398 (+802088us) log.cc:844] Serialized 14 byte log entry
0709 12:15:59.695614 (+   216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:59.695668 (+    54us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:15:59.695725 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:59.695799 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:59.695913 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":688708,"prepare.queue_time_us":1479419,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":153647,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":71107,"thread_start_us":168575,"threads_started":1,"wal-append.queue_time_us":349009}
I20260709 12:15:59.722190 11252 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:59.714814 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.693683 19778 raft_consensus.cc:2764] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:15:59.623075 19788 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.745644 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:15:59.714005 11250 rpcz_store.cc:276] 0709 12:15:58.252075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.620801 (+368726us) service_pool.cc:224] Handling call
0709 12:15:58.621271 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:58.656367 (+ 35096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:58.656399 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:58.656909 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:15:58.873042 (+216133us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:59.694172 (+821130us) log.cc:844] Serialized 14 byte log entry
0709 12:15:59.694393 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:59.694465 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:59.694520 (+    55us) raft_consensus.cc:1685] finished
0709 12:15:59.694532 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:59.694654 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":878447,"prepare.queue_time_us":821584,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":182374,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":15163,"thread_start_us":94845,"threads_started":1,"wal-append.queue_time_us":219644}
I20260709 12:15:59.739199 19708 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.003s
I20260709 12:15:59.717360 19635 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.768198 11296 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:15:59.754696 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.720237 19656 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:59.738041 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.773496 19702 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:15:59.782687 19790 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:15:59.783833 19776 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.042s
I20260709 12:15:59.770932 19710 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.786005 19710 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.786866 19771 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:59.788197 19710 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.771317 19788 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.791285 19788 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.759992 19670 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:15:59.792340 19670 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.793093 19668 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.793787 19761 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:15:59.794240 19761 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.794270 19708 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:15:59.794762 19761 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:15:59.794823 19708 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:15:59.796263 11932 raft_consensus.cc:1240] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:15:59.741760 19774 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.799597 19774 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.719853 19654 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:59.800460 19654 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.801391 19739 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.802762 19654 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:15:59.767688 19707 raft_consensus.cc:3055] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:15:59.803630 19739 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:15:59.803736 19707 raft_consensus.cc:740] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:59.795869 19708 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.804791 19708 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:15:59.804711 19707 consensus_queue.cc:260] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.775118 19490 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:59.806118 19707 raft_consensus.cc:3060] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:15:59.806365 19490 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:59.807695 19490 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:15:59.814109 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46016.6us
I20260709 12:15:59.815117 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:15:59.819108 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3369) took 1507 ms (client timeout 1617 ms). Trace:
W20260709 12:15:59.819602 11254 rpcz_store.cc:269] 0709 12:15:58.311564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.470973 (+159409us) service_pool.cc:224] Handling call
0709 12:15:59.818855 (+1347882us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b5400379e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:15:59.819016 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2260910336}
W20260709 12:15:59.861943 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 46927us
I20260709 12:15:59.753829 19512 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:15:59.865075 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:15:59.771377 19672 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.823498 11253 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
W20260709 12:15:59.850838 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:59.869866 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 193481us
I20260709 12:15:59.873472 11322 pending_rounds.cc:85] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 15
I20260709 12:15:59.873816 11322 pending_rounds.cc:107] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:15:59.874075 11322 raft_consensus.cc:2889] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:59.874265 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.874823 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:15:59.875439 11253 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.876497 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.877297 11257 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:15:59.878172 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:15:59.880462 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.879793 11901 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:15:59.771984 19711 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:59.738340 19751 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:59.822359 11147 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.194:37243 timed out after 1.617s (SENT) [suppressed 13 similar messages]
I20260709 12:15:59.883057 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:59.775883 19641 raft_consensus.cc:697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:15:59.771705 19501 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:59.886485 18816 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:59.920696 18816 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.921212 18816 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:59.824693 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:15:59.827816 19696 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.366s	user 0.011s	sys 0.022s
I20260709 12:15:59.907330 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:15:59.867759 19646 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:59.822005 19654 raft_consensus.cc:2804] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:15:59.866636 19776 consensus_queue.cc:1243] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 28.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:15:59.918313 11253 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:15:59.924990 11931 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:15:59.924160 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:15:59.888993 19771 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:15:59.884639 19793 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:59.932579 11296 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:15:59.927107 19696 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.465s	user 0.012s	sys 0.023s
I20260709 12:15:59.816649 19635 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.929880 19664 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:15:59.888800 19710 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.045s
I20260709 12:15:59.919989 19751 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.928754 19501 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:15:59.948071 19659 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:59.976662 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:15:59.970196 19711 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.972047 19501 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:15:59.977880 11253 raft_consensus.cc:2393] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:15:59.980306 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.022936 11254 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:15:59.985757 11249 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:15:59.993280 18816 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:00.010990 19795 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:16:00.018182 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.031627 11252 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:00.019214 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.032902 11256 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:16:00.020556 11257 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:00.021076 11250 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
W20260709 12:16:00.021688 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.960709 19793 raft_consensus.cc:2804] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:00.027819 11181 raft_consensus.cc:2468] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
I20260709 12:16:00.042081 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.042929 11250 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:16:00.045271 11250 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:15:59.939113 19512 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:15:59.960518 19670 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.049317 11251 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:16:00.049677 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.058878 11249 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:00.073858 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.074823 11250 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:15:59.974009 19696 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:59.975354 19659 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:15:59.978818 19710 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.076176 18816 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.077806 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:00.082021 11323 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:00.085865 11252 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:00.087515 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.088811 11256 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:16:00.072743 19793 raft_consensus.cc:697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:00.090512 19659 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:00.112519 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 290159us
W20260709 12:16:00.115666 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 183118us
I20260709 12:16:00.136457 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:00.144026 11323 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:16:00.097077 19710 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.186888 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.040519 19641 consensus_queue.cc:237] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.120648 19692 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.084874 19696 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.132575 19654 raft_consensus.cc:697] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:16:00.221709 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.236815 19793 consensus_queue.cc:237] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.237367 19751 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.050056 19699 consensus_queue.cc:260] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.139281 19664 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:00.252640 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.260476 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.185952 19501 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:00.268921 19776 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:00.271008 19696 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.269389 19776 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.185364 19668 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.077s
I20260709 12:16:00.260793 19702 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:00.238087 19692 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.284940 19810 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:00.292312 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.248612 19774 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:16:00.301347 19664 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.262168 19090 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:16:00.284242 19799 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:16:00.267802 19635 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.267043 19739 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:16:00.266521 19747 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:00.280143 19771 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:00.353384 19810 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.370222 11249 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:00.311512 19696 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:00.314591 19776 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:16:00.371639 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 111334us
I20260709 12:16:00.368315 19090 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.364868 19702 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:00.364126 19814 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:00.337524 19501 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.385596 19635 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.389261 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:00.392590 19090 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:00.393953 19672 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
W20260709 12:16:00.402347 19761 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.001s
I20260709 12:16:00.358745 19654 consensus_queue.cc:237] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.395876 19814 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.387698 19810 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:00.383488 19747 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.421604 11250 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:00.420854 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.399367 19006 raft_consensus.cc:3060] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:00.393565 19668 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:16:00.390936 19696 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.410041 19813 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.395732 19702 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:00.424176 19708 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.619s	user 0.002s	sys 0.010s
I20260709 12:16:00.417865 19672 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:00.422109 19812 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.428669 19753 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:00.409545 19635 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.429056 19710 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:00.442045 19769 consensus_meta.cc:287] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.690s	user 0.011s	sys 0.002s
W20260709 12:16:00.460882 19806 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:16:00.461906 19804 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:16:00.462726 11934 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: [23.16-23.16]
W20260709 12:16:00.472357 19707 consensus_meta.cc:287] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.666s	user 0.006s	sys 0.006s
I20260709 12:16:00.460659 19817 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:00.418336 19761 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.451434 19816 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.453960 19753 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.478334 19753 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:00.451949 19813 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.474943 19664 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.473304 19769 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.499398 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.500253 11250 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:00.457248 19812 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.519618 19812 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.522843 19635 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.523185 19769 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.523849 19501 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.498840 19702 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.526324 19813 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.511792 19696 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.528753 19812 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:00.508026 19646 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.004s
I20260709 12:16:00.511233 19747 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.479952 19816 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.532383 11927 raft_consensus.cc:2410] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:16:00.532931 19816 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.474644 19771 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.535799 19006 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:00.538178 19747 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.505410 19820 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.529367 19635 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.546703 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.551723 11322 raft_consensus.cc:2410] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:16:00.547014 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:00.547895 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 118492us
I20260709 12:16:00.548007 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.497977 19788 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:00.554409 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:00.552371 11327 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:16:00.538335 19824 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:16:00.555030 11927 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:16:00.530054 19812 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.500716 19707 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:00.565080 11250 raft_consensus.cc:1240] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:16:00.567307 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.568285 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.575201 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.575974 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.585779 11327 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:16:00.563668 19742 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:16:00.565841 19710 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.587157 19707 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.587834 19710 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.589839 19710 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.587242 19690 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
W20260709 12:16:00.599582 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:00.600324 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.538821 19747 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.609480 11327 raft_consensus.cc:1275] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:16:00.552695 19820 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.501262 19664 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:00.596251 19823 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:16:00.617837 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.621589 11327 raft_consensus.cc:2410] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 10.
I20260709 12:16:00.592962 19668 consensus_queue.cc:1243] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 27.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:16:00.589533 19707 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.625656 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.628250 11322 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:16:00.631606 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.606635 19771 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.635223 19090 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.637188 19829 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:16:00.621251 19833 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.639406 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.652174 11249 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:16:00.639968 19820 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.641108 11927 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: []
W20260709 12:16:00.642582 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.624953 19827 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.665841 11256 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:16:00.643862 19747 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.645802 11178 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:16:00.669242 11178 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:00.646410 11250 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:16:00.646600 11180 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:16:00.647037 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.702186 11328 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:16:00.703469 11328 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:00.704766 11328 raft_consensus.cc:1240] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:16:00.705099 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.648905 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.556319 19635 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.650296 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:16:00.653985 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:00.708412 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.709040 11253 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:00.709689 11249 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:16:00.710042 11249 pending_rounds.cc:85] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 14
I20260709 12:16:00.710251 11249 pending_rounds.cc:107] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:16:00.710458 11249 raft_consensus.cc:2889] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:00.654649 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.656292 11252 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:16:00.697468 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:00.657038 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.643543 11181 raft_consensus.cc:1240] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:16:00.712821 11182 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:00.659080 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.661280 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:00.711037 11257 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 19.
I20260709 12:16:00.713544 11255 raft_consensus.cc:3060] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:00.714138 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 52958.7us
I20260709 12:16:00.664458 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.629329 19710 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.716109 11322 raft_consensus.cc:1240] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:00.719087 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.670171 11178 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.672640 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.720446 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:00.673969 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:00.721213 11323 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:00.721366 11927 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:00.674783 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.723094 11325 raft_consensus.cc:1275] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:16:00.675799 19753 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.656015 19816 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.720548 18972 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:00.738438 19816 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.667057 19707 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.659181 19833 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.648097 19654 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.679222 19664 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.679328 19668 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:00.680984 19812 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.681948 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.631284 19814 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.685101 19761 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.687071 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.687402 19776 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.688784 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.774816 11331 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:16:00.693706 19659 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.005s
I20260709 12:16:00.694166 19810 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.650162 19829 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:00.695924 19692 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.697472 19090 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.697512 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.698829 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.715310 11251 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:00.723685 11147 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.721520 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.788820 11322 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 20.
I20260709 12:16:00.790206 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.726929 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.716123 11328 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:16:00.740753 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.742442 11255 raft_consensus.cc:2468] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:00.650911 19771 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.758050 11323 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:16:00.751230 18816 consensus_queue.cc:1059] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:16:00.749352 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.730044 19690 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:16:00.771970 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 155192us
I20260709 12:16:00.720129 19742 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:16:00.778090 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.735276 19640 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:16:00.782167 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:00.794374 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 67531.8us
I20260709 12:16:00.711963 19827 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:00.796713 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 47498.7us
I20260709 12:16:00.800196 11331 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 22.
I20260709 12:16:00.746191 19724 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:16:00.814392 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:00.820484 11324 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:16:00.771723 19751 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.810472 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.778539 11330 raft_consensus.cc:2393] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:00.739557 19656 consensus_queue.cc:1059] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.038s
I20260709 12:16:00.815478 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.820786 11331 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:16:00.803404 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.808748 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.845810 11255 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:00.812171 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.846845 11296 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.740002 19710 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.761318 19654 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.826354 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:00.827154 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.829701 19543 raft_consensus.cc:3055] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:16:00.773696 19828 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
W20260709 12:16:00.830673 11176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3557) took 1933 ms (client timeout 1721 ms). Trace:
W20260709 12:16:00.831271 19834 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:16:00.831496 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:00.876398 11323 raft_consensus.cc:2393] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 22: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:00.772033 19668 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.833382 19708 raft_consensus.cc:3055] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:16:00.835388 19771 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.838292 19776 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.820062 19742 consensus_queue.cc:1243] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 28.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:16:00.881654 19832 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:16:00.892073 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:00.893369 11323 raft_consensus.cc:2393] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:00.869024 19664 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:00.928017 19664 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.928656 19664 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:00.819953 19696 consensus_queue.cc:1048] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:16:00.930742 19696 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:00.933995 19710 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:00.937508 11322 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:16:00.938409 19654 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.937947 19829 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.840399 11329 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:16:00.943702 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:00.944900 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.945785 11254 raft_consensus.cc:2368] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:16:00.870545 19741 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 12:16:00.948150 11254 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:16:00.950946 19635 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:00.951588 19838 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:16:00.840719 19810 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:00.843088 19816 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.827186 18642 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:00.853494 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:00.945406 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.745891 19727 consensus_queue.cc:1048] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:16:00.755090 19707 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:00.799129 19690 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:00.866819 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.962170 18642 consensus_queue.cc:1243] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:16:00.962719 18642 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:00.806567 19761 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:00.852098 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:00.865958 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 79671.9us
W20260709 12:16:00.966828 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 114856us
I20260709 12:16:00.813391 19812 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:16:00.869143 11176 rpcz_store.cc:269] 0709 12:15:58.897569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.897696 (+   127us) service_pool.cc:224] Handling call
0709 12:16:00.830458 (+1932762us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b540016f8d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:00.830603 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1828481408}
I20260709 12:16:00.833477 19827 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.878841 11328 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:00.782392 19722 consensus_queue.cc:1059] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 12:16:00.797093 19774 consensus_queue.cc:1048] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 12:16:00.844673 19090 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:00.895367 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:00.868774 19543 raft_consensus.cc:740] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:00.898626 19778 raft_consensus.cc:3055] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:16:00.900761 19802 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.761s	user 0.002s	sys 0.000s
I20260709 12:16:00.905217 19744 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:00.907019 19801 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.982862 19690 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:00.914392 19833 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:00.917245 19835 log.cc:927] Time spent T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.158s	user 0.000s	sys 0.003s
I20260709 12:16:00.785866 19659 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:00.983181 19742 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:00.860035 19751 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:00.957978 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.994565 19812 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:00.958086 11256 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:16:01.030921 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.032478 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:01.033075 11926 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:01.036664 11251 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:16:00.964236 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:00.962527 11901 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:00.880924 19708 raft_consensus.cc:740] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:01.031529 19692 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.041128 19692 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.041831 19725 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:01.041965 19690 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.043031 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.043466 19692 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:00.983392 19761 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:01.043860 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.044341 19761 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:01.032395 19812 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:00.838464 11929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1533) took 1014 ms. Trace:
I20260709 12:16:01.007222 19778 raft_consensus.cc:740] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:01.048204 19778 consensus_queue.cc:260] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.001371 19814 raft_consensus.cc:2804] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:01.051105 11326 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:01.009697 19744 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:01.001663 19829 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:01.004047 19668 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:01.056996 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:01.017809 19801 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.012840 19753 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.018770 19543 consensus_queue.cc:260] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.021783 19780 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:00.995303 19810 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.062494 19006 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:16:01.064078 11249 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:16:01.065663 11179 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:16:01.027107 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 200830us
W20260709 12:16:01.028704 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 220096us
I20260709 12:16:00.963090 18642 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:16:01.040158 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 75972.1us
I20260709 12:16:00.968168 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:01.041461 19708 consensus_queue.cc:260] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.042348 19725 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:16:00.983240 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 259614us
I20260709 12:16:01.052459 11176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.079222 11176 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 30.
I20260709 12:16:01.079303 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:01.049986 11929 rpcz_store.cc:276] 0709 12:15:59.823722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.823863 (+   141us) service_pool.cc:224] Handling call
0709 12:16:00.838150 (+1014287us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54000429d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:00.838373 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588571904}
I20260709 12:16:00.979364 11325 raft_consensus.cc:1240] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:01.011051 19090 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:01.049583 19778 raft_consensus.cc:3060] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:01.070024 19006 consensus_queue.cc:1243] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 26.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:16:01.071651 11296 leader_election.cc:400] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:16:01.085654 11296 leader_election.cc:403] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:01.069733 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:01.068387 18642 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.087076 18642 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:01.057283 19744 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:01.053867 19833 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:01.089403 19743 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.089676 11223 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:01.090371 19848 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:01.092423 11175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.117307 11175 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:01.105908 19512 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:01.064599 19814 raft_consensus.cc:697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:16:01.115511 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44045.4us
I20260709 12:16:01.118340 11147 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.127902 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:16:01.129244 11173 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:16:01.130623 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.130754 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:01.131711 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.132474 11173 raft_consensus.cc:2410] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:16:01.132393 11926 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
W20260709 12:16:01.122047 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 159591us
W20260709 12:16:01.114184 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 146133us
I20260709 12:16:01.082307 19725 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:01.135018 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:01.135022 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.135847 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:01.124576 19722 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:16:01.123685 19825 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:16:01.140565 19753 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.142588 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.114923 19543 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:16:01.147290 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.119897 19833 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:01.150182 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:01.196866 11250 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:01.150422 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:16:01.174472 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.198222 11173 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:01.120054 19848 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:01.199496 11296 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:01.149396 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:01.191056 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.191942 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.069536 19810 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.119766 19816 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 22. Current term is 23.
I20260709 12:16:01.204097 11250 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:01.210218 11174 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 27: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:01.210222 11250 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:01.215301 11254 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:16:01.215581 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69328us
I20260709 12:16:01.143965 19828 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:01.218335 11256 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:16:01.142758 19708 raft_consensus.cc:3060] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:16:01.222587 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 86810.6us
I20260709 12:16:01.171146 19743 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:16:01.230295 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 96354.6us
I20260709 12:16:01.231212 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.233174 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.233505 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.233856 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.194955 19498 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:01.194697 19793 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:01.195129 19772 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:01.219616 19842 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:16:01.244585 11929 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:01.240255 19780 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:01.246574 11927 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:16:01.208963 19833 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:01.247049 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:01.247718 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 105230us
I20260709 12:16:01.208213 19771 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:01.190775 19820 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:01.142248 19822 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 12:16:01.268874 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:01.209745 19848 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:01.298424 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.282970 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.279654 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:01.300819 11327 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:01.285912 11326 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:16:01.231900 19755 raft_consensus.cc:2804] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:01.251899 19707 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:01.243429 19498 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:01.272428 19793 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:01.271945 19743 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:01.279192 19739 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:01.209892 19753 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.286430 19742 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.287300 19780 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:01.338265 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 138846us
I20260709 12:16:01.344156 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.324990 19832 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.341884 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.248580 19641 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:16:01.294515 19656 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:01.351388 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.351840 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:01.357117 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 105487us
I20260709 12:16:01.357792 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:01.274160 19741 consensus_queue.cc:1059] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:16:01.289872 19853 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:01.368793 11252 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:01.370312 11900 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:01.331079 19498 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:01.376144 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:16:01.340260 19544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.386s	user 0.000s	sys 0.024s
I20260709 12:16:01.339025 19793 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:01.332419 19755 raft_consensus.cc:697] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:01.297777 19692 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.065s
I20260709 12:16:01.234594 19090 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:01.381799 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.350494 19659 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:01.382447 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.349681 19739 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.370783 19816 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:01.351265 19780 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:01.377544 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:01.285388 19814 consensus_queue.cc:237] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:01.431563 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 54139us
I20260709 12:16:01.433763 19654 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.382557 19490 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:01.435089 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.435647 19810 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:16:01.436507 19810 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:01.436895 19810 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.437939 19743 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.438225 19755 consensus_queue.cc:237] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.317752 19820 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.373271 19614 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:16:01.379356 19853 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:01.380168 19544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.432s	user 0.000s	sys 0.025s
I20260709 12:16:01.418362 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.361621 19771 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:01.436725 11174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:16:01.403293 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:01.402323 19741 raft_consensus.cc:3055] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:16:01.437382 19810 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:01.451400 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.461128 11252 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:16:01.467077 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.434327 11177 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:16:01.503294 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 52000.9us
I20260709 12:16:01.503540 11177 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:01.378500 19690 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.400981 19812 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:01.435701 19846 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:16:01.499607 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:01.504643 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.441653 19747 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:01.426559 19816 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:01.362337 19742 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.468194 19710 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.511446 11248 raft_consensus.cc:1240] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:16:01.509114 11177 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.380512 19670 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.126s
I20260709 12:16:01.530496 19863 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:01.369725 19832 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.532285 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:16:01.534471 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.535694 11323 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: []
I20260709 12:16:01.543392 11933 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:16:01.568342 11177 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:01.568536 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.496673 19490 raft_consensus.cc:697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:01.497653 19741 raft_consensus.cc:740] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:01.497324 19771 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:01.434391 19654 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.590996 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.591110 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.472183 19828 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:01.474789 19659 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 12:16:01.571046 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.571666 11253 raft_consensus.cc:2393] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:01.571973 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:01.543656 11926 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.16-25.16]
W20260709 12:16:01.566391 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 280247us
I20260709 12:16:01.598358 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.538559 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:01.600322 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 61828.2us
I20260709 12:16:01.578353 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.508430 19822 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:01.538596 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.603262 11933 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:01.601966 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:01.565379 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.511533 19753 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:01.567330 19690 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:01.590674 19664 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.004s
W20260709 12:16:01.610312 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:01.621805 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 49923.2us
I20260709 12:16:01.612469 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.606945 11182 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
W20260709 12:16:01.594765 19846 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:16:01.618497 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:01.628163 11933 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:01.591496 19771 raft_consensus.cc:3060] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:01.626693 11932 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:16:01.591545 19544 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.630609 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:01.625109 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.527768 19761 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.531963 19747 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.634176 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:01.634684 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.637827 11257 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:01.638760 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:01.643432 18642 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:01.643934 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:01.570111 19710 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:01.646853 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:01.649752 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.651929 11147 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:01.656996 11933 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
W20260709 12:16:01.631949 19046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.024s	sys 0.011s
I20260709 12:16:01.659459 18642 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:01.660074 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.631696 19753 raft_consensus.cc:3060] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:01.628234 19827 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:01.628605 19755 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:01.675045 18642 consensus_queue.cc:1059] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:16:01.684433 18642 consensus_queue.cc:1243] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 23.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:16:01.639590 19501 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:01.687281 11254 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:16:01.677904 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:01.664340 19046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.024s	sys 0.011s
I20260709 12:16:01.640379 19822 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:01.653693 19863 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:01.662981 19588 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:01.677539 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.654313 19743 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:01.695559 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3627) took 1023 ms. Trace:
I20260709 12:16:01.677131 19847 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:01.700335 11179 consensus_meta.cc:287] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
W20260709 12:16:01.701906 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 50075.3us
I20260709 12:16:01.776419 11179 raft_consensus.cc:2468] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
W20260709 12:16:01.716441 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 56524.8us
I20260709 12:16:01.687312 19862 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.691048 11177 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:16:01.746898 19854 log.cc:927] Time spent T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:16:01.784914 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:01.744895 19751 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.604701 19871 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:16:01.632714 19690 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.786032 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:01.786453 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:01.688375 11253 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:16:01.731804 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 97188.6us
I20260709 12:16:01.787078 11326 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:01.651332 19231 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:16:01.642915 19741 consensus_queue.cc:260] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.717442 19828 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:01.730022 19861 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:16:01.635421 19816 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.733875 11324 rpcz_store.cc:276] 0709 12:16:00.671841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.674437 (+  2596us) service_pool.cc:224] Handling call
0709 12:16:00.675035 (+   598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:00.820962 (+145927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.695193 (+874231us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001c4dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:01.695462 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334330368}
I20260709 12:16:01.779044 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.637480 19490 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.780463 19640 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.785276 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.808480 11177 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:16:01.788311 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.639910 19544 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.778643 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:01.787307 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:01.731091 19827 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.746021 19727 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:01.630003 19664 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.801290 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3396) took 1076 ms. Trace:
I20260709 12:16:01.802630 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3313) took 1033 ms. Trace:
I20260709 12:16:01.804883 11179 raft_consensus.cc:1275] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:16:01.738394 19864 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:01.735374 19870 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:01.827283 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:01.782699 19739 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.741137 19654 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:01.824869 19640 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.751618 19822 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:01.739534 19543 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.544s	user 0.003s	sys 0.006s
I20260709 12:16:01.833966 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.834196 19788 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:01.903031 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.751099 19753 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:16:01.836808 19837 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 1.054s	user 0.001s	sys 0.004s
I20260709 12:16:01.837730 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:01.954957 11253 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:01.956595 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:01.746112 19851 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
W20260709 12:16:01.739872 19841 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.903s	user 0.004s	sys 0.001s
I20260709 12:16:01.741370 19498 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:01.883479 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56264.2us
I20260709 12:16:01.885275 11323 rpcz_store.cc:276] 0709 12:16:00.769342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.953052 (+183710us) service_pool.cc:224] Handling call
0709 12:16:00.966063 (+ 13011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:01.568047 (+601984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.802281 (+234234us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54001c43d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:01.802514 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309394304}
I20260709 12:16:01.745841 19588 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:01.829440 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50881.7us
I20260709 12:16:01.961009 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:01.840164 11328 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
W20260709 12:16:01.748466 19708 consensus_meta.cc:287] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.002s
I20260709 12:16:01.809290 19741 raft_consensus.cc:3060] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:01.973989 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.887375 19853 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:01.884034 11257 rpcz_store.cc:276] 0709 12:16:00.725058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.845333 (+120275us) service_pool.cc:224] Handling call
0709 12:16:00.845678 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:01.638236 (+792558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.801022 (+162786us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400067a54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:01.801222 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821237632}
I20260709 12:16:01.798182 19812 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:01.750103 19747 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:01.912407 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.981194 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.981317 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:01.910719 11326 raft_consensus.cc:2468] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
W20260709 12:16:01.894172 19722 consensus_meta.cc:287] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.002s
I20260709 12:16:01.984400 11173 raft_consensus.cc:2410] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:16:01.918998 11325 raft_consensus.cc:1275] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:16:01.952610 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 65812.3us
I20260709 12:16:01.840030 19727 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:01.883289 19751 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:01.750765 19793 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:01.967227 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.898036 19672 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:01.982090 11257 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:01.983145 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:01.792294 19862 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:01.983951 11327 raft_consensus.cc:1240] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:16:01.949954 11329 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:01.991983 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:01.994736 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.995667 11177 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:16:01.982932 19870 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:01.880590 19827 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:01.953358 19817 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:02.007045 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46127.4us
I20260709 12:16:02.007352 11178 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:16:02.110330 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.958858 19543 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.127831 11257 raft_consensus.cc:2468] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:02.004467 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:02.069615 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:02.072644 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.132779 11932 raft_consensus.cc:1275] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:16:02.133220 11932 pending_rounds.cc:85] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 16
I20260709 12:16:02.133440 11932 pending_rounds.cc:107] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:16:02.133697 11932 raft_consensus.cc:2889] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:16:02.017129 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:02.159171 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.062662 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:01.983139 19788 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.113055 19878 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:02.230026 19878 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.230531 19878 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:02.231532 19640 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.229519 19543 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.233469 19793 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:02.113504 19876 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.324s	user 0.000s	sys 0.000s
I20260709 12:16:02.104990 19672 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:16:02.236016 11253 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:16:02.115306 19870 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.118721 19862 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.989760 19751 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.107198 19727 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.120915 11178 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:16:02.312582 11178 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1456) took 1672 ms. Trace:
I20260709 12:16:02.124473 19664 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:02.313117 11178 rpcz_store.cc:276] 0709 12:16:00.639745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.645393 (+  5648us) service_pool.cc:224] Handling call
0709 12:16:00.645685 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.006814 (+1361129us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b540029cfa0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6661a1d 00007f3ec66481a4 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66eeb49 00007f3ec66ef416 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419
0709 12:16:02.312193 (+305379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.312499 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369500416}
W20260709 12:16:02.113508 19865 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.396s	user 0.000s	sys 0.000s
I20260709 12:16:02.113677 19877 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:02.125880 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 122217us
I20260709 12:16:01.961318 19690 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.326845 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:01.992427 19832 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:02.329734 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.217489 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.219370 11296 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:16:02.114679 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:02.333766 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 114485us
I20260709 12:16:02.257539 11177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.265292 19543 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.334911 19543 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:02.335624 19543 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.336160 19755 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.336732 19755 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.338609 19543 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.338827 19755 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:02.228816 19512 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.105s	user 0.010s	sys 0.000s
I20260709 12:16:02.269928 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.340090 19863 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.340289 11257 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:16:02.340607 19863 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.341776 19812 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.341774 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.342587 19812 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.343631 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:02.344978 19812 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.345099 11251 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:01.959146 19822 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.348285 19822 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.273206 19588 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.952644 19231 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.349682 19231 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.352770 19231 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:01.993553 19725 consensus_meta.cc:287] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.800s	user 0.008s	sys 0.003s
I20260709 12:16:02.353801 19725 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.355437 11934 raft_consensus.cc:2393] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:02.356873 19725 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:01.999735 19722 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:01.962387 19698 consensus_queue.cc:1059] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.166s
I20260709 12:16:02.274389 19710 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:01.750420 19847 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.275054 19672 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.359248 19672 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.360195 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.362412 19672 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.008836 19747 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.267s
I20260709 12:16:02.285295 19753 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.365712 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
W20260709 12:16:02.307737 11223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.040s	sys 0.053s
W20260709 12:16:02.308555 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44728.9us
I20260709 12:16:02.288136 19614 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:02.371039 11223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.040s	sys 0.053s
W20260709 12:16:02.228739 19744 consensus_meta.cc:287] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.106s	user 0.007s	sys 0.001s
I20260709 12:16:02.372107 19744 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.374169 19744 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.339686 19755 raft_consensus.cc:2804] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:02.125986 19871 consensus_queue.cc:1059] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.197s
I20260709 12:16:02.375005 19753 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:02.376323 19872 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.399s	user 0.000s	sys 0.000s
W20260709 12:16:02.377286 19881 log.cc:927] Time spent T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:16:02.322126 19656 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.311528 19751 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:02.322425 19635 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:02.325060 19829 consensus_meta.cc:287] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.266s	user 0.007s	sys 0.005s
I20260709 12:16:02.386662 19847 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:01.972272 19654 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.351s
W20260709 12:16:02.322369 19760 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 28 current term: 28 is higher.
I20260709 12:16:02.137907 19817 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.404040 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.414858 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3631) took 1280 ms. Trace:
I20260709 12:16:02.310886 19788 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.383852 19727 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:02.323941 19090 consensus_queue.cc:1059] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 26.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:16:02.390810 19739 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.378326 19776 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:16:02.423898 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.424644 11179 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:16:02.382583 19614 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.394528 19755 raft_consensus.cc:697] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:02.401340 19672 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.425539 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2847) took 2715 ms. Trace:
I20260709 12:16:02.437932 11248 rpcz_store.cc:276] 0709 12:15:59.709830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.710000 (+   170us) service_pool.cc:224] Handling call
0709 12:15:59.710459 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.535769 (+1825310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.425289 (+889520us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54001cf2d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:02.425483 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4141185792}
I20260709 12:16:02.415449 11331 rpcz_store.cc:276] 0709 12:16:01.134746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.146612 (+ 11866us) service_pool.cc:224] Handling call
0709 12:16:01.146990 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.148902 (+  1912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.148936 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:01.189438 (+ 40502us) log.cc:844] Serialized 26 byte log entry
0709 12:16:01.383834 (+194396us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:01.383873 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.383939 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.414626 (+1030687us) raft_consensus.cc:1685] finished
0709 12:16:02.414648 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.414792 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":237,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":24210,"thread_start_us":145169,"threads_started":1,"wal-append.queue_time_us":433328}
I20260709 12:16:02.398257 19656 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.365000 19877 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.444413 11179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:16:02.444622 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:02.450415 11248 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 21: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:02.442417 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:02.445792 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:02.441746 19778 consensus_meta.cc:287] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.327s	user 0.011s	sys 0.003s
I20260709 12:16:02.452693 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:02.456928 11331 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:16:02.457325 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:16:02.427299 19771 consensus_meta.cc:287] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.000s
I20260709 12:16:02.458271 11248 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:02.458514 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:02.462977 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:02.464130 11323 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:02.474308 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.475062 11932 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:16:02.475898 19699 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:02.476384 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.477007 11932 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:16:02.478248 11926 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:16:02.479777 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.481035 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.484061 11931 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:16:02.484588 11248 raft_consensus.cc:2468] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:16:02.405351 19635 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.485884 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:02.463088 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:02.492008 19740 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:02.492497 19882 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:16:02.492939 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.493400 11223 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:02.493026 11250 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.420117 19816 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:16:02.485071 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:16:02.501302 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:02.503175 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.446655 19664 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.505283 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.511152 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.512290 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.513208 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.515517 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.516284 19793 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.516888 19793 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.519906 19793 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.521319 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:02.523649 11248 raft_consensus.cc:1240] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:02.524170 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3403) took 1295 ms. Trace:
I20260709 12:16:02.524384 11248 rpcz_store.cc:276] 0709 12:16:01.228719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.522825 (+1294106us) service_pool.cc:224] Handling call
0709 12:16:02.523525 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.523989 (+   464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.524127 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:02.525353 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.526003 11248 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:02.436555 19739 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:02.503108 19848 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 1.173s	user 0.012s	sys 0.000s
I20260709 12:16:02.528544 11323 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:16:02.530604 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.499127 19847 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:02.502696 19708 raft_consensus.cc:3055] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:02.500109 19853 raft_consensus.cc:3055] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:16:02.504619 19780 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 1.075s	user 0.005s	sys 0.008s
I20260709 12:16:02.443951 19672 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.502558 19747 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:02.538339 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98"
I20260709 12:16:02.502350 19827 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:02.503588 19544 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.503662 19710 raft_consensus.cc:2804] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:02.540354 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:02.537961 19833 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.222s	user 0.012s	sys 0.000s
W20260709 12:16:02.552578 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51406us
I20260709 12:16:02.565258 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:02.550832 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50093.7us
I20260709 12:16:02.497804 19771 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.566710 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:02.550266 19883 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:16:02.567006 11223 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:02.542701 11296 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:02.540983 19847 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:02.526561 19832 consensus_queue.cc:1048] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:16:02.548763 19793 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:02.571650 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:02.551362 19853 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:02.541646 19708 raft_consensus.cc:740] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:02.576896 19739 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.563585 19231 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:02.529605 19664 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.558408 19753 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.604658 19828 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.610818 19828 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.578336 19847 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:02.616866 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.564392 19747 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.597594 19793 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.540416 19848 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.597571 19798 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.569703 19833 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.508415 19740 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.639319 19747 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.640056 11248 raft_consensus.cc:2410] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:16:02.640069 19614 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:02.642230 19656 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.642747 19710 raft_consensus.cc:697] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:16:02.643129 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78004.4us
I20260709 12:16:02.666350 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:02.643821 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 76874.9us
I20260709 12:16:02.644448 19231 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:02.644524 19882 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:16:02.645660 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 74079.8us
W20260709 12:16:02.647471 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.670854 11331 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:16:02.615798 19828 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.609383 19863 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.672029 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:02.673269 11932 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:16:02.674118 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.634647 19793 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:02.561774 19755 consensus_queue.cc:237] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.675210 11324 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
I20260709 12:16:02.557899 19780 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.676551 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.682979 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2869) took 1536 ms. Trace:
I20260709 12:16:02.644722 19635 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.682997 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:02.688359 11178 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:02.700887 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.669497 19788 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.695132 19894 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:16:02.696283 19648 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:16:02.703631 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1370) took 1497 ms. Trace:
I20260709 12:16:02.700795 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.677531 19544 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:02.685331 11249 rpcz_store.cc:276] 0709 12:16:01.146734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.146924 (+   190us) service_pool.cc:224] Handling call
0709 12:16:01.147232 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.186254 (+ 39022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.186295 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:01.186944 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:16:02.499867 (+1312923us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b1800362920. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:16:02.598591 (+ 98724us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:02.598632 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.599329 (+   697us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.682698 (+ 83369us) raft_consensus.cc:1685] finished
0709 12:16:02.682718 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.682920 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38583,"raft.queue_time_us":43238,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":30217,"spinlock_wait_cycles":21179776,"thread_start_us":98546,"threads_started":1,"wal-append.queue_time_us":1431097}
I20260709 12:16:02.669369 19882 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:02.709142 19833 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:02.697971 19817 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.710467 19664 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.696805 19892 raft_consensus.cc:2804] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:02.686437 19863 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:02.712608 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:02.712881 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.714793 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1369) took 1509 ms. Trace:
I20260709 12:16:02.714973 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:02.716576 19880 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 12:16:02.720077 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:02.732739 11255 rpcz_store.cc:276] 0709 12:16:01.205886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.754363 (+548477us) service_pool.cc:224] Handling call
0709 12:16:01.754895 (+   532us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:16:01.755499 (+   604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.755546 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:16:01.756551 (+  1005us) log.cc:844] Serialized 78 byte log entry
0709 12:16:01.882948 (+126397us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:16:02.309807 (+426859us) log.cc:844] Serialized 14 byte log entry
0709 12:16:02.310243 (+   436us) log.cc:844] Serialized 14 byte log entry
0709 12:16:02.321886 (+ 11643us) log.cc:844] Serialized 14 byte log entry
0709 12:16:02.505178 (+183292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.505289 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.714573 (+209284us) raft_consensus.cc:1685] finished
0709 12:16:02.714593 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.714734 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11934,"prepare.queue_time_us":1026179,"prepare.run_cpu_time_us":144,"prepare.run_wall_time_us":5468,"raft.queue_time_us":65288,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":374,"thread_start_us":658985,"threads_started":3,"wal-append.queue_time_us":370877}
I20260709 12:16:02.746675 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:02.747457 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:02.748096 19889 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:16:02.726702 19882 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:02.706512 19891 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:16:02.720960 11929 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:02.724408 11935 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:02.708722 19877 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.725417 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.667837 19853 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.668237 19708 consensus_queue.cc:260] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.721019 19543 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:02.720386 11257 rpcz_store.cc:276] 0709 12:16:01.206235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.361280 (+1155045us) service_pool.cc:224] Handling call
0709 12:16:02.361687 (+   407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:16:02.362372 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.362405 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:16:02.363187 (+   782us) log.cc:844] Serialized 52 byte log entry
0709 12:16:02.507056 (+143869us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:16:02.571794 (+ 64738us) log.cc:844] Serialized 14 byte log entry
0709 12:16:02.649891 (+ 78097us) log.cc:844] Serialized 14 byte log entry
0709 12:16:02.650109 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.650204 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.703341 (+ 53137us) raft_consensus.cc:1685] finished
0709 12:16:02.703371 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.703555 (+   184us) inbound_call.cc:177] Queueing success response
0709 12:16:02.709246 (+  5691us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b1800413da0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3eca5c058f 00007f3ec35ff571 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd
Metrics: {"mutex_wait_us":94514,"prepare.queue_time_us":37107,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":82870,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":33105,"spinlock_wait_cycles":33797888,"thread_start_us":179349,"threads_started":2,"wal-append.queue_time_us":240238}
I20260709 12:16:02.748122 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:02.755167 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:02.723964 19894 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:02.759460 11249 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:16:02.762439 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:02.762769 11327 raft_consensus.cc:1275] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:16:02.732270 19863 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:02.891258 11327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1690) took 1109 ms. Trace:
I20260709 12:16:02.891793 11327 rpcz_store.cc:276] 0709 12:16:01.782013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.492644 (+710631us) service_pool.cc:224] Handling call
0709 12:16:02.762372 (+269728us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b5800170498. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c7b71 00007f3ec35c7a22 00007f3ecc4e08d9 00007f3ecb63aee0 00007f3ecb7040ab 00007f3ecb704131 00007f3ecb6c5fc0 00007f3ecb6b0929 00007f3ecb6b03d2 00007f3ec4c27bd4
0709 12:16:02.762532 (+   160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.838152 (+ 75620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.891177 (+ 53025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6800128}
I20260709 12:16:02.713258 19901 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.745033 19892 raft_consensus.cc:697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:02.925833 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3404) took 1696 ms. Trace:
I20260709 12:16:02.931061 11249 rpcz_store.cc:276] 0709 12:16:01.229045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.924653 (+1695608us) service_pool.cc:224] Handling call
0709 12:16:02.925098 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.925339 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.925377 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:02.925438 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:16:02.925501 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.925584 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.925755 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:02.756528 19544 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:02.929785 19849 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:02.754381 19817 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.933111 11249 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:16:02.936714 11249 pending_rounds.cc:85] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 15
I20260709 12:16:02.936983 11249 pending_rounds.cc:107] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:16:02.937248 11249 raft_consensus.cc:2889] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:02.937999 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1372) took 1604 ms. Trace:
I20260709 12:16:02.938372 11249 rpcz_store.cc:276] 0709 12:16:01.334158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.932387 (+1598229us) service_pool.cc:224] Handling call
0709 12:16:02.932690 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.937714 (+  5024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.937939 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:02.927778 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3405) took 1698 ms. Trace:
I20260709 12:16:02.940943 11255 rpcz_store.cc:276] 0709 12:16:01.229417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.926805 (+1697388us) service_pool.cc:224] Handling call
0709 12:16:02.927203 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.927437 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.927475 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:02.927535 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:02.927576 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.927633 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.927725 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:02.943179 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:02.764896 19771 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:02.664963 19798 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.910513 19894 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:02.891021 19231 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.923626 19863 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:02.944074 19708 raft_consensus.cc:3060] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:02.966055 19740 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:02.971185 19890 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:16:02.971686 19860 negotiation.cc:336] Failed RPC negotiation. Trace:
0709 12:16:01.095335 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:44782 (local address 127.9.198.194:37243)
0709 12:16:01.748778 (+653443us) server_negotiation.cc:207] Beginning negotiation
0709 12:16:01.748818 (+    40us) server_negotiation.cc:400] Waiting for connection header
0709 12:16:01.748881 (+    63us) server_negotiation.cc:408] Connection header received
0709 12:16:01.749106 (+   225us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0709 12:16:01.749137 (+    31us) server_negotiation.cc:462] Received NEGOTIATE request from client
0709 12:16:01.749464 (+   327us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0709 12:16:01.792537 (+ 43073us) server_negotiation.cc:227] Negotiated authn=SASL
0709 12:16:01.955854 (+163317us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0709 12:16:01.957891 (+  2037us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0709 12:16:02.414550 (+456659us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0709 12:16:02.460575 (+ 46025us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0709 12:16:02.614217 (+153642us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
0709 12:16:02.664625 (+ 50408us) server_negotiation.cc:882] Received SASL_INITIATE request from client
0709 12:16:02.668289 (+  3664us) server_negotiation.cc:893] Client requested to use mechanism: PLAIN
0709 12:16:02.695024 (+ 26735us) server_negotiation.cc:923] calling sasl_server_start()
0709 12:16:02.695256 (+   232us) server_negotiation.cc:1092] Received PLAIN auth, user=slave
0709 12:16:02.695360 (+   104us) server_negotiation.cc:378] Sending SASL_SUCCESS NegotiatePB response
0709 12:16:02.695594 (+   234us) server_negotiation.cc:1036] Waiting for connection context
0709 12:16:02.710988 (+ 15394us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44782: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":414105,"thread_start_us":267708,"threads_started":1}
W20260709 12:16:02.972126 19800 negotiation.cc:336] Failed RPC negotiation. Trace:
0709 12:15:59.686603 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.9.198.194:37243 (local address 127.0.0.1:44782)
0709 12:16:00.463983 (+777380us) negotiation.cc:107] Waiting for socket to connect
0709 12:16:00.464076 (+    93us) client_negotiation.cc:175] Beginning negotiation
0709 12:16:00.464600 (+   524us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0709 12:16:01.788190 (+1323590us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0709 12:16:01.797783 (+  9593us) client_negotiation.cc:404] Received NEGOTIATE response from server
0709 12:16:01.883314 (+ 85531us) client_negotiation.cc:190] Negotiated authn=SASL
0709 12:16:01.954865 (+ 71551us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0709 12:16:01.954973 (+   108us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0709 12:16:02.127127 (+172154us) client_negotiation.cc:281] Received TLS_HANDSHAKE NegotiatePB response
0709 12:16:02.134927 (+  7800us) client_negotiation.cc:543] Received TLS_HANDSHAKE response from server
0709 12:16:02.399107 (+264180us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0709 12:16:02.400931 (+  1824us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0709 12:16:02.447597 (+ 46666us) client_negotiation.cc:564] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0709 12:16:02.602464 (+154867us) client_negotiation.cc:639] Initiating SASL PLAIN handshake
0709 12:16:02.602504 (+    40us) client_negotiation.cc:672] calling sasl_client_start()
0709 12:16:02.602616 (+   112us) client_negotiation.cc:830] callback for SASL_CB_AUTHNAME
0709 12:16:02.602838 (+   222us) client_negotiation.cc:262] Sending SASL_INITIATE NegotiatePB request
0709 12:16:02.710350 (+107512us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.9.198.194:37243: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":562269,"thread_start_us":238137,"threads_started":1}
I20260709 12:16:02.934075 19543 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:02.974143 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 1170) took 1008 ms. Trace:
I20260709 12:16:02.974788 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1374) took 1560 ms. Trace:
I20260709 12:16:02.979831 11327 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:16:02.966527 11331 pending_rounds.cc:85] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 14
I20260709 12:16:02.981303 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:02.959872 19853 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:02.727752 19710 consensus_queue.cc:237] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.987169 11931 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:02.993136 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1375) took 1578 ms. Trace:
I20260709 12:16:02.994594 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:02.992429 11331 pending_rounds.cc:107] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 28.15
I20260709 12:16:02.996337 11331 raft_consensus.cc:2889] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:02.991066 11257 rpcz_store.cc:276] 0709 12:16:01.414238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.945122 (+1530884us) service_pool.cc:224] Handling call
0709 12:16:02.974238 (+ 29116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.974428 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.974459 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:02.974511 (+    52us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:16:02.974544 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.974614 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.974731 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:02.990974 11933 rpcz_store.cc:276] 0709 12:16:01.965317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.965521 (+   204us) service_pool.cc:224] Handling call
0709 12:16:01.965911 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.998147 (+ 32236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.998206 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:02.129936 (+131730us) log.cc:844] Serialized 26 byte log entry
0709 12:16:02.503499 (+373563us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:16:02.503543 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.504657 (+  1114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.973888 (+469231us) raft_consensus.cc:1685] finished
0709 12:16:02.973913 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.974066 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17053,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":17433,"thread_start_us":180207,"threads_started":1,"wal-append.queue_time_us":323886}
W20260709 12:16:03.002282 19727 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.002s
I20260709 12:16:02.982105 19231 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:03.003659 19878 consensus_meta.cc:287] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.001s
I20260709 12:16:02.948156 19817 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.010440 11257 raft_consensus.cc:1240] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:16:02.983561 19848 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.019747 11327 raft_consensus.cc:1275] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:16:02.994019 11249 rpcz_store.cc:276] 0709 12:16:01.414665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.943222 (+1528557us) service_pool.cc:224] Handling call
0709 12:16:02.990338 (+ 47116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.990659 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.990693 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:02.990736 (+    43us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:16:02.990771 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.992840 (+  2069us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.992990 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:16:02.988644 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 226259us
I20260709 12:16:02.935274 19901 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:02.972801 19892 consensus_queue.cc:237] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:02.996299 19890 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.004182 19878 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.005090 19771 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:03.034139 19901 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.020259 19780 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.019479 19664 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:03.031963 19231 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.031229 19890 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:03.021543 19877 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:16:03.058255 11249 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:16:03.043602 19848 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:16:03.045715 19544 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:16:03.086848 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3409) took 1307 ms. Trace:
W20260709 12:16:03.092607 19810 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.580s	user 0.013s	sys 0.001s
I20260709 12:16:03.093204 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:03.093045 19543 consensus_queue.cc:1048] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:03.095994 11249 rpcz_store.cc:276] 0709 12:16:01.779585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.030940 (+1251355us) service_pool.cc:224] Handling call
0709 12:16:03.056343 (+ 25403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:03.058849 (+  2506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.086784 (+ 27935us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.089268 19544 consensus_queue.cc:1243] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer 786851d4227e4dd89b75ccdea0b1f03b log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:16:03.113368 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 91967.2us
I20260709 12:16:03.113832 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:03.115257 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:03.117997 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:03.118760 11182 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:16:03.119676 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:03.120229 11182 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:16:03.095944 19880 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:16:03.123961 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:03.125873 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:03.126858 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:03.128855 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:03.099427 19810 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.136312 19810 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:03.118144 19848 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.129711 19880 consensus_queue.cc:1243] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer bc92f9d2c6c5477182fa33eba4368ec1 log is divergent from this leader: its last log entry 26.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:16:03.141146 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.129876 19878 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:03.127981 19903 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:16:03.122331 19882 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.143679 11182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
W20260709 12:16:03.133126 19904 log.cc:927] Time spent T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:16:03.145394 11249 raft_consensus.cc:2393] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:03.161162 11182 raft_consensus.cc:2410] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:16:03.161921 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:16:03.146077 11254 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52330 (request call id 1373) took 1763 ms (client timeout 2169 ms). Trace:
W20260709 12:16:03.162914 11254 rpcz_store.cc:269] 0709 12:16:01.382976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.464109 (+ 81133us) service_pool.cc:224] Handling call
0709 12:16:03.145909 (+1681800us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400109d54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:03.146021 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410006400}
I20260709 12:16:03.148698 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:03.149943 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:03.151160 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45582 (request call id 2905) took 1681 ms. Trace:
I20260709 12:16:03.133014 19877 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:03.158420 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.159090 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:03.164927 11173 rpcz_store.cc:276] 0709 12:16:01.469281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.651464 (+182183us) service_pool.cc:224] Handling call
0709 12:16:03.146899 (+1495435us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400363654. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:03.151024 (+  4125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581352704}
I20260709 12:16:03.167689 11328 raft_consensus.cc:2393] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 28: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:03.170188 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.178715 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:03.179613 11248 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:03.181742 19863 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.181223 11254 raft_consensus.cc:1240] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: []
I20260709 12:16:03.171101 11935 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:03.184163 11182 raft_consensus.cc:1275] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:16:03.186336 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3412) took 1397 ms. Trace:
I20260709 12:16:03.187045 11248 rpcz_store.cc:276] 0709 12:16:01.789074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.183905 (+1394831us) service_pool.cc:224] Handling call
0709 12:16:03.185684 (+  1779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.185907 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.185945 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:03.186002 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:03.186049 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.186133 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.186267 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.190717 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3410) took 1410 ms. Trace:
I20260709 12:16:03.190999 11173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "016411114c6d42039bd04025378dfc98" is_pre_election: true
I20260709 12:16:03.191220 11254 rpcz_store.cc:276] 0709 12:16:01.780056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.180686 (+1400630us) service_pool.cc:224] Handling call
0709 12:16:03.181033 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.190437 (+  9404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.190658 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:16:03.192338 11330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.009s	sys 0.005s
W20260709 12:16:03.192883 11330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.009s	sys 0.005s
I20260709 12:16:03.193113 11248 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:16:03.194172 11173 raft_consensus.cc:2410] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 22 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 18.
I20260709 12:16:03.194859 11176 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:16:03.195449 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2874) took 1369 ms. Trace:
I20260709 12:16:03.167333 19877 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:03.159953 19817 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.150825 19848 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.184900 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2902) took 1754 ms. Trace:
I20260709 12:16:03.277976 11322 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3325) took 1514 ms. Trace:
I20260709 12:16:03.246119 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3328) took 1473 ms. Trace:
I20260709 12:16:03.233244 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1467) took 1439 ms. Trace:
I20260709 12:16:03.251808 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:03.298043 11324 rpcz_store.cc:276] 0709 12:16:01.772698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.800358 (+ 27660us) service_pool.cc:224] Handling call
0709 12:16:01.800663 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.682271 (+881608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.245892 (+563621us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54001c3ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:03.246057 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010801024}
I20260709 12:16:03.248610 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:03.268330 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:03.300076 11331 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:16:03.209851 18642 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:16:03.301169 18642 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:03.236385 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3312) took 2467 ms. Trace:
I20260709 12:16:03.301775 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:03.209965 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.210292 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.230582 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1376) took 1724 ms. Trace:
I20260709 12:16:03.230937 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2904) took 1764 ms. Trace:
I20260709 12:16:03.303685 11175 rpcz_store.cc:276] 0709 12:16:01.793557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.232234 (+1438677us) service_pool.cc:224] Handling call
0709 12:16:03.232615 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.232830 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.232870 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:03.232925 (+    55us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:16:03.232969 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.233039 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.233174 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.219736 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3413) took 1324 ms. Trace:
I20260709 12:16:03.304284 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.216137 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1368) took 2010 ms. Trace:
I20260709 12:16:03.238986 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:03.301999 11329 rpcz_store.cc:276] 0709 12:16:00.769059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.943113 (+174054us) service_pool.cc:224] Handling call
0709 12:16:00.943447 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:01.991292 (+1047845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.236116 (+1244824us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54000e6354. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:03.236315 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236531584}
I20260709 12:16:03.280929 11250 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:03.306270 11252 rpcz_store.cc:276] 0709 12:16:01.205511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.533111 (+327600us) service_pool.cc:224] Handling call
0709 12:16:01.572726 (+ 39615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.671296 (+ 98570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.671339 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:01.671936 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 12:16:01.774915 (+102979us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:16:01.774959 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.215569 (+1440610us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b540010c054. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:03.215656 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:03.215728 (+    72us) raft_consensus.cc:1685] finished
0709 12:16:03.215744 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.216054 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226007936,"thread_start_us":102601,"threads_started":1,"wal-append.queue_time_us":326776}
I20260709 12:16:03.225188 11927 raft_consensus.cc:1240] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 20. Current term is 21. Ops: []
W20260709 12:16:03.220360 11934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 1166) took 2189 ms (client timeout 1878 ms). Trace:
I20260709 12:16:03.304859 11176 rpcz_store.cc:276] 0709 12:16:01.431094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.190891 (+1759797us) service_pool.cc:224] Handling call
0709 12:16:03.191223 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.195255 (+  4032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.195416 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.301923 18642 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.310149 11176 pending_rounds.cc:85] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Aborting all ops after (but not including) 14
I20260709 12:16:03.305392 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3321) took 2467 ms. Trace:
I20260709 12:16:03.209703 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.297130 11182 rpcz_store.cc:276] 0709 12:16:01.430381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.170851 (+1740470us) service_pool.cc:224] Handling call
0709 12:16:03.171206 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.184650 (+ 13444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.184850 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:16:03.311944 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.279441 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3414) took 1317 ms. Trace:
I20260709 12:16:03.312278 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:03.310564 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.312827 11254 rpcz_store.cc:276] 0709 12:16:01.961661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.278357 (+1316696us) service_pool.cc:224] Handling call
0709 12:16:03.278782 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.279017 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.279054 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:03.279109 (+    55us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:16:03.279167 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.279236 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.279367 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.320907 18642 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:03.310741 11176 pending_rounds.cc:107] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:16:03.229785 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:03.305399 11253 rpcz_store.cc:276] 0709 12:16:01.895092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.957591 (+ 62499us) service_pool.cc:224] Handling call
0709 12:16:03.219562 (+1261971us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54000884d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:03.219671 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242484608}
W20260709 12:16:03.309221 11934 rpcz_store.cc:269] 0709 12:16:01.030717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.031042 (+   325us) service_pool.cc:224] Handling call
0709 12:16:03.220147 (+2189105us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540069e454. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:03.220272 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3016396160}
I20260709 12:16:03.324463 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:03.322249 11176 raft_consensus.cc:2889] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:03.303896 11255 rpcz_store.cc:276] 0709 12:16:01.506438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.949184 (+1442746us) service_pool.cc:224] Handling call
0709 12:16:02.949542 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:02.987912 (+ 38370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.987944 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:02.988395 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:16:03.095469 (+107074us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:16:03.117001 (+ 21532us) log.cc:844] Serialized 14 byte log entry
0709 12:16:03.127810 (+ 10809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.127891 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:03.230347 (+102456us) raft_consensus.cc:1685] finished
0709 12:16:03.230370 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.230518 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19339,"prepare.queue_time_us":21491,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":51238,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":20041,"thread_start_us":102371,"threads_started":2,"wal-append.queue_time_us":122780}
W20260709 12:16:03.326260 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 104074us
I20260709 12:16:03.235242 11174 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:03.327174 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:03.326088 11256 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:16:03.328509 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2877) took 1501 ms. Trace:
I20260709 12:16:03.327206 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1468) took 1529 ms. Trace:
I20260709 12:16:03.297382 11322 rpcz_store.cc:276] 0709 12:16:01.763192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.765773 (+  2581us) service_pool.cc:224] Handling call
0709 12:16:01.766260 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:02.333472 (+567212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.333505 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:02.368314 (+ 34809us) log.cc:844] Serialized 26 byte log entry
0709 12:16:02.539713 (+171399us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:02.563935 (+ 24222us) log.cc:844] Serialized 14 byte log entry
0709 12:16:02.571216 (+  7281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.214262 (+643046us) spinlock_profiling.cc:243] Waited 7.5 ms on lock 0x7b18000e9da0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:16:03.275494 (+ 61232us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b18000e9da0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff522 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:16:03.277482 (+  1988us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540026cad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:03.277548 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:03.277610 (+    62us) raft_consensus.cc:1685] finished
0709 12:16:03.277705 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.277895 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14577,"prepare.queue_time_us":26305,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":20774,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":25068,"spinlock_wait_cycles":1312868992,"thread_start_us":128961,"threads_started":1,"wal-append.queue_time_us":178206}
I20260709 12:16:03.311233 11330 rpcz_store.cc:276] 0709 12:16:00.837900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.995026 (+157126us) service_pool.cc:224] Handling call
0709 12:16:00.995341 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:00.995677 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:16:01.742665 (+746988us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b48002601d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecb615aa1 00007f3ecb60cb37 00007f3ecb60c9ca 00007f3ecb70426f 00007f3ecb703f37 00007f3ecb71855c 00007f3ecb718508 00007f3ecb718488 00007f3ecb718443 00007f3ecb7176ec 00007f3eca641a96
0709 12:16:02.495399 (+752734us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b1800185e60. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec62944bc 00007f3ec35e248b 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fc455 00007f3eca0e43c7 00007f3ec66f69a7 00007f3ec67060bf
0709 12:16:02.495553 (+   154us) log.cc:844] Serialized 14 byte log entry
0709 12:16:03.197137 (+701584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:03.197203 (+    66us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:16:03.197244 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.305211 (+107967us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.305338 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39862,"prepare.queue_time_us":291143,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":29149,"raft.run_cpu_time_us":3656,"raft.run_wall_time_us":750410,"spinlock_wait_cycles":11794176,"thread_start_us":972821,"threads_started":3,"wal-append.queue_time_us":188699}
W20260709 12:16:03.331713 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.329056 11174 rpcz_store.cc:276] 0709 12:16:01.797713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.234597 (+1436884us) service_pool.cc:224] Handling call
0709 12:16:03.235053 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.326950 (+ 91897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.327148 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.195798 11248 rpcz_store.cc:276] 0709 12:16:01.826264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.189209 (+1362945us) service_pool.cc:224] Handling call
0709 12:16:03.189606 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.195192 (+  5586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.195397 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.337334 11175 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: [23.16-23.16]
W20260709 12:16:03.217794 19741 consensus_meta.cc:287] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.226s	user 0.009s	sys 0.004s
W20260709 12:16:03.203397 19751 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.799s	user 0.010s	sys 0.003s
W20260709 12:16:03.268406 19793 consensus_meta.cc:287] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
W20260709 12:16:03.306246 19847 consensus_meta.cc:287] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.591s	user 0.007s	sys 0.001s
I20260709 12:16:03.323148 19878 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:03.341176 11175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3570) took 1533 ms. Trace:
I20260709 12:16:03.199285 19882 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.231024 19670 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.197381 19812 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:03.269881 19801 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.213299 19543 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.341707 19648 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.322701 19877 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:454: Failure
Failed
Bad status: Timed out: Client connection negotiation failed: client connection to 127.9.198.194:37243: received 0 of 4 requested bytes
I20260709 12:16:03.342458 19848 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.254427 19863 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.343444 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3579) took 1125 ms. Trace:
I20260709 12:16:03.523566 11182 rpcz_store.cc:276] 0709 12:16:02.218132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.327601 (+1109469us) service_pool.cc:224] Handling call
0709 12:16:03.327894 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.343071 (+ 15177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.343101 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:03.343153 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:03.343201 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.343283 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.343391 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14823,"raft.queue_time_us":115604,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103}
I20260709 12:16:03.346657 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:03.328859 11256 rpcz_store.cc:276] 0709 12:16:01.827107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.260204 (+1433097us) service_pool.cc:224] Handling call
0709 12:16:03.260640 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:03.328260 (+ 67620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.328454 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.357780 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:03.528077 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 170425us
I20260709 12:16:03.369269 11926 raft_consensus.cc:2468] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:16:03.340372 11248 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:03.530409 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3418) took 1076 ms. Trace:
I20260709 12:16:03.530881 11248 rpcz_store.cc:276] 0709 12:16:02.453544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.339859 (+886315us) service_pool.cc:224] Handling call
0709 12:16:03.340178 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.530090 (+189912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.530339 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.532204 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:03.532966 11248 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:03.342243 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:03.387516 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:03.196978 19810 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.397790 11179 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:16:03.265789 19761 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.411067 11174 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:03.548987 11174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3581) took 1283 ms. Trace:
I20260709 12:16:03.549527 11174 rpcz_store.cc:276] 0709 12:16:02.265154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.334156 (+1069002us) service_pool.cc:224] Handling call
0709 12:16:03.410875 (+ 76719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.548609 (+137734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.548901 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.414791 11175 rpcz_store.cc:276] 0709 12:16:01.807694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.325875 (+1518181us) service_pool.cc:224] Handling call
0709 12:16:03.331344 (+  5469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:03.340900 (+  9556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.341116 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.331809 19822 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:03.426332 19878 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.439035 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
I20260709 12:16:03.441991 19890 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.445766 19670 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.342298 19760 consensus_queue.cc:1059] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 27.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 12:16:03.592840 19760 raft_consensus.cc:3055] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:16:03.593302 19890 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.593329 19760 raft_consensus.cc:740] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:03.452407 11178 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:16:03.594122 19878 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.425083 11250 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1366) took 2222 ms. Trace:
I20260709 12:16:03.595530 11250 rpcz_store.cc:276] 0709 12:16:01.203202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.203402 (+   200us) service_pool.cc:224] Handling call
0709 12:16:01.203879 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.492625 (+1288746us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b4c00015700. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec49b8a20 00007f3ec49c9a29 00007f3ec498f644 00007f3ec49b6f2d 00007f3ec49b6b93 00007f3ec66e718b 00007f3ec66e7cb5 00007f3ec66e2f30 00007f3ec66e47de 00007f3ec66eeb49 00007f3ec66ef416
0709 12:16:03.280621 (+787996us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b58006c6ed0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66480aa 00007f3ec66ca447 00007f3ec66e2fe7 00007f3ec66e47de 00007f3ec66eeb49 00007f3ec66ef416 00007f3ec66ebd8f 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4
0709 12:16:03.413148 (+132527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.413179 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:03.424605 (+ 11426us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:16:03.424686 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.424795 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.424991 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37996,"raft.queue_time_us":41728,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":1949984896}
I20260709 12:16:03.411027 19751 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.535701 11296 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:03.537333 19708 consensus_meta.cc:287] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.005s
I20260709 12:16:03.538018 19827 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:03.425318 19740 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:03.604866 11326 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:16:03.422418 19847 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.415086 19793 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.410830 19755 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.129s
I20260709 12:16:03.539930 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:03.627308 11252 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:03.545466 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:03.545998 19798 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.547200 11182 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:16:03.415762 19742 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.597610 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:03.602802 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 67191.8us
I20260709 12:16:03.484925 19812 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.607962 11249 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:16:03.608147 19864 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.660s	user 0.018s	sys 0.001s
I20260709 12:16:03.528293 19543 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.563418 19810 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.629570 11248 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:03.657482 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:03.632071 11182 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3585) took 1365 ms. Trace:
I20260709 12:16:03.649436 11250 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:03.484529 19648 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.605783 19906 raft_consensus.cc:2804] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:03.661533 19648 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.658876 11182 rpcz_store.cc:276] 0709 12:16:02.266530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.546448 (+1279918us) service_pool.cc:224] Handling call
0709 12:16:03.546839 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.631665 (+ 84826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.631995 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:03.594636 19760 consensus_queue.cc:260] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.661562 19543 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.665035 19498 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:03.665935 11254 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:03.663352 19760 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:03.671144 11250 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:16:03.671697 11250 pending_rounds.cc:85] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 15
I20260709 12:16:03.671978 11250 pending_rounds.cc:107] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:16:03.672282 11250 raft_consensus.cc:2889] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:03.674360 11250 raft_consensus.cc:1275] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:16:03.675498 11250 raft_consensus.cc:1240] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 21. Current term is 22. Ops: []
I20260709 12:16:03.677198 11250 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:16:03.687592 11252 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:03.666265 19864 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:03.625476 19740 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:03.829756 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:03.839195 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:03.825918 11328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.025s	sys 0.052s
W20260709 12:16:03.855392 11328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.025s	sys 0.052s
I20260709 12:16:03.840431 19670 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:03.841560 19847 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:03.836570 11177 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.008s	sys 0.015s
W20260709 12:16:03.882718 11177 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.010s	sys 0.018s
I20260709 12:16:03.842741 19793 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:03.842825 19827 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:03.846412 11327 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:16:03.846535 19864 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:03.837450 19654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.011s	sys 0.011s
I20260709 12:16:03.851145 19751 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:03.871680 11222 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.005s	sys 0.022s
W20260709 12:16:03.899261 11222 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.005s	sys 0.022s
W20260709 12:16:03.899711 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 643415us
W20260709 12:16:03.809948 19159 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:16:03.857111 19812 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:03.902302 19231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.017s	sys 0.027s
W20260709 12:16:03.905572 19231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.017s	sys 0.027s
I20260709 12:16:03.860670 19814 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:03.895650 19847 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:03.906563 19847 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:03.935529 19812 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.936514 19812 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:03.941967 11147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.004s	sys 0.007s
W20260709 12:16:03.942543 11147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.004s	sys 0.007s
W20260709 12:16:03.893784 19654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.012s	sys 0.011s
I20260709 12:16:03.889519 19670 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:16:03.949545 11148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.014s	sys 0.018s
W20260709 12:16:03.949193 11146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.013s	sys 0.019s
W20260709 12:16:03.953050 11146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.013s	sys 0.019s
W20260709 12:16:03.942965 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 627018us
W20260709 12:16:03.954708 11935 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.000s	sys 0.012s
W20260709 12:16:03.960753 11935 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.000s	sys 0.012s
W20260709 12:16:03.949242 11176 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.025s
W20260709 12:16:03.962615 11176 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.000s	sys 0.026s
W20260709 12:16:03.952442 11148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.014s	sys 0.018s
W20260709 12:16:03.964138 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 636989us
W20260709 12:16:03.954226 19747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.000s	sys 0.032s
I20260709 12:16:03.893289 19827 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:03.970351 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:04.019167 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3406) took 2396 ms. Trace:
I20260709 12:16:04.007387 19812 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.022185 11257 rpcz_store.cc:276] 0709 12:16:01.623004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.998753 (+1375749us) service_pool.cc:224] Handling call
0709 12:16:03.006587 (+  7834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:03.011206 (+  4619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.018856 (+1007650us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b54002a6cd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:04.019101 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8193920}
I20260709 12:16:03.967598 19845 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:03.976994 19894 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.991s	user 0.011s	sys 0.007s
W20260709 12:16:04.024287 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 54074.6us
I20260709 12:16:03.972977 19742 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.976007 19878 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:03.973397 19708 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:04.008002 19772 consensus_queue.cc:1059] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:16:03.895191 19864 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:04.028447 11930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.012s	sys 0.003s
W20260709 12:16:04.091277 11930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.013s	sys 0.003s
I20260709 12:16:04.008502 19810 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:04.019114 19747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.000s	sys 0.033s
I20260709 12:16:04.011938 19648 consensus_queue.cc:1059] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:16:03.975775 19853 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:03.958436 19670 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.098888 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:04.026474 19820 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.116295 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:04.101358 11253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.019s	sys 0.007s
W20260709 12:16:04.121990 11253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.020s	sys 0.007s
I20260709 12:16:04.085525 19812 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:04.128388 11252 raft_consensus.cc:1240] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:04.136834 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.138307 11250 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:16:04.140137 11253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3416) took 2027 ms. Trace:
I20260709 12:16:04.031365 19894 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.085171 19742 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.090360 19878 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.097931 19708 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:04.173084 11179 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3584) took 1906 ms (client timeout 1869 ms). Trace:
W20260709 12:16:04.173723 11179 rpcz_store.cc:269] 0709 12:16:02.266233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.425322 (+159089us) service_pool.cc:224] Handling call
0709 12:16:04.172775 (+1747453us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b54001f3bd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:04.172981 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2178807168}
I20260709 12:16:04.177893 11248 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:04.126650 19853 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.162433 19847 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:04.184019 19870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.449s	user 0.014s	sys 0.044s
W20260709 12:16:04.187817 19870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.455s	user 0.014s	sys 0.045s
I20260709 12:16:04.162693 19923 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:16:04.131876 11900 connection.cc:256] RPC call timeout handler was delayed by -1.009025013000155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
I20260709 12:16:04.148867 19812 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:04.205894 11173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.038s
W20260709 12:16:04.197732 11933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.046s	sys 0.075s
W20260709 12:16:04.207901 11933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.046s	sys 0.076s
I20260709 12:16:04.207948 11181 raft_consensus.cc:1240] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:16:04.194422 19923 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:04.207415 11173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.011s	sys 0.038s
I20260709 12:16:04.201768 19659 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:04.210780 11173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1465) took 2428 ms. Trace:
W20260709 12:16:04.209539 19877 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.002s
I20260709 12:16:04.226012 11176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 3576) took 2317 ms. Trace:
W20260709 12:16:04.226459 19921 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:16:04.230094 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2883) took 1778 ms. Trace:
I20260709 12:16:04.230684 11255 rpcz_store.cc:276] 0709 12:16:02.452126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.337988 (+885862us) service_pool.cc:224] Handling call
0709 12:16:03.338342 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:16:03.338731 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.338763 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:16:03.360734 (+ 21971us) log.cc:844] Serialized 52 byte log entry
0709 12:16:04.026071 (+665337us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:16:04.026126 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.026564 (+   438us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:04.229803 (+203239us) raft_consensus.cc:1685] finished
0709 12:16:04.229834 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.230010 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":463886,"threads_started":1,"wal-append.queue_time_us":750681}
W20260709 12:16:04.232980 19760 consensus_meta.cc:287] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
I20260709 12:16:04.226526 11176 rpcz_store.cc:276] 0709 12:16:01.908888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.309619 (+1400731us) service_pool.cc:224] Handling call
0709 12:16:03.309966 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:03.328767 (+ 18801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.328814 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:03.329260 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 12:16:03.963190 (+633930us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:03.963253 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.963373 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:04.225729 (+262356us) raft_consensus.cc:1685] finished
0709 12:16:04.225757 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.225936 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":552429,"threads_started":1,"wal-append.queue_time_us":812053}
I20260709 12:16:04.218675 19659 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.234525 19878 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.236066 19853 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.236510 19659 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:04.237036 19812 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.237936 19708 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.238591 19708 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.239085 19708 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.241135 19708 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:04.246153 19894 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:04.209239 19923 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:04.215812 11173 rpcz_store.cc:276] 0709 12:16:01.782445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.196081 (+1413636us) service_pool.cc:224] Handling call
0709 12:16:03.196448 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.196645 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:16:03.583615 (+386970us) log.cc:844] Serialized 14 byte log entry
0709 12:16:04.210338 (+626723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:04.210413 (+    75us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:16:04.210476 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.210560 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.210701 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":424,"prepare.queue_time_us":761459,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":37690,"raft.queue_time_us":31964,"thread_start_us":1012376,"threads_started":2}
I20260709 12:16:04.216735 19827 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:04.283735 19814 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:04.284493 19814 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:04.219903 19877 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.277477 19894 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.274292 19670 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:04.299909 19906 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.057s	sys 0.070s
W20260709 12:16:04.305178 19906 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.058s	sys 0.071s
I20260709 12:16:04.305470 19906 raft_consensus.cc:697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:16:04.317876 11297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.004s	sys 0.024s
W20260709 12:16:04.319839 11297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.004s	sys 0.024s
I20260709 12:16:04.302284 19894 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:04.323231 11296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.012s	sys 0.016s
I20260709 12:16:04.325253 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:04.324961 11296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.012s	sys 0.016s
I20260709 12:16:04.316689 19742 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:04.326849 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:04.331521 19827 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:04.324762 19159 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.333573 19744 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:04.331521 19708 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.333324 19670 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:04.335007 19659 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:16:04.358975 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 351474us
I20260709 12:16:04.361609 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.362556 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:04.365803 11255 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:04.366582 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:04.367947 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:04.372073 19928 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:04.372251 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:04.372575 19928 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.373009 19928 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:04.357730 19755 consensus_queue.cc:1059] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:16:04.373991 19755 raft_consensus.cc:3055] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:16:04.374198 19922 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:04.374917 19922 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.365772 19708 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.375464 19922 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:04.367972 19936 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:16:04.372394 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.377203 19708 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.374436 19755 raft_consensus.cc:740] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:04.377797 19870 consensus_queue.cc:1059] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:04.377889 11252 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:04.382658 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:04.386166 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 302651us
I20260709 12:16:04.354820 19744 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.386205 11255 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:16:04.322427 19906 consensus_queue.cc:237] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.387861 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:04.388176 11251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3401) took 3153 ms (client timeout 1957 ms). Trace:
W20260709 12:16:04.391505 11251 rpcz_store.cc:269] 0709 12:16:01.227996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.229900 (+  1904us) service_pool.cc:224] Handling call
0709 12:16:04.381333 (+3151433us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:04.381449 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2412592384}
I20260709 12:16:04.365372 19740 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.352074 19159 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.371847 19742 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:16:04.395756 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.352715 19827 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.374009 11147 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:04.401930 19793 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:04.367615 19820 consensus_queue.cc:1048] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:16:04.390820 19744 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:04.404927 11927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1353) took 2863 ms. Trace:
I20260709 12:16:04.410513 11256 raft_consensus.cc:1275] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:16:04.412111 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.424268 11251 raft_consensus.cc:2393] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:04.367846 19670 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.413264 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.423319 11927 rpcz_store.cc:276] 0709 12:16:01.541726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.541855 (+   129us) service_pool.cc:224] Handling call
0709 12:16:03.224936 (+1683081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.308919 (+ 83983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.404576 (+1095657us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540069e450. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:04.404805 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3757563520}
W20260709 12:16:04.415650 19931 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.108s	user 0.001s	sys 0.001s
I20260709 12:16:04.426102 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:04.396631 19659 consensus_queue.cc:1048] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:16:04.388760 19708 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.020s
W20260709 12:16:04.421233 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.408682 19870 raft_consensus.cc:3055] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:16:04.399469 19742 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.410373 19877 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:04.445001 19934 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:16:04.449244 11177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45486 (request call id 1466) took 2661 ms. Trace:
I20260709 12:16:04.417472 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:04.425033 11068 maintenance_manager.cc:419] P bbfb3415e272477dac9ba599cc40a89d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033371
I20260709 12:16:04.476504 19847 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:04.482887 19847 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.483273 19847 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:04.437026 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.437698 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:04.451370 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:04.644126 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 229389us
I20260709 12:16:04.449769 11177 rpcz_store.cc:276] 0709 12:16:01.788412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.211232 (+1422820us) service_pool.cc:224] Handling call
0709 12:16:03.211589 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.211845 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:16:03.883447 (+671602us) log.cc:844] Serialized 14 byte log entry
0709 12:16:04.218942 (+335495us) log.cc:844] Serialized 14 byte log entry
0709 12:16:04.448085 (+229143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:04.448160 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:16:04.448226 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.448971 (+   745us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.449167 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":764480,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":92,"raft.queue_time_us":22900,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":1234822,"threads_started":3,"wal-append.queue_time_us":515027}
I20260709 12:16:04.645232 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:04.481595 19829 raft_consensus.cc:3055] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:16:04.448392 19774 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:04.432049 19937 consensus_queue.cc:1048] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.147s
I20260709 12:16:04.448137 19742 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:16:04.557921 19892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.357s	user 0.028s	sys 0.061s
I20260709 12:16:04.420722 19755 consensus_queue.cc:260] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:04.488076 19933 log.cc:927] Time spent T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:16:04.436857 19870 raft_consensus.cc:740] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:16:04.660591 11147 connection.cc:256] RPC call timeout handler was delayed by -1.3801317810000455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:16:04.663303 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:04.664041 19940 log.cc:927] Time spent T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:16:04.664215 11927 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:16:04.487109 19894 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.580878 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:04.554683 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 347265us
I20260709 12:16:04.580092 19772 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.574354 19944 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.609673 19943 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:16:04.634382 19938 log.cc:927] Time spent T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 12:16:04.640817 19512 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:04.679744 19512 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.682039 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:04.682320 19950 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:16:04.683504 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:04.683693 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:04.685448 19512 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.686362 19512 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:04.686862 19512 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:04.417470 19810 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:16:04.653144 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:04.689280 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:04.690003 11328 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:04.689980 19944 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.691097 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:04.692279 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:04.646382 19829 raft_consensus.cc:740] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:04.677927 19772 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.693281 19829 consensus_queue.cc:260] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.694373 19829 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:04.696012 19864 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:04.697976 19672 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.699939 11328 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:04.666234 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:04.721217 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:04.666536 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:04.654848 19892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.454s	user 0.029s	sys 0.072s
I20260709 12:16:04.673159 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:04.675938 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:04.678025 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:04.686945 11933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52012 (request call id 1183) took 1546 ms. Trace:
I20260709 12:16:04.436611 19793 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:04.701296 19853 raft_consensus.cc:3055] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:04.657855 19774 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.704989 19670 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:04.674682 19755 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:04.706502 19827 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:04.669561 19870 consensus_queue.cc:260] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:04.678714 19951 consensus_queue.cc:1059] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:16:04.715574 19772 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:04.581534 19887 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:04.717025 19943 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:04.700197 19878 raft_consensus.cc:3055] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:04.673260 11251 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
W20260709 12:16:04.692404 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.664582 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:04.977795 11323 raft_consensus.cc:3060] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:04.978184 19956 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:04.979435 19956 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:04.980453 11178 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 3583) took 2714 ms (client timeout 2177 ms). Trace:
W20260709 12:16:04.981014 11178 rpcz_store.cc:269] 0709 12:16:02.265870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.668985 (+403115us) service_pool.cc:224] Handling call
0709 12:16:04.979887 (+2310902us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54002b8254. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:04.980126 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717479808}
W20260709 12:16:04.982446 11249 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2896) took 1831 ms (client timeout 1736 ms). Trace:
W20260709 12:16:04.983034 11249 rpcz_store.cc:269] 0709 12:16:03.150683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.161444 (+ 10761us) service_pool.cc:224] Handling call
0709 12:16:04.981743 (+1820299us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b5400108e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:04.982144 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017090432}
I20260709 12:16:04.990705 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2897) took 1689 ms. Trace:
I20260709 12:16:04.991279 11248 rpcz_store.cc:276] 0709 12:16:03.301328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.668925 (+367597us) service_pool.cc:224] Handling call
0709 12:16:03.669316 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:04.181018 (+511702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.990003 (+808985us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400379e54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:04.990435 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148393344}
I20260709 12:16:04.992964 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.708748 11326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3338) took 1458 ms. Trace:
W20260709 12:16:04.716842 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:04.711165 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.574492 19708 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:05.075930 11326 rpcz_store.cc:276] 0709 12:16:03.250192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.255137 (+  4945us) service_pool.cc:224] Handling call
0709 12:16:03.255445 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.605318 (+349873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.708473 (+1103155us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54003228d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:04.708683 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802716416}
I20260709 12:16:05.077922 19708 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.078533 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.079432 11326 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:16:05.080713 19708 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:04.717051 19741 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:05.082146 19741 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:05.082731 19741 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:05.084607 19774 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.085994 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:04.846680 11147 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.536s (SENT) [suppressed 4 similar messages]
I20260709 12:16:05.087074 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:05.087960 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:05.089699 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:05.090575 11927 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:16:04.722729 11329 raft_consensus.cc:2393] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 21: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
W20260709 12:16:04.848196 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 461039us
I20260709 12:16:05.093618 11148 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:05.095559 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:04.781852 19923 consensus_meta.cc:287] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.002s
I20260709 12:16:05.097517 19923 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.937747 19878 raft_consensus.cc:740] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:16:05.102989 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 651568us
I20260709 12:16:05.102886 19878 consensus_queue.cc:260] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.104875 19878 raft_consensus.cc:3060] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:04.942214 19870 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:04.934036 11933 rpcz_store.cc:276] 0709 12:16:03.140147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.140296 (+   149us) service_pool.cc:224] Handling call
0709 12:16:03.140641 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:03.141029 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:16:03.582051 (+441022us) log.cc:844] Serialized 14 byte log entry
0709 12:16:04.437015 (+854964us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:04.437494 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:16:04.437654 (+   160us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:16:04.437691 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.437755 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:04.686680 (+248925us) raft_consensus.cc:1685] finished
0709 12:16:04.686700 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.686854 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":809119,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":5155,"raft.queue_time_us":38804,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":15355,"thread_start_us":1276145,"threads_started":3,"wal-append.queue_time_us":767952}
W20260709 12:16:04.947446 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 228488us
I20260709 12:16:04.722437 19512 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:16:04.947894 19814 consensus_meta.cc:287] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.663s	user 0.012s	sys 0.002s
I20260709 12:16:04.737293 19159 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:04.766613 19944 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.114044 19814 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.116765 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.972759 19793 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.117877 19793 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:05.118307 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:05.119282 11248 raft_consensus.cc:2468] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
W20260709 12:16:04.974660 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.985144 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.779623 19922 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.002468 19827 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:05.004194 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:05.004896 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.004709 19887 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.101341 19923 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.009034 19888 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:05.010892 19943 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.767483 19928 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.018757 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:05.019011 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:04.960479 19951 consensus_queue.cc:1243] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:16:05.020186 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 344206us
I20260709 12:16:05.021046 19949 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:05.021658 19742 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:04.957140 19853 raft_consensus.cc:740] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:04.668700 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:05.027248 11250 raft_consensus.cc:1275] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:16:05.085865 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:05.114022 19820 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.785329 19847 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.120385 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:05.117321 19814 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:05.007813 19710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.754s	user 0.002s	sys 0.012s
I20260709 12:16:05.129681 11323 raft_consensus.cc:2468] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:16:05.138160 11328 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:16:05.138375 11932 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 31 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 19.
W20260709 12:16:05.155659 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 309255us
I20260709 12:16:05.158241 11324 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:05.160372 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.140802 11253 rpcz_store.cc:276] 0709 12:16:02.112307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.323627 (+1211320us) service_pool.cc:224] Handling call
0709 12:16:03.358926 (+ 35299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.359197 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:16:03.423694 (+ 64497us) log.cc:844] Serialized 14 byte log entry
0709 12:16:04.139670 (+715976us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:04.139745 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:16:04.139810 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.139900 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.140057 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16989,"prepare.queue_time_us":61885,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":16929,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":4918,"thread_start_us":698610,"threads_started":1}
I20260709 12:16:05.146664 19949 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.171226 11930 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:16:05.171445 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:05.172458 11255 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
W20260709 12:16:05.173036 11901 connection.cc:256] RPC call timeout handler was delayed by -1.2070555530001457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 12:16:05.173807 11249 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:05.175264 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:05.176344 11249 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:05.178298 19798 consensus_queue.cc:1048] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:05.179323 19958 raft_consensus.cc:3055] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:16:05.179948 19958 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:05.181720 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3415) took 2960 ms. Trace:
I20260709 12:16:05.181991 19774 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.183074 19951 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.182854 19853 consensus_queue.cc:260] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.183871 19951 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.185221 19853 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:16:05.185729 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.186386 19648 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:05.186980 19951 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.187515 19708 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.222877 19708 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.225135 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:05.171309 19710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.969s	user 0.002s	sys 0.013s
I20260709 12:16:05.187634 19946 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:05.182883 19774 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.183128 11181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45582 (request call id 2901) took 3737 ms. Trace:
I20260709 12:16:05.188400 19820 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.189245 19772 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:16:05.189769 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:05.190109 19827 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.191263 19888 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.192438 11325 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:05.197976 11251 raft_consensus.cc:2393] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 23: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:05.182242 11254 rpcz_store.cc:276] 0709 12:16:02.111988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:03.320004 (+1208016us) service_pool.cc:224] Handling call
0709 12:16:03.323618 (+  3614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:03.668316 (+344698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.055075 (+1386759us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54002a58d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:05.072683 (+ 17608us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764634368}
I20260709 12:16:05.200340 19847 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:05.201941 19742 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.202641 19928 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.203307 11252 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:05.203418 11324 raft_consensus.cc:2468] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:05.206885 19949 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:05.209281 19822 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:05.210759 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 115482us
I20260709 12:16:05.211951 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:05.214573 19963 log.cc:927] Time spent T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:16:05.215255 19922 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:05.215725 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65316.6us
I20260709 12:16:05.216591 19956 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:05.250540 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 63438.2us
I20260709 12:16:05.216773 11249 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:05.217693 11256 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:16:05.218701 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:05.219224 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.218745 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:05.240612 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 113944us
I20260709 12:16:05.226153 11327 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:05.233202 11181 rpcz_store.cc:276] 0709 12:16:01.430089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.785897 (+355808us) service_pool.cc:224] Handling call
0709 12:16:01.786161 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:04.208441 (+2422280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.075585 (+867144us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b540022fe54. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:05.167776 (+ 92191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5569168768}
I20260709 12:16:05.229914 19946 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.245345 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:05.246955 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 72242us
I20260709 12:16:05.232986 19774 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.227618 19887 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.238888 19772 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:05.225579 19708 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:05.243263 19827 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:05.243541 19710 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:05.261135 19946 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:05.283892 19956 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:05.284178 19822 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.286386 11073 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 22, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 22 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:05.289937 11329 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 19.
I20260709 12:16:05.294953 19814 consensus_queue.cc:1048] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:05.204944 19958 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.298211 19670 consensus_queue.cc:1059] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:05.274068 19888 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:05.299058 19774 consensus_queue.cc:1175] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" responder_term: 25 status { last_received { term: 22 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 22 index: 13 } }
I20260709 12:16:05.293359 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:05.305920 11255 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:05.275949 19772 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:05.288728 11073 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.300748 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:05.315379 11930 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 14
W20260709 12:16:05.318279 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 106575us
I20260709 12:16:05.318280 11930 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:16:05.325261 11930 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:05.328202 11223 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:05.330703 19959 raft_consensus.cc:2804] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:05.330799 11223 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:05.331372 19959 raft_consensus.cc:697] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:05.331940 19159 raft_consensus.cc:2804] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:05.331830 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:05.332569 11223 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:05.332664 19959 consensus_queue.cc:237] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.337313 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:05.337670 19975 consensus_queue.cc:1048] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:16:05.327330 19710 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:05.341002 19741 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.345644 19820 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:05.346302 19950 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:05.332610 19159 raft_consensus.cc:697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:05.345731 19742 consensus_queue.cc:1059] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:16:05.350653 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:05.343555 19888 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:05.357671 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.351179 11073 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.361312 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.333159 19887 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:16:05.354977 19951 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.338605 19922 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:16:05.373376 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:05.374717 19512 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.359437 19888 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:05.379240 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 99317.7us
I20260709 12:16:05.368471 19741 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.407464 19490 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:05.408082 19822 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.408936 11327 raft_consensus.cc:1240] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:05.413292 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:05.393754 19888 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:05.414032 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.435281 11328 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:05.439116 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:05.440276 19887 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:16:05.443645 19936 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:05.444285 19936 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.444727 19936 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:05.449819 19512 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:05.463306 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60340.9us
W20260709 12:16:05.449697 19744 consensus_meta.cc:287] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.947s	user 0.021s	sys 0.001s
I20260709 12:16:05.410892 19951 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.492290 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.498507 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:05.491427 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:05.450917 19772 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.516243 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.478809 19820 consensus_queue.cc:1048] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:16:05.521680 19822 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:05.522671 19822 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.525578 19822 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:05.526223 19906 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:05.529368 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 179993us
W20260709 12:16:05.532953 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46396.9us
I20260709 12:16:05.538672 19814 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:05.488023 19744 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.539222 19814 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:05.550055 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.547081 19906 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.568440 19906 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:05.572497 19708 raft_consensus.cc:3055] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:05.573117 19708 raft_consensus.cc:740] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:05.576555 19951 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:05.566120 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:05.587463 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 47129.3us
I20260709 12:16:05.598354 11252 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
W20260709 12:16:05.655983 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.682143 11250 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:05.682288 11328 raft_consensus.cc:1240] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:05.585757 19814 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:05.690518 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.703560 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:05.704556 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:05.709991 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 106675us
I20260709 12:16:05.509403 19159 consensus_queue.cc:237] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:05.711463 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.746413 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:05.763551 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 185309us
I20260709 12:16:05.763731 11326 raft_consensus.cc:2468] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 30.
I20260709 12:16:05.711026 19951 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.763209 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.778367 11328 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 28 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:16:05.598819 19906 consensus_queue.cc:1059] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.126s
W20260709 12:16:05.784003 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.752930 11327 raft_consensus.cc:2468] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:05.792870 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.797101 11933 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:05.766086 19772 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:05.788699 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:05.812404 19747 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:05.813490 11249 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26. Ops: []
W20260709 12:16:05.815050 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:05.835395 11148 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:05.839753 11325 consensus_meta.cc:287] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.575s	user 0.012s	sys 0.001s
I20260709 12:16:05.840492 11325 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:16:05.798247 11073 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 22 to 28. New cstate: current_term: 28 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.841928 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:05.842993 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.808075 19512 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:05.880581 11327 raft_consensus.cc:2468] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:16:05.885489 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 203614us
I20260709 12:16:05.844050 19950 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:16:05.896925 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:05.905923 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.880079 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.879591 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:05.917483 11251 raft_consensus.cc:2393] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 22: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:05.882542 19512 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:05.936508 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:05.937493 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:05.905164 19950 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:05.955720 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 158915us
I20260709 12:16:05.954831 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:05.959111 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 206784us
I20260709 12:16:05.814437 19708 consensus_queue.cc:260] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.905862 19870 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:05.963239 19980 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 12:16:05.964057 19982 log.cc:927] Time spent T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:16:05.969702 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3661) took 1150 ms. Trace:
I20260709 12:16:05.969817 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:05.970271 11252 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33812 (request call id 2911) took 1557 ms (client timeout 2026 ms). Trace:
I20260709 12:16:05.952943 19512 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:05.972225 11322 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:05.958638 11331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3657) took 1100 ms. Trace:
I20260709 12:16:05.972791 19755 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.979584 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.981806 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:05.931622 19906 raft_consensus.cc:3055] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:16:05.993039 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.885918 19951 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:06.001803 11252 rpcz_store.cc:269] 0709 12:16:04.413311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.425940 (+ 12629us) service_pool.cc:224] Handling call
0709 12:16:05.935999 (+1510059us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54002a4ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:05.970194 (+ 34195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267677568}
I20260709 12:16:05.958446 19950 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:06.016543 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:06.019029 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:05.930044 19490 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:16:05.929517 19936 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.025530 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:06.071276 11323 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:16:06.028460 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.044150 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.044623 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.074697 19959 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.075410 19959 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.076367 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:06.077690 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:06.045325 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 135918us
I20260709 12:16:06.046020 19710 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:16:06.048667 19798 raft_consensus.cc:3055] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:16:06.052727 11901 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:06.056257 19670 raft_consensus.cc:3055] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:16:06.057940 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.100931 11249 raft_consensus.cc:2393] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 28: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:06.059067 19751 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:06.070432 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 226684us
I20260709 12:16:06.097934 11927 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:16:06.097445 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 87840.8us
I20260709 12:16:06.066146 11257 raft_consensus.cc:1240] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:06.097368 11328 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:06.008601 19744 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:05.922127 19747 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:05.995450 19870 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.026890 19887 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.024102 19906 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:06.000370 11330 rpcz_store.cc:276] 0709 12:16:04.658216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.658351 (+   135us) service_pool.cc:224] Handling call
0709 12:16:04.658650 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:04.658863 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:16:05.148904 (+490041us) log.cc:844] Serialized 14 byte log entry
0709 12:16:05.750236 (+601332us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:05.765095 (+ 14859us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:16:05.774308 (+  9213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.795987 (+ 21679us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:05.808521 (+ 12534us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32006,"prepare.queue_time_us":451026,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":71851,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":857377,"threads_started":3,"wal-append.queue_time_us":681004}
I20260709 12:16:06.006809 11331 rpcz_store.cc:276] 0709 12:16:04.650019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.651145 (+  1126us) service_pool.cc:224] Handling call
0709 12:16:04.663561 (+ 12416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:04.663800 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:16:05.258230 (+594430us) log.cc:844] Serialized 14 byte log entry
0709 12:16:05.685738 (+427508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:05.707184 (+ 21446us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:16:05.707243 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.724063 (+ 16820us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:05.750813 (+ 26750us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43669,"prepare.queue_time_us":446470,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":60129,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":758187,"threads_started":3,"wal-append.queue_time_us":260320}
W20260709 12:16:06.133160 11295 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.134788 11327 raft_consensus.cc:1275] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:16:06.136547 19959 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:06.138064 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 156438us
I20260709 12:16:06.132848 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:06.110109 19798 raft_consensus.cc:740] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:06.142020 11329 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:16:06.142448 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 89913.7us
I20260709 12:16:06.144970 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.099879 19710 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:06.146147 11930 raft_consensus.cc:2410] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:16:06.146633 19710 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.147466 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.148211 19710 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:06.149117 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.150658 11322 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:06.151721 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.146701 19998 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:06.153554 19998 raft_consensus.cc:697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:06.154340 11251 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:16:06.154595 19998 consensus_queue.cc:237] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.160601 19951 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.161216 19951 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.152933 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.147126 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.170421 20015 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:16:06.171069 20003 consensus_queue.cc:1059] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:16:06.172086 19946 consensus_meta.cc:287] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.859s	user 0.005s	sys 0.006s
I20260709 12:16:06.109035 19670 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:06.173748 19670 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.174878 19670 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:06.177902 19887 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.180040 19958 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:06.180576 19887 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.109723 19975 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:16:06.182169 19878 consensus_meta.cc:287] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 1.077s	user 0.010s	sys 0.006s
I20260709 12:16:06.027928 19755 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.183012 19975 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:06.183597 19975 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:06.183915 11325 raft_consensus.cc:1240] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:06.182228 11073 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 29, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 29 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:06.185907 11073 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.187021 19870 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.118001 19742 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:06.187674 19747 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.187938 19742 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.187865 11073 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 19 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.190191 19742 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.190413 11073 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.191466 19951 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:06.192093 19853 consensus_meta.cc:287] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.940s	user 0.002s	sys 0.009s
W20260709 12:16:06.197296 19888 consensus_meta.cc:287] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.000s
W20260709 12:16:06.189203 19949 consensus_meta.cc:287] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.908s	user 0.005s	sys 0.007s
W20260709 12:16:06.199546 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 58017.5us
W20260709 12:16:06.218899 11067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.023s	sys 0.020s
I20260709 12:16:06.239571 11322 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
I20260709 12:16:06.172686 19946 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:06.244302 11067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.023s	sys 0.020s
I20260709 12:16:06.245414 11330 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 28. Current term is 29. Ops: []
I20260709 12:16:06.209298 19888 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.219726 19990 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 12:16:06.209131 19949 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.220405 19747 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:06.254204 19995 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:16:06.257304 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.206084 19751 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:06.255451 19814 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.002s
I20260709 12:16:06.269502 11323 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:06.279223 11255 raft_consensus.cc:2393] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 20: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:16:06.255857 11067 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.677s	user 0.024s	sys 0.020s
W20260709 12:16:06.286434 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.257505 19990 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:16:06.297784 19990 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:06.298764 19990 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.299369 19946 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.300043 19990 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:06.300513 11073 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 20 to 23, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 23 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:06.280471 11253 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
W20260709 12:16:06.292055 11220 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.292487 11078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 262) took 40699 ms (client timeout 15000 ms). Trace:
I20260709 12:16:06.292233 19950 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.293267 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.302306 11931 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:06.303592 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.309136 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.310081 11249 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:06.306720 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.314038 11257 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:16:06.315689 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.319654 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.297297 11329 raft_consensus.cc:2468] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:16:06.323570 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.328094 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:06.329217 11933 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:06.332401 11252 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:06.339296 11248 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:06.338744 11254 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:06.290563 19814 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.343400 11148 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.345046 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.346804 11253 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
W20260709 12:16:06.346271 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.297793 19747 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.331553 19936 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:06.342757 19793 consensus_meta.cc:287] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 1.224s	user 0.013s	sys 0.003s
I20260709 12:16:06.355832 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:06.353027 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 180769us
I20260709 12:16:06.342552 19887 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.365806 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:06.371454 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.375053 19751 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.319296 19906 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.376982 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.376044 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.380254 11250 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:06.371310 11073 catalog_manager.cc:5697] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 25 to 30. New cstate: current_term: 30 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.385763 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.387034 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.387734 11250 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:06.388518 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:06.388991 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.389496 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.392457 19512 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.394315 19888 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.394536 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.395968 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:06.378191 19946 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.397827 19870 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.382710 19853 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:06.384357 19951 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.398926 19878 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:06.399453 19922 raft_consensus.cc:2804] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:06.368518 20010 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.412565 11252 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 29 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
W20260709 12:16:06.403677 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.404480 19751 consensus_queue.cc:1059] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:06.406193 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:06.414463 11250 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:16:06.397294 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:06.382431 19772 consensus_meta.cc:287] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.003s
I20260709 12:16:06.365165 19793 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:06.398434 20017 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:16:06.398399 19708 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:06.410323 11249 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:06.419116 11254 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:06.420065 11073 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 22 to 26, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 26 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.411932 19906 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:06.425149 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.425753 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:06.426430 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.426826 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 120744us
I20260709 12:16:06.429795 11326 raft_consensus.cc:1240] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:16:06.415359 19847 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.431913 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.432653 19814 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:06.437461 19793 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:06.437812 19772 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:06.438271 19772 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:06.438678 19772 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:06.442672 19747 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.442951 19751 raft_consensus.cc:3055] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:06.416074 19922 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:06.446063 19922 raft_consensus.cc:3060] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:06.447714 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52014 (request call id 1375) took 1073 ms. Trace:
W20260709 12:16:06.448267 20019 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:16:06.448175 11931 rpcz_store.cc:276] 0709 12:16:05.380264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:05.380384 (+   120us) service_pool.cc:224] Handling call
0709 12:16:06.447471 (+1067087us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540069fad4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:06.447618 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1997546880}
I20260709 12:16:06.422870 19946 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.452490 19946 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.455401 19755 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.455622 19772 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.430083 19870 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.460407 19870 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.460692 11073 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:06.302769 11078 rpcz_store.cc:269] 0709 12:15:25.521862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.522430 (+   568us) service_pool.cc:224] Handling call
0709 12:16:06.216219 (+40693789us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:58.229102 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:58.240787 (+ 11685us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:58.240865 (+    78us) write_op.cc:435] Acquired schema lock
0709 12:15:58.240885 (+    20us) tablet.cc:662] Decoding operations
0709 12:15:58.243350 (+  2465us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:58.243504 (+   154us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:58.243543 (+    39us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:15:58.247056 (+  3513us) tablet.cc:701] Row locks acquired
0709 12:15:58.247201 (+   145us) write_op.cc:260] PREPARE: finished
0709 12:15:58.247617 (+   416us) write_op.cc:270] Start()
0709 12:15:58.248000 (+   383us) write_op.cc:276] Timestamp: P: 1783599358247466 usec, L: 0
0709 12:15:58.248063 (+    63us) op_driver.cc:348] REPLICATION: starting
0709 12:15:58.250512 (+  2449us) log.cc:844] Serialized 7569 byte log entry
0709 12:15:59.194157 (+943645us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec35e20ac 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b
0709 12:15:59.741963 (+547806us) op_driver.cc:464] REPLICATION: finished
0709 12:16:00.249321 (+507358us) write_op.cc:301] APPLY: starting
0709 12:16:00.253044 (+  3723us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:00.552273 (+299229us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:00.553878 (+  1605us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:01.736362 (+1182484us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:01.904269 (+167907us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 12:16:01.951295 (+ 47026us) write_op.cc:312] APPLY: finished
0709 12:16:02.222485 (+271190us) log.cc:844] Serialized 172 byte log entry
0709 12:16:02.409607 (+187122us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:02.436036 (+ 26429us) write_op.cc:454] Released schema lock
0709 12:16:02.445314 (+  9278us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:02.454569 (+  9255us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:02.454606 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1167249,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":492767,"apply.run_cpu_time_us":87010,"apply.run_wall_time_us":2213199,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83699,"num_ops":19,"prepare.queue_time_us":121822,"prepare.run_cpu_time_us":86562,"prepare.run_wall_time_us":1207701,"raft.queue_time_us":261954,"raft.run_cpu_time_us":37859,"raft.run_wall_time_us":227163,"replication_time_us":1493703,"spinlock_wait_cycles":7075712,"thread_start_us":1684696,"threads_started":5,"wal-append.queue_time_us":1399582,"wal-append.run_cpu_time_us":25997,"wal-append.run_wall_time_us":2603232}]]}
I20260709 12:16:06.456048 19755 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.462450 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:06.464903 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:06.466864 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 69663.1us
W20260709 12:16:06.470364 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.406275 19887 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.443512 19751 raft_consensus.cc:740] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:06.470121 19755 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.479985 19755 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:06.474857 20015 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 12:16:06.482357 20015 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:16:06.482744 20015 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:06.483745 20015 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.485343 20015 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:06.498131 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:06.499285 11330 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:06.457845 19772 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.489277 11223 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:06.469113 19744 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:06.518129 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.516134 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:06.466038 19490 consensus_queue.cc:1059] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:16:06.457859 19847 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:06.522521 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.521286 19956 raft_consensus.cc:2804] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:06.425220 20010 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.522994 11073 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 26, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 26 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.524163 11326 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:06.547120 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:06.548419 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 83601.7us
I20260709 12:16:06.549268 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:06.513831 19755 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:06.557323 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 94947.4us
W20260709 12:16:06.561857 11296 leader_election.cc:336] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:06.518620 19870 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:06.567417 11148 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:06.452716 19798 consensus_queue.cc:260] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.573403 20009 consensus_queue.cc:1048] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 12:16:06.573740 19755 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.575953 19755 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.577740 20022 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:06.578357 19751 consensus_queue.cc:260] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.571569 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:06.577198 11073 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 27, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 27 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:06.583058 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.590152 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:06.594044 20020 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:06.561882 19946 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.569269 19956 raft_consensus.cc:697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 28 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:06.591992 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:06.599660 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:06.580180 19512 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.608157 11328 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 29: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:06.608259 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:06.616014 11330 raft_consensus.cc:2368] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 26. Current term is 27.
I20260709 12:16:06.620823 20011 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:16:06.620882 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:06.626865 19951 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:06.627317 19951 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:06.627705 19951 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:06.628610 19958 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:06.605545 19943 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.630069 19943 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.632261 19943 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.630965 11147 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:06.634621 19741 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:06.635177 19741 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:06.636057 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.636209 19741 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.636737 11255 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:06.644945 11255 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:06.646901 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.647938 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.603804 19870 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:06.602447 19949 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:06.654465 11932 raft_consensus.cc:2393] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:06.654798 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.679560 11253 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
I20260709 12:16:06.679742 11249 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:16:06.679778 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 61975.8us
I20260709 12:16:06.666553 11254 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:06.672436 11248 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
W20260709 12:16:06.678364 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 119955us
I20260709 12:16:06.613251 19747 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:06.687276 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.694264 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.695015 11255 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:06.677917 19751 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:06.609221 19946 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.700075 11250 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:06.705363 19956 consensus_queue.cc:237] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.713440 19946 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.718909 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:06.720867 19943 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:06.721302 19943 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.680830 19950 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.721673 19943 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:06.722280 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.722704 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:06.723116 11932 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:06.725088 11249 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:06.724346 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:06.705250 20026 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:16:06.701999 11148 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.728423 11250 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:16:06.732511 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:06.732999 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 44878.9us
I20260709 12:16:06.733870 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.736585 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:06.709461 20003 raft_consensus.cc:3055] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:06.745198 11147 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:06.737416 20011 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:06.747896 20011 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.748277 20011 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:06.750378 19922 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.753773 19772 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:06.754202 19772 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.754554 19772 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:06.756258 19958 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:06.756978 19742 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:06.757535 19742 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:06.758555 19742 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:06.762277 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60412.7us
I20260709 12:16:06.763079 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:06.763942 19958 raft_consensus.cc:2804] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:16:06.764374 19958 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:06.764425 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.764886 19958 raft_consensus.cc:3060] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:06.767432 11249 raft_consensus.cc:1240] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 24. Current term is 25. Ops: [24.16-24.16]
W20260709 12:16:06.770386 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.770982 20009 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:06.771530 20009 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.776561 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:06.777331 11935 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:06.778194 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:06.779850 11927 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:16:06.780290 11927 pending_rounds.cc:85] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 15
I20260709 12:16:06.780519 11927 pending_rounds.cc:107] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:16:06.780763 11927 raft_consensus.cc:2889] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:06.782630 19946 raft_consensus.cc:3060] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:06.783886 19755 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:06.784454 19755 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.785095 11935 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:06.787169 19755 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.727229 19512 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:06.741089 19975 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.004s
I20260709 12:16:06.764190 11932 raft_consensus.cc:2410] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:16:06.799531 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.792407 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:06.797816 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 237116us
I20260709 12:16:06.738291 19870 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:16:06.806982 11296 leader_election.cc:336] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:06.807487 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:06.740815 20003 raft_consensus.cc:740] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:06.793732 20009 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:06.830042 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 304498us
W20260709 12:16:06.837751 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.804883 19870 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.845773 19923 raft_consensus.cc:2804] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:06.806668 19772 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.846410 19923 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:06.846880 19923 raft_consensus.cc:3060] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:06.847466 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.848232 11250 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:16:06.848344 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.841332 11222 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:06.843010 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:06.807931 19975 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.845908 19870 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:06.845283 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.851627 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.853807 11324 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:06.854833 11250 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:06.822376 19708 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:16:06.858116 11250 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:06.858402 19975 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:06.858693 19708 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:16:06.862295 20027 log.cc:927] Time spent T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:16:06.862885 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.864738 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:06.866396 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 59450.1us
I20260709 12:16:06.866904 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:06.867165 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:06.867515 11932 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
I20260709 12:16:06.867620 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.868731 20003 consensus_queue.cc:260] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.853344 19755 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:06.871615 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:06.871762 19943 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.838404 19922 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.856133 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.873988 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:06.826452 19887 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.805509 19512 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:06.878448 11932 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
I20260709 12:16:06.882746 11927 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:16:06.855479 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63169us
W20260709 12:16:06.884997 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:06.885561 19906 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:06.887518 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.848244 19772 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.892030 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:06.897886 19906 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:06.893807 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:06.899688 11067 maintenance_manager.cc:643] P bbfb3415e272477dac9ba599cc40a89d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 2.413s	user 0.064s	sys 0.025s Metrics: {"bytes_written":5218,"delete_count":0,"lbm_write_time_us":6417,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":4,"reinsert_count":0,"thread_start_us":666671,"threads_started":1,"update_count":198,"wal-append.queue_time_us":642275}
I20260709 12:16:06.900903 11148 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:06.894603 19922 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:06.806818 19958 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:06.895507 20021 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:16:06.908630 11296 leader_election.cc:336] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:06.901835 19772 raft_consensus.cc:2804] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:16:06.918601 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 73396.6us
I20260709 12:16:06.920342 11147 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:06.922413 19742 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:06.922994 19742 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:06.924381 19772 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.924098 19742 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.924856 19772 raft_consensus.cc:3060] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:06.911430 19990 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:06.930007 19990 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:06.937953 11251 raft_consensus.cc:1217] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Deduplicated request from leader. Original: 17.10->[19.11-19.11]   Dedup: 19.11->[]
I20260709 12:16:06.938477 11251 raft_consensus.cc:1240] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:16:06.937474 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2921) took 1155 ms. Trace:
W20260709 12:16:06.944399 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.933269 11068 maintenance_manager.cc:419] P bbfb3415e272477dac9ba599cc40a89d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.261627
I20260709 12:16:06.933243 19946 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:06.933373 19943 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:06.952566 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:06.956271 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 79352us
W20260709 12:16:06.956533 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 115264us
I20260709 12:16:06.941076 19922 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:06.966151 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 57558.3us
W20260709 12:16:06.966889 11296 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:06.940268 11256 rpcz_store.cc:276] 0709 12:16:05.781764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:05.793201 (+ 11437us) service_pool.cc:224] Handling call
0709 12:16:05.832277 (+ 39076us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:16:06.097028 (+264751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:06.097074 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:16:06.165973 (+ 68899us) log.cc:844] Serialized 52 byte log entry
0709 12:16:06.348553 (+182580us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:16:06.370593 (+ 22040us) log.cc:844] Serialized 14 byte log entry
0709 12:16:06.370793 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:06.370864 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:06.440341 (+ 69477us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b18002edbc0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec3579c0a 00007f3ecccb9219 00007f3eca5c058f 00007f3ec35ff571 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd
0709 12:16:06.693536 (+253195us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b18002edbc0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:16:06.937209 (+243673us) raft_consensus.cc:1685] finished
0709 12:16:06.937241 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:06.937410 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21303,"prepare.queue_time_us":17589,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":122637,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":12404,"spinlock_wait_cycles":41189504,"thread_start_us":380680,"threads_started":2,"wal-append.queue_time_us":238174}
I20260709 12:16:06.967248 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:06.971961 20033 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 12:16:06.972821 20030 log.cc:927] Time spent T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:16:06.978690 19958 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:06.980680 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:06.987174 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.006961 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:06.979405 20038 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:16:07.009982 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:07.019057 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.019164 11256 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
W20260709 12:16:07.021525 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.023015 19512 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.028709 11257 raft_consensus.cc:1240] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: [23.19-23.19]
I20260709 12:16:06.985302 19923 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:07.031069 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64208.6us
W20260709 12:16:07.009131 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:07.029016 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:07.037706 11222 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.020318 19906 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:16:07.045744 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:07.049310 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 148474us
I20260709 12:16:07.050510 11148 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:07.052780 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:07.011161 19772 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.060732 11148 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:07.065802 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:07.069919 11256 raft_consensus.cc:1240] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 28. Current term is 29. Ops: []
I20260709 12:16:07.074307 11148 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:07.069156 11222 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.077335 11222 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.055642 19512 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:07.111388 19772 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:07.115162 19923 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:07.084084 19887 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:07.143849 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.146443 19708 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:07.075771 19958 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:07.155040 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:07.155912 20010 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.156481 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:07.072301 19922 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:07.159854 19990 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.167013 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.168223 11326 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 28: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:07.177810 19847 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:07.181010 11223 leader_election.cc:400] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 26. Current term is 27.
I20260709 12:16:07.181465 11223 leader_election.cc:403] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
W20260709 12:16:07.181720 20039 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.172s	user 0.003s	sys 0.000s
I20260709 12:16:07.182449 19847 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:07.085103 20045 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:16:07.190667 11326 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:16:07.145418 19951 consensus_meta.cc:287] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.004s
I20260709 12:16:07.192596 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:07.192628 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:07.191817 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 131993us
W20260709 12:16:07.197309 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:07.198434 19975 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:16:07.206039 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 131783us
I20260709 12:16:07.207109 11148 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:07.226016 11900 leader_election.cc:336] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:07.227753 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 158622us
W20260709 12:16:07.228709 11222 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.229166 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.229907 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.231073 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:07.226696 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.233562 11256 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:16:07.237356 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:07.238133 11251 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:07.240476 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:07.225819 19772 raft_consensus.cc:2804] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:16:07.246526 11900 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.247138 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.201195 19951 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.232791 19742 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:07.205013 19923 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:16:07.202999 19870 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.246089 19975 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:07.244239 19772 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.258208 19772 raft_consensus.cc:3060] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:16:07.263530 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.263979 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:07.265106 11900 leader_election.cc:336] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.265681 19922 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:07.265882 19975 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:07.267282 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:07.289410 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60743.6us
W20260709 12:16:07.290124 11900 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:07.253443 20011 consensus_meta.cc:287] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
W20260709 12:16:07.262288 20020 consensus_meta.cc:287] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
I20260709 12:16:07.254038 19990 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:07.302642 11222 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.320489 19958 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:07.325269 11222 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.327162 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.306202 20020 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:07.322284 20002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.008s	sys 0.022s
I20260709 12:16:07.334898 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:07.334503 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 88823.5us
I20260709 12:16:07.315227 19742 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:07.295871 20011 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.325017 20048 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:16:07.323024 19798 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:16:07.337080 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 180656us
W20260709 12:16:07.341809 20002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.009s	sys 0.022s
W20260709 12:16:07.342053 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.344854 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:07.359813 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.357597 11249 raft_consensus.cc:1240] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 23. Current term is 24. Ops: []
W20260709 12:16:07.357859 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.360639 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:07.345418 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:07.364902 11222 leader_election.cc:336] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.365463 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:07.368275 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 138474us
I20260709 12:16:07.375808 19870 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.376814 19951 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:07.378597 20011 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.379223 11331 raft_consensus.cc:1275] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:16:07.379633 19975 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:07.380028 19975 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:07.380261 19707 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:16:07.380461 19975 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:07.335078 19998 consensus_queue.cc:1059] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:16:07.383612 19998 raft_consensus.cc:3055] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:16:07.383999 19998 raft_consensus.cc:740] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:07.384809 19772 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.386188 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:07.387557 19772 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:07.390681 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.339373 19751 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:07.339700 19906 raft_consensus.cc:2764] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:16:07.394078 11328 raft_consensus.cc:2393] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:07.343158 19923 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.399120 11148 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:07.401568 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.403900 11327 raft_consensus.cc:2410] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:16:07.459712 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 99951.2us
W20260709 12:16:07.466150 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 101284us
W20260709 12:16:07.466958 11222 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.467319 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:07.460707 20022 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:07.460598 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56643.4us
W20260709 12:16:07.471920 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 111389us
I20260709 12:16:07.385037 19998 consensus_queue.cc:260] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.406770 19951 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:16:07.483794 11222 leader_election.cc:336] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.484436 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:07.488762 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 89747.1us
I20260709 12:16:07.398568 19870 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.076s
I20260709 12:16:07.469907 19742 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:07.500624 20040 consensus_queue.cc:1059] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.133s
I20260709 12:16:07.480546 20055 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:07.463718 19923 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:07.525480 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.527088 19923 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:07.475435 20022 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:16:07.479107 19512 consensus_queue.cc:1048] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 12:16:07.530529 20049 log.cc:927] Time spent T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:16:07.538702 11296 connection.cc:256] RPC call timeout handler was delayed by -1.1522367479999502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
W20260709 12:16:07.514685 11222 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.476725 19922 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.518007 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.520454 19998 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:07.494283 19887 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:07.559191 19512 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:07.560084 19512 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.560545 19922 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.545341 11251 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 29.
I20260709 12:16:07.556253 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.563289 19922 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.563328 11323 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:16:07.563598 20009 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:16:07.564340 11147 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:07.564497 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.565117 11323 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:07.565451 19512 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:07.571035 11329 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:07.573127 11323 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 29.
I20260709 12:16:07.545214 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:07.574157 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.574193 11325 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:07.575524 11331 raft_consensus.cc:1240] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 27. Current term is 28. Ops: []
I20260709 12:16:07.572060 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:07.578151 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63511.9us
I20260709 12:16:07.578343 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:07.578907 11222 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.579355 11222 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.583109 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:07.505587 19906 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 12:16:07.545053 19742 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:07.521042 19946 consensus_queue.cc:1059] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:16:07.585749 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:07.615635 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:07.617132 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.615061 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.626124 19923 raft_consensus.cc:3055] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:07.626574 20048 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:07.626833 19923 raft_consensus.cc:740] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:07.627300 20048 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:07.627931 20048 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:07.628455 20060 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.628286 19923 consensus_queue.cc:260] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.629020 20060 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.629976 20051 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:07.584712 20055 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:07.586645 20050 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.646077 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.620761 19887 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:16:07.647981 11933 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:07.650664 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.651082 11325 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:07.617756 19922 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.625023 20054 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:07.653348 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.654954 11329 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 27: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:07.630568 19958 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:07.660128 19958 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.661387 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.662147 11325 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:16:07.662565 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.630599 20051 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:07.672081 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:07.673434 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53988.8us
I20260709 12:16:07.649816 19923 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:07.673429 19922 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.674809 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.676363 20060 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.676802 19922 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.677011 20062 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.677700 20054 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 29 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:07.677670 20062 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.678848 20054 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.682994 11329 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 27: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:07.683817 11148 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.683924 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.668741 19951 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.689879 20011 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.690317 19951 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.691134 19742 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:07.692390 19847 consensus_meta.cc:287] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 12:16:07.692615 19951 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.692939 19847 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 26. Current term is 27.
I20260709 12:16:07.693305 19951 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:07.695842 20055 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.672873 20051 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:07.647881 20050 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.719326 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.690379 20011 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:07.738348 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 52748.6us
I20260709 12:16:07.748138 11329 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 23
W20260709 12:16:07.749017 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 74638.5us
W20260709 12:16:07.752700 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 178094us
I20260709 12:16:07.689102 19975 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.753055 19906 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.758787 19906 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.759824 20050 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:07.760969 19906 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:07.766049 11147 leader_election.cc:336] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Timed out: RequestConsensusVote RPC to 127.9.198.196:39119 timed out after 1.549s (SENT)
I20260709 12:16:07.749555 19853 raft_consensus.cc:3055] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:07.770695 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:07.770953 11325 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:16:07.775053 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.754197 19958 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.776064 11935 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:16:07.777935 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.750342 19922 consensus_queue.cc:1059] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:16:07.761828 19847 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:07.775612 19741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.132s	user 0.011s	sys 0.006s
I20260709 12:16:07.774881 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:07.782326 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:07.785840 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.786486 11930 raft_consensus.cc:1240] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 30. Current term is 31. Ops: []
I20260709 12:16:07.786543 11325 raft_consensus.cc:2468] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:07.769201 19853 raft_consensus.cc:740] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:07.791733 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.792546 11325 raft_consensus.cc:2376] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 29. Re-sending same reply.
I20260709 12:16:07.787689 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.794234 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.795421 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:07.796514 20050 raft_consensus.cc:2749] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:16:07.798228 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:07.797755 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:07.785315 11065 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4a634acd9ff543cca551c8a28e72d6c4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica fecfc03de52a4308bc9b045cad5aac1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:07.800019 19906 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:07.793691 19887 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:07.805069 19887 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:07.805594 19887 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:07.806710 19853 consensus_queue.cc:260] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.808056 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:07.808585 19853 raft_consensus.cc:3060] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:07.812134 20070 consensus_queue.cc:1059] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:16:07.785614 19741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.012s	sys 0.006s
I20260709 12:16:07.794760 11325 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
W20260709 12:16:07.818037 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.824863 11329 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:07.826679 20077 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:07.828294 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.831364 11325 raft_consensus.cc:2393] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 27: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:07.828819 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.834043 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.836607 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.832976 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:07.834749 19751 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:07.841548 19751 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:07.842020 19751 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:07.842399 11328 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 12:16:07.842924 11328 pending_rounds.cc:85] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 13
I20260709 12:16:07.843199 11328 pending_rounds.cc:107] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:16:07.843490 11328 raft_consensus.cc:2889] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:07.846629 20011 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.850150 19975 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.833915 19512 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:16:07.857129 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:07.857908 11326 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:07.857996 11253 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:16:07.858902 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.859685 11326 raft_consensus.cc:2393] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 28: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:07.862334 11325 raft_consensus.cc:2410] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:16:07.839301 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:07.791126 19847 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.861995 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:07.878087 11254 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:07.835196 20077 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:07.864442 19512 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:07.880252 20010 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:07.882587 19906 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:16:07.898280 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.883211 19747 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:07.882309 20077 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:07.886904 19956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.000s	sys 0.017s
W20260709 12:16:07.910374 19956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.135s	user 0.000s	sys 0.018s
I20260709 12:16:07.883430 20020 consensus_queue.cc:1059] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.025s
I20260709 12:16:07.894431 20010 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:07.921303 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 94108.6us
I20260709 12:16:07.922668 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:07.924154 19906 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:07.931463 11257 raft_consensus.cc:1240] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:07.913254 19747 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:07.934406 20020 raft_consensus.cc:3055] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:16:07.934931 20020 raft_consensus.cc:740] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 30 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:16:07.935843 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:07.936357 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.949018 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:07.936395 19747 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:07.949998 11253 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:07.921314 20010 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:07.956092 11147 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:07.965200 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:07.966621 11255 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:07.973815 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:07.974584 11252 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:07.974782 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1385) took 1207 ms. Trace:
I20260709 12:16:07.975274 11254 rpcz_store.cc:276] 0709 12:16:06.767090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:06.774548 (+  7458us) service_pool.cc:224] Handling call
0709 12:16:06.774843 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:07.878541 (+1103698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:07.974492 (+ 95951us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002a4ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:07.974716 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2536763264}
I20260709 12:16:07.978410 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:07.983119 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60552.9us
I20260709 12:16:07.985766 11254 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 21.
I20260709 12:16:07.988896 11254 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:16:07.967425 19853 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:16:07.992647 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.993716 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:07.989339 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
W20260709 12:16:07.996033 11296 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:07.996714 11253 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:07.996537 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.003294 20083 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:08.003568 19751 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.008348 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:08.008940 19853 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:08.009828 20011 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:08.010017 11147 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:08.010450 20011 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.010134 19853 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.010886 20011 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.011037 19742 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:08.012835 19742 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:08.013425 19742 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:08.014550 19742 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.001462 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.030682 11252 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:08.033293 11325 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:16:08.033593 11255 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
W20260709 12:16:08.035341 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.049216 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.060158 19847 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:08.070782 11249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52330 (request call id 1386) took 1303 ms. Trace:
I20260709 12:16:08.071310 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:08.071256 11249 rpcz_store.cc:276] 0709 12:16:06.767413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:06.768263 (+   850us) service_pool.cc:224] Handling call
0709 12:16:06.768543 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:07.360596 (+592053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:08.050737 (+690141us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54002a49d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:08.070712 (+ 19975us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354253056}
I20260709 12:16:07.935902 20020 consensus_queue.cc:260] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.072059 11930 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 21.
I20260709 12:16:08.079382 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:08.075759 11257 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:08.089303 11257 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:16:08.091747 11297 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.095073 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.095248 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.096850 11249 raft_consensus.cc:2410] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:16:08.099800 11256 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 27: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:08.108421 20088 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:16:08.111630 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:08.099066 19512 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:08.113051 20003 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:08.117378 19751 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:08.141880 11256 raft_consensus.cc:1240] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:16:08.132408 19512 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.149883 11249 raft_consensus.cc:1275] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:16:08.087221 20051 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:08.150177 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 154159us
I20260709 12:16:08.131652 20020 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:08.100909 19887 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.151304 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:08.139408 19906 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:08.169910 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:08.169998 11324 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 28. Current term is 29. Ops: []
I20260709 12:16:08.126902 20048 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.154733 19512 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.180284 11251 raft_consensus.cc:1240] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 24. Current term is 25. Ops: []
I20260709 12:16:08.181142 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.184417 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.172863 19906 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.186910 20083 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:08.187932 19906 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:08.185344 11256 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:08.198073 11329 raft_consensus.cc:1240] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 23. Current term is 24. Ops: []
W20260709 12:16:08.208766 11222 leader_election.cc:336] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.209488 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.210515 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.213286 11250 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:08.216620 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.217746 19847 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:08.220098 20062 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:08.229277 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:08.231294 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:08.233925 11935 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:16:08.231657 11223 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.217208 11256 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:16:08.239440 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.240563 11250 raft_consensus.cc:2468] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:08.242175 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:08.245465 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:08.246301 11252 raft_consensus.cc:2393] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 27: Already voted for candidate 016411114c6d42039bd04025378dfc98 in this term.
I20260709 12:16:08.246971 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.250420 11926 raft_consensus.cc:3055] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:08.251025 11926 raft_consensus.cc:740] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:08.252151 11926 consensus_queue.cc:260] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.211678 20054 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:08.259394 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 92590us
I20260709 12:16:08.258421 19956 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:08.258191 20038 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:08.253544 11926 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:16:08.260350 11296 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.260263 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51535.5us
I20260709 12:16:08.266362 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.264909 11935 pending_rounds.cc:85] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 15
I20260709 12:16:08.264513 11326 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:08.267006 11935 pending_rounds.cc:107] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:16:08.267283 11935 raft_consensus.cc:2889] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:16:08.271648 11222 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.275257 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.286917 11930 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
I20260709 12:16:08.245023 20077 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:08.301939 11222 leader_election.cc:336] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.302887 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.269793 20054 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:08.311055 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 50736.1us
I20260709 12:16:08.311498 20054 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:08.311815 11296 leader_election.cc:336] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.312309 20051 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:08.312268 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.315567 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:08.316332 11928 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 31: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:08.317421 11323 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:08.294596 20097 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:16:08.297364 19847 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:16:08.330726 11222 leader_election.cc:336] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.331540 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.314894 19956 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:08.317385 19741 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:08.339661 11933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52016 (request call id 1562) took 1847 ms (client timeout 1549 ms). Trace:
W20260709 12:16:08.340085 11933 rpcz_store.cc:269] 0709 12:16:06.492213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:06.492363 (+   150us) service_pool.cc:224] Handling call
0709 12:16:08.339477 (+1847114us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54006301d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:08.339594 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2596574080}
W20260709 12:16:08.341691 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:08.342132 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:08.342415 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.336689 19990 raft_consensus.cc:3055] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:16:08.346498 11296 leader_election.cc:336] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.346863 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 99925.6us
I20260709 12:16:08.332001 20051 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:08.353384 11931 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:16:08.353834 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.333578 19887 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:08.346657 19847 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:08.345059 19990 raft_consensus.cc:740] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:16:08.359938 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.366819 11222 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.368219 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.338943 20011 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.374318 19708 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.375139 19708 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.375866 20011 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:08.356915 20097 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:08.378209 20077 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:08.378387 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.379272 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.379973 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.380450 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:08.386718 11296 leader_election.cc:336] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.387452 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.393798 11926 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 19. (term mismatch)
I20260709 12:16:08.394449 11926 pending_rounds.cc:85] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 18
I20260709 12:16:08.394769 11926 pending_rounds.cc:107] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 23.19
I20260709 12:16:08.395104 11926 raft_consensus.cc:2889] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:08.404304 11328 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:16:08.414160 11326 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
W20260709 12:16:08.415863 20099 log.cc:927] Time spent T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:16:08.418879 19708 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:08.368651 20048 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:08.435391 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:08.431450 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 64705.7us
I20260709 12:16:08.391870 20077 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:08.448632 20011 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:16:08.449980 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63315.2us
W20260709 12:16:08.450841 11296 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.451210 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.452153 20077 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:08.452682 11296 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.453147 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.454424 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:08.455029 11254 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.455549 19512 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.457120 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:08.458742 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:08.437170 11147 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:08.439248 11222 leader_election.cc:336] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.440721 11926 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:16:08.442606 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.444671 11257 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:16:08.358326 19741 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.398578 20097 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:08.459314 11328 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.460213 11222 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.446146 11248 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:16:08.461548 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:08.403492 19943 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
I20260709 12:16:08.468266 11257 raft_consensus.cc:2410] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:16:08.462783 11147 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:08.470577 11147 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:08.470818 11222 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.474747 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.459177 19847 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:16:08.469904 11254 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:16:08.482868 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.463196 20011 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:16:08.496336 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:08.476516 19708 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:08.499215 19708 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.499656 19708 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:08.500428 19741 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:08.472815 19742 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:08.504169 19742 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:08.468796 20048 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:08.505729 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.475509 20009 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:08.504627 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:08.427165 19990 consensus_queue.cc:260] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:08.521795 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69134.9us
I20260709 12:16:08.502480 20107 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:08.504868 19853 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:16:08.531030 11328 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 22.
W20260709 12:16:08.503378 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.542315 11222 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.519294 20109 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:16:08.514213 20098 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.367s	user 0.003s	sys 0.000s
I20260709 12:16:08.554936 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:08.555717 11325 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 20.
I20260709 12:16:08.557417 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:08.558272 11330 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 29: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:08.559540 11326 raft_consensus.cc:1275] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:16:08.561754 20104 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 12:16:08.561913 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:08.563372 11250 raft_consensus.cc:2410] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 31 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 19.
W20260709 12:16:08.567102 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.568189 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.569247 11296 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.569706 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:08.570703 20102 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.187s	user 0.001s	sys 0.001s
I20260709 12:16:08.571439 19990 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:08.572081 11331 raft_consensus.cc:1275] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:16:08.506377 19742 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:08.549799 11222 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.578294 20048 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:08.540829 20006 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.579514 20006 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:08.580246 11222 leader_election.cc:336] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.580749 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.581859 20107 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:08.582641 11931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1565) took 1965 ms. Trace:
I20260709 12:16:08.583046 11931 rpcz_store.cc:276] 0709 12:16:06.617619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:06.617804 (+   185us) service_pool.cc:224] Handling call
0709 12:16:08.345457 (+1727653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:08.355373 (+  9916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:08.582400 (+227027us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54006301d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:08.582557 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3953691904}
I20260709 12:16:08.584290 20006 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:08.578378 11329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3382) took 1029 ms. Trace:
I20260709 12:16:08.584651 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:08.582101 11222 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.587204 11325 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 12:16:08.578459 11296 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.592227 11932 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:16:08.592984 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:08.594024 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.595145 11329 rpcz_store.cc:276] 0709 12:16:07.549279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:07.863433 (+314154us) service_pool.cc:224] Handling call
0709 12:16:07.863695 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:08.198569 (+334874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:08.578134 (+379565us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54003318d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:08.578317 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768564992}
I20260709 12:16:08.595422 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.598137 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:08.628221 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 46138.8us
I20260709 12:16:08.628268 11323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59670 (request call id 1723) took 1077 ms. Trace:
W20260709 12:16:08.629068 11222 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.630300 11222 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.634126 11324 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3363) took 1124 ms. Trace:
I20260709 12:16:08.637452 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.629473 11323 rpcz_store.cc:276] 0709 12:16:07.550610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:07.947305 (+396695us) service_pool.cc:224] Handling call
0709 12:16:07.947646 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:08.318233 (+370587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:08.628009 (+309776us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54004290d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:08.628196 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849954688}
I20260709 12:16:08.634718 11324 rpcz_store.cc:276] 0709 12:16:07.510090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:07.518694 (+  8604us) service_pool.cc:224] Handling call
0709 12:16:07.519104 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:08.171783 (+652679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:08.628245 (+456462us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b54002855d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:08.634042 (+  5797us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1496460160}
I20260709 12:16:08.644675 11148 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:08.559561 20010 consensus_meta.cc:287] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.582s	user 0.000s	sys 0.010s
I20260709 12:16:08.648337 11252 raft_consensus.cc:1240] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 19. Current term is 20. Ops: []
W20260709 12:16:08.649868 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 71433.3us
W20260709 12:16:08.650981 11296 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.653604 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.654929 19853 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:08.656518 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52012 (request call id 1202) took 1078 ms. Trace:
I20260709 12:16:08.656904 11932 rpcz_store.cc:276] 0709 12:16:07.578275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:07.582861 (+  4586us) service_pool.cc:224] Handling call
0709 12:16:08.656342 (+1073481us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54004312d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:08.656456 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2319129856}
I20260709 12:16:08.658197 11250 raft_consensus.cc:1240] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:08.626395 20011 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:08.659861 20011 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.660943 19956 consensus_queue.cc:1048] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:08.646205 19888 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:08.662261 11296 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.662421 20054 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:08.662704 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.663978 20048 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:08.664191 20108 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.258s	user 0.002s	sys 0.000s
I20260709 12:16:08.665650 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.666025 19747 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:08.666909 20113 log.cc:927] Time spent T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:16:08.669446 19847 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:08.669997 19847 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:08.669812 19949 consensus_queue.cc:1048] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:08.670922 19847 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.666570 19747 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.673070 11254 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
W20260709 12:16:08.667745 11222 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.675012 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:08.673698 19747 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:08.676900 11296 leader_election.cc:336] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.675367 11222 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.677431 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.681733 20048 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:08.683578 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.681664 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:08.588279 20076 raft_consensus.cc:3055] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:16:08.685921 20076 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:08.686277 19888 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:08.686466 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.687297 19943 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:16:08.686955 20076 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.688107 20076 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:16:08.688773 11222 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.684866 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.685463 20054 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:08.689951 11222 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.696632 19888 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:08.694437 20010 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:08.697575 11222 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.698112 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:08.698738 20010 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:08.698864 19888 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:08.596367 20032 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.690410 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.689169 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:08.705178 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.715705 11222 leader_election.cc:336] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.716265 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:08.722040 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:08.690382 19923 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:08.732412 11222 leader_election.cc:336] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.730878 20010 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:08.736351 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 51499.3us
W20260709 12:16:08.742046 11296 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.743201 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:08.740386 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 51272.8us
I20260709 12:16:08.741134 20054 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:08.749505 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:08.744551 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.647190 20055 consensus_queue.cc:1059] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:16:08.721454 20048 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:08.755116 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.758971 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:08.748489 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 214840us
W20260709 12:16:08.757288 11296 leader_election.cc:336] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.796223 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:08.796800 11900 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.797521 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.797663 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.798206 20055 raft_consensus.cc:3055] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:16:08.798615 20055 raft_consensus.cc:740] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:16:08.798723 11296 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.798878 11900 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.799154 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.799577 20055 consensus_queue.cc:260] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.800836 20055 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:08.771039 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:08.753926 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.759366 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 233964us
I20260709 12:16:08.801846 19943 raft_consensus.cc:2804] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:08.802057 11931 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:08.802366 19943 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.802757 19943 raft_consensus.cc:3060] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:16:08.802078 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48221.3us
W20260709 12:16:08.804157 20105 log.cc:927] Time spent T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.400s	user 0.004s	sys 0.000s
I20260709 12:16:08.806375 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.808269 20050 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:08.809171 20097 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:08.721520 20032 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.811305 19744 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.812583 20116 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:08.816256 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.755745 20010 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:08.739432 20015 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.071s
W20260709 12:16:08.843940 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.828950 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:08.830252 19744 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:08.838203 19708 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:08.845419 11324 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 22
W20260709 12:16:08.846360 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.848695 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:08.844933 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:08.854394 11222 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.855050 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:08.857972 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:08.856038 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.861739 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.862358 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:08.876543 11296 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.877250 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.824683 20054 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:08.899713 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 91999us
W20260709 12:16:08.908517 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.908985 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.913237 11931 raft_consensus.cc:2468] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
W20260709 12:16:08.914294 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 52568.1us
W20260709 12:16:08.917778 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.864078 19751 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:16:08.891004 19707 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:16:08.888947 19949 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:16:08.952132 11222 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:08.952883 11324 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
W20260709 12:16:08.959446 11222 leader_election.cc:336] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.960208 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:08.923038 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.966140 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:08.935496 20111 log.cc:927] Time spent T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:16:08.908007 20048 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:16:08.950361 20119 log.cc:927] Time spent T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:16:08.980279 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:08.981882 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.984192 11931 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
W20260709 12:16:08.982746 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:08.938835 19708 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:08.986142 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.986456 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:08.988586 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 181176us
I20260709 12:16:08.964932 19751 raft_consensus.cc:2804] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:08.921836 19742 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 12:16:08.990994 11147 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:08.999945 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 91444.7us
I20260709 12:16:08.925822 20117 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
W20260709 12:16:09.000981 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:08.999187 11222 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:08.945889 19943 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.001427 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:09.006625 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:08.991310 19512 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.000384 19747 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:09.000584 20097 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:09.004447 19751 raft_consensus.cc:697] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:09.008322 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:09.010280 10011 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 11207)
I20260709 12:16:09.015641 11251 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:16:09.014004 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:09.020008 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:09.020716 11256 raft_consensus.cc:2368] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22.
I20260709 12:16:09.009557 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:09.020946 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:09.021160 11931 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:09.021694 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 114264us
I20260709 12:16:09.024945 11329 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:09.028625 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:09.028483 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:09.031023 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:09.008447 19708 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:09.034232 11253 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 12:16:09.010360 20125 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:08.922462 20011 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 12:16:09.017235 19949 consensus_queue.cc:1059] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.023s
I20260709 12:16:09.062373 11929 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:16:09.030493 19847 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:09.075734 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:09.076467 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:09.074702 11931 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:16:09.076342 11248 raft_consensus.cc:1240] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:09.078428 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:09.042402 19888 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:09.080387 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:09.085160 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:09.038113 20116 consensus_queue.cc:1059] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:16:09.063942 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:09.071326 19847 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:09.112913 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49144.7us
I20260709 12:16:09.077555 20117 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:09.084511 20006 raft_consensus.cc:3055] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:16:09.102530 20053 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:09.101338 20088 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:09.129545 11222 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:09.139808 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:09.130273 19847 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:09.141709 20006 raft_consensus.cc:740] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:09.130726 20062 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:09.143646 19742 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:16:09.161886 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 76809.1us
I20260709 12:16:09.083814 20097 consensus_queue.cc:1048] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:16:09.144289 20053 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:09.165323 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.149616 20088 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:09.147400 20010 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.109776 20015 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:16:09.160660 19512 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:09.129778 19490 consensus_queue.cc:1059] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.031s
W20260709 12:16:09.171272 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 132109us
I20260709 12:16:09.129570 11067 maintenance_manager.cc:643] P bbfb3415e272477dac9ba599cc40a89d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 2.169s	user 0.124s	sys 0.040s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":26800,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":103190,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":22,"lbm_write_time_us":22009,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":10,"peak_mem_usage":275379,"reinsert_count":0,"thread_start_us":201330,"threads_started":2,"update_count":213}
I20260709 12:16:09.110074 19751 consensus_queue.cc:237] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.168431 20097 consensus_queue.cc:1243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:16:09.170748 20053 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:09.172430 20126 raft_consensus.cc:3055] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:16:09.175413 20088 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:09.198455 11068 maintenance_manager.cc:419] P bbfb3415e272477dac9ba599cc40a89d: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 12:16:09.201501 11067 maintenance_manager.cc:643] P bbfb3415e272477dac9ba599cc40a89d: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 12:16:09.201880 20143 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:09.202421 20143 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:09.202627 19949 raft_consensus.cc:3055] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:09.202921 20143 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:09.203203 19949 raft_consensus.cc:740] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:09.204406 19949 consensus_queue.cc:260] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.209086 19744 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.154043 20117 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.174671 19708 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:09.175734 20062 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:09.188095 19943 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:09.232372 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 91838us
I20260709 12:16:09.200423 20126 raft_consensus.cc:740] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:09.234048 19949 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:09.236958 20138 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:09.238193 20040 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:09.179487 20010 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.246778 11148 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:09.236294 19512 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:09.229210 19853 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.225896 20116 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.222988 20136 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:16:09.260139 20040 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:09.232373 20131 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:16:09.225279 20006 consensus_queue.cc:260] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:09.286773 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 123026us
I20260709 12:16:09.287860 11296 leader_election.cc:400] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22.
I20260709 12:16:09.288236 11296 leader_election.cc:403] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:09.274546 19943 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:16:09.282857 20116 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.283342 20062 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.310051 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:09.320192 11324 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:09.278641 19708 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.307457 20054 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:16:09.322815 11322 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:16:09.285110 20139 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:16:09.270403 19853 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.324395 20139 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:16:09.300511 20146 consensus_queue.cc:1059] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:16:09.288959 20141 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:09.327443 20141 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:16:09.327538 20146 raft_consensus.cc:3055] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:09.327853 20141 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:09.327968 20146 raft_consensus.cc:740] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:09.328763 20146 consensus_queue.cc:260] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.328764 20141 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.274375 20126 consensus_queue.cc:260] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.330938 19512 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.331610 20141 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:09.325419 20139 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.362200 20097 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.401000 20009 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.324105 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:09.445647 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 157905us
W20260709 12:16:09.450191 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 176895us
I20260709 12:16:09.412982 20146 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:09.404773 20139 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.449180 20006 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:09.447803 20076 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243) with higher term. Message: Invalid argument: T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 21. Current term is 22.
I20260709 12:16:09.448623 20138 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.456379 20126 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:09.463135 20048 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:09.504242 11324 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:16:09.485373 20055 consensus_meta.cc:287] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.684s	user 0.023s	sys 0.000s
I20260709 12:16:09.438236 20097 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.413386 20040 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.455011 20009 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.531538 20149 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:09.545001 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 98288.2us
I20260709 12:16:09.546813 11254 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 28. Current term is 29. Ops: []
I20260709 12:16:09.550315 20117 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.552264 20116 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:09.555483 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.534612 20038 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.539052 20055 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.538687 20161 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.575680 20038 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.578766 20139 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.554647 19853 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.560303 20010 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.512255 20138 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.562665 20040 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:09.541606 20149 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.590360 20055 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.627252 19708 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:09.639789 20159 log.cc:927] Time spent T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:16:09.639961 20053 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.640693 20009 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.646560 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.591421 20161 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.649778 20163 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.651082 11256 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:16:09.658463 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:09.660436 11899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.647362 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:09.671010 11932 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 28. Current term is 29. Ops: []
I20260709 12:16:09.630427 20156 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:16:09.675618 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.676355 20138 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.635851 20032 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.677052 20162 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:09.677729 20162 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:09.680634 20161 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.681568 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.642975 20135 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:09.682435 20135 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.683765 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.653271 20038 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.686435 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.693025 11929 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 12:16:09.694238 11322 raft_consensus.cc:2410] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:16:09.695025 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:09.629792 20143 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:09.710634 20166 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:16:09.711314 20055 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.714243 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.715225 11328 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:16:09.715085 20053 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:09.720017 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:09.720798 11932 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 31.
I20260709 12:16:09.685384 20135 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.696880 20010 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.728817 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:09.724884 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 178628us
I20260709 12:16:09.730187 11324 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:09.648723 19708 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.732962 11322 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:09.738090 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:09.735304 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.678790 20162 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:09.741108 19742 consensus_meta.cc:287] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.000s
W20260709 12:16:09.757938 20152 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.249s	user 0.004s	sys 0.000s
W20260709 12:16:09.759544 20167 log.cc:927] Time spent T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:16:09.737370 20050 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:09.696074 20163 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.747871 20135 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:16:09.810683 11900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.742332 20053 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.061s
I20260709 12:16:09.784701 19742 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:09.738611 20010 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.782742 20148 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:16:09.842252 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:09.810093 20135 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:09.849678 20175 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:16:09.847051 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:09.804484 20050 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.835561 20003 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:09.837169 19512 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.847139 20032 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:16:09.851933 20125 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.769s	user 0.014s	sys 0.007s
I20260709 12:16:09.851635 20135 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:16:09.866111 11900 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 126 similar messages]
I20260709 12:16:09.877259 20010 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:09.886528 20172 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:16:09.851464 20149 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:09.869877 19512 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.904423 19512 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.874395 20125 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:09.858286 20175 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.874068 20003 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.849575 20148 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:09.870491 20032 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:09.928190 20170 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:16:09.922072 20143 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.941397 19708 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:09.953263 20076 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:09.956056 19847 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.790s	user 0.008s	sys 0.014s
W20260709 12:16:09.975764 20088 consensus_meta.cc:287] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.750s	user 0.019s	sys 0.008s
W20260709 12:16:10.002453 11296 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.003152 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.003980 20186 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:16:10.006119 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.006685 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:10.020744 10011 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 11207)
W20260709 12:16:10.022500 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.999428 20050 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:10.027295 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.028860 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:10.035259 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.991027 20076 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:09.994256 19847 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.022339 20003 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.027874 20150 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:16:10.026343 20124 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.033797 19708 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.100422 19708 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.103543 19708 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.103714 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.104148 19708 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.104341 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:09.996040 20088 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:10.106076 20146 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.000s
I20260709 12:16:10.107071 20088 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:10.106986 19708 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.109967 11325 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:10.113076 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.114651 20038 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.115209 20038 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.116209 20138 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:10.117772 20138 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:10.097067 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:10.118499 20138 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:16:10.120496 20141 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:10.120702 11901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:10.121059 20141 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.124830 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.125128 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:10.126842 11900 leader_election.cc:336] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.129504 19956 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.129920 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.130177 19956 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.127447 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:10.046468 11296 leader_election.cc:336] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.051677 20148 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.054267 20191 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.057957 20050 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:16:10.058457 20186 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.078809 20076 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.085577 20032 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.153946 20138 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:10.154704 20138 consensus_queue.cc:1243] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer fecfc03de52a4308bc9b045cad5aac1e log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:16:10.155045 20097 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.034534 19742 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.156114 19742 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.158638 19742 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.086364 20124 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.159394 19956 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.151640 20191 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.160902 20124 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:10.087543 20189 log.cc:927] Time spent T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:16:10.088681 11248 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:10.089258 19847 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.090101 20175 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.090662 11326 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:16:10.092255 20182 log.cc:927] Time spent T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:16:10.163297 20141 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.093407 11255 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:10.096086 20163 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:10.099324 20178 log.cc:927] Time spent T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:16:10.135004 11296 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:10.117357 20038 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.149920 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.097074 20135 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.185496 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.146824 20040 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:10.180400 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 133719us
W20260709 12:16:10.189411 11296 leader_election.cc:336] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.191088 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.191545 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.193279 11331 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:16:10.213902 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.214622 11931 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
W20260709 12:16:10.216426 11296 leader_election.cc:336] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.216720 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:10.217249 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:10.217448 11931 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:10.220721 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.221477 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.187592 20097 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:10.224298 11257 raft_consensus.cc:2410] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:16:10.225729 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:10.227296 11296 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.227942 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.227763 11930 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:16:10.229499 11296 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.226418 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.232779 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.188223 19747 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.194708 20009 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.230451 11330 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:10.251507 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:10.252147 11928 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:10.235016 11932 raft_consensus.cc:1275] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:16:10.235682 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.242015 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.242677 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.256632 11249 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:16:10.194621 20038 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:16:10.227505 20048 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:10.193928 19708 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.287252 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.287997 11257 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:10.288195 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.238555 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.255192 11250 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:16:10.274300 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.290305 11929 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:16:10.276566 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.248716 11931 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:16:10.176172 20186 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.193413 19956 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:10.285323 11255 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:10.290712 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:10.291987 11930 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:16:10.297415 11249 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:10.299937 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.307525 11932 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:16:10.315969 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.316743 11934 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 29.
I20260709 12:16:10.253039 19747 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.320465 19747 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.329680 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.331877 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:10.332726 20135 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:10.338349 20192 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:16:10.338236 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.273716 20009 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.339318 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.351962 11251 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 30 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:16:10.340029 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.340898 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.305616 20138 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.308514 20097 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.309228 19956 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.217862 20050 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:10.347234 11323 raft_consensus.cc:2393] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:10.347599 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.350008 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.385521 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:10.386298 11326 raft_consensus.cc:2410] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 29 because replica has last-logged OpId of term: 28 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:16:10.350785 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.359210 11254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3498) took 1346 ms. Trace:
I20260709 12:16:10.366217 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.387197 11254 rpcz_store.cc:276] 0709 12:16:09.012689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:09.014082 (+  1393us) service_pool.cc:224] Handling call
0709 12:16:09.014352 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:09.547272 (+532920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:10.358878 (+811606us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540033d1d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:10.359125 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224057088}
I20260709 12:16:10.394562 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:10.366217 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:10.366955 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.396124 11330 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:16:10.396186 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 259214us
I20260709 12:16:10.368191 11255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33798 (request call id 3506) took 1337 ms. Trace:
I20260709 12:16:10.397521 11255 rpcz_store.cc:276] 0709 12:16:09.030547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:09.030691 (+   144us) service_pool.cc:224] Handling call
0709 12:16:10.368026 (+1337335us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001dd3d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66f02fa 00007f3ecb6b1b17 00007f3ec4c2c6d4 00007f3ec4c2c662 00007f3ec4c2c591 00007f3ec4c2c50c 00007f3ec4c2b7c2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:10.368133 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441316736}
I20260709 12:16:10.397931 20151 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:10.376878 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.398638 20151 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.399573 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.399679 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:10.400496 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:10.382339 11248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2945) took 1337 ms. Trace:
I20260709 12:16:10.402163 19847 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:16:10.402340 20015 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:10.402675 19847 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.403154 19847 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:16:10.403215 20015 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:10.403965 20015 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:10.404271 20015 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:10.404601 20015 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:10.401770 11248 rpcz_store.cc:276] 0709 12:16:09.045085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:09.166512 (+121427us) service_pool.cc:224] Handling call
0709 12:16:09.181120 (+ 14608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:10.161843 (+980723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:10.381978 (+220135us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54002a4ed4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:10.382258 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086633344}
I20260709 12:16:10.406091 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:10.406637 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:10.406862 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:10.407536 11248 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:10.407569 20188 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:10.408206 20188 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.408564 20188 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:10.410212 19751 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:16:10.407569 11222 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:10.412333 19751 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:10.412751 19751 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.413146 19751 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:10.413811 19512 consensus_queue.cc:1048] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:10.415500 19512 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:10.415995 19512 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:10.418236 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9424adcaacdf40f3bbd2f639ef2a48c2" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.421396 19956 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.387310 20097 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.386886 19490 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:10.423895 20097 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:10.362200 20186 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.424896 20186 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:16:10.426712 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.427457 11324 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:10.427698 20009 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.386904 11250 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 20.
I20260709 12:16:10.378726 20138 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.430558 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.361335 20048 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.431782 20138 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.432027 20185 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:16:10.432369 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:10.433854 20143 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:16:10.434244 20143 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:10.435559 11257 raft_consensus.cc:2410] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:16:10.410244 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:10.439541 11325 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:10.438823 11253 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:16:10.437260 11324 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:16:10.448053 11323 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:16:10.456388 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.416930 19512 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.462426 11328 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:16:10.469448 11248 raft_consensus.cc:2468] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
W20260709 12:16:10.471941 11296 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.472071 11249 raft_consensus.cc:2468] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:16:10.472476 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:10.384840 20062 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:10.476624 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.478863 11935 raft_consensus.cc:2410] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 30 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:16:10.480135 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:10.481125 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.446754 20022 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.485930 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.445190 20038 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:10.487671 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.489392 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.495808 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:10.496992 11296 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:10.499414 20186 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:16:10.501247 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.499222 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.501128 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:10.512957 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.514830 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.515825 11931 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:10.455681 20187 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:16:10.552492 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 105849us
I20260709 12:16:10.473804 20125 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.514973 11296 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:10.564654 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49781.4us
I20260709 12:16:10.548301 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.565523 11296 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:16:10.547833 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 73382.8us
I20260709 12:16:10.431888 20048 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.484720 20097 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:16:10.479270 20169 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.479719 20158 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.444693 20135 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:16:10.549293 20187 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.573789 11331 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:10.460350 20188 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.424192 19490 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.616977 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.583319 11928 raft_consensus.cc:2468] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:16:10.588977 11325 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:10.583885 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:10.633004 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.536650 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.511210 20038 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.561936 20148 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:10.514132 20022 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.564525 20151 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.562844 20009 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:10.639083 20038 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:10.577337 20125 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.576050 20186 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:10.574931 20088 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:10.643040 20076 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:10.652480 11252 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:16:10.699024 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 63159us
I20260709 12:16:10.700249 11252 pending_rounds.cc:85] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 13
I20260709 12:16:10.700709 11252 pending_rounds.cc:107] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 24.14
W20260709 12:16:10.660351 11073 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 174) took 13725 ms (client timeout 15000 ms). Trace:
I20260709 12:16:10.661991 11075 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:10.672223 20200 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:16:10.475210 20143 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.676093 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:10.677018 11249 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:16:10.575793 19956 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:10.582280 20050 raft_consensus.cc:2804] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:10.688292 11254 pending_rounds.cc:85] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 13
I20260709 12:16:10.688467 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.722609 11254 pending_rounds.cc:107] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:16:10.491914 20062 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.701017 11252 raft_consensus.cc:2889] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:10.583798 20128 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:10.701001 11148 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:10.575286 20054 consensus_queue.cc:1059] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.105s
I20260709 12:16:10.676795 20187 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:10.705070 11075 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 18 to 19, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 19 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:10.723417 11251 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
I20260709 12:16:10.723400 11254 raft_consensus.cc:2889] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:10.672010 20163 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:16:10.733979 11250 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:10.734791 11249 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:16:10.599735 20169 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.686751 19943 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.632802 20162 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.100s
W20260709 12:16:10.701303 11073 rpcz_store.cc:269] 0709 12:15:56.935190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:56.935463 (+   273us) service_pool.cc:224] Handling call
0709 12:16:10.660261 (+13724798us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:06.844176 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:06.844878 (+   702us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:06.844936 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:16:06.844956 (+    20us) tablet.cc:662] Decoding operations
0709 12:16:07.001127 (+156171us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:07.014251 (+ 13124us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:07.014293 (+    42us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:16:07.015630 (+  1337us) tablet.cc:701] Row locks acquired
0709 12:16:07.015692 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:16:07.015903 (+   211us) write_op.cc:270] Start()
0709 12:16:07.016106 (+   203us) write_op.cc:276] Timestamp: P: 1783599367015822 usec, L: 0
0709 12:16:07.016149 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:16:07.017013 (+   864us) log.cc:844] Serialized 5642 byte log entry
0709 12:16:07.382972 (+365959us) op_driver.cc:464] REPLICATION: finished
0709 12:16:07.534015 (+151043us) write_op.cc:301] APPLY: starting
0709 12:16:07.534126 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:07.577109 (+ 42983us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:07.583335 (+  6226us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:08.352389 (+769054us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:08.453860 (+101471us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=56,mrs_lookups=0
0709 12:16:08.453921 (+    61us) write_op.cc:312] APPLY: finished
0709 12:16:08.454388 (+   467us) log.cc:844] Serialized 129 byte log entry
0709 12:16:08.594705 (+140317us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:08.693834 (+ 99129us) write_op.cc:454] Released schema lock
0709 12:16:08.696796 (+  2962us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:08.736384 (+ 39588us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:08.742446 (+  6062us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":697260,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":150739,"apply.run_cpu_time_us":57308,"apply.run_wall_time_us":1185501,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62759,"num_ops":14,"prepare.queue_time_us":199232,"prepare.run_cpu_time_us":8131,"prepare.run_wall_time_us":173226,"replication_time_us":366645,"thread_start_us":321001,"threads_started":3,"wal-append.queue_time_us":227273,"wal-append.run_cpu_time_us":1306,"wal-append.run_wall_time_us":1013279}]]}
I20260709 12:16:10.722379 20204 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.735682 19923 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:16:10.680519 20148 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.619707 20158 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.702401 20009 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.743824 20143 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:10.804275 20218 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.805197 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.718681 20088 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.717005 20076 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.765249 20126 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:10.764573 20138 raft_consensus.cc:3055] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:16:10.747763 20011 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:16:10.740505 19956 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.756623 19943 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.817243 11075 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 15 to 24, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 24 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:10.745592 20128 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.837563 11296 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.840198 20150 raft_consensus.cc:2804] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:10.840562 19747 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.840751 20150 raft_consensus.cc:697] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:10.841120 19747 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.841754 20150 consensus_queue.cc:237] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:10.831257 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 247495us
I20260709 12:16:10.755105 20015 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.845690 20015 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.846935 19490 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:10.821077 20138 raft_consensus.cc:740] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:10.833400 20222 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:10.856976 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:10.833580 20126 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:10.771749 20204 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.872347 11932 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:10.895473 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:10.900969 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 43829us
I20260709 12:16:10.918450 11147 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:10.805418 20218 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.861158 20048 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:10.872982 11075 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:10.922480 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.928483 11253 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:16:10.858148 20222 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.946496 20222 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:10.949718 20138 consensus_queue.cc:260] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:10.951261 20138 raft_consensus.cc:3060] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:10.954666 20225 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:10.955726 20126 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.863065 20227 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:10.865374 20186 raft_consensus.cc:697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:10.929980 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:10.932170 20207 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:16:10.979945 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50054.8us
W20260709 12:16:10.939344 20203 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:16:10.867623 20169 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.871728 20009 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.867050 19747 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.938031 20088 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:10.940567 20158 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:10.940691 19847 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 12:16:10.941452 20125 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:10.938179 11075 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 18 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 23 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:10.943403 20148 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:11.021243 10011 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 11207)
I20260709 12:16:10.945055 19956 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:10.945573 20038 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:10.867287 20185 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:10.931694 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:10.932399 19943 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.024880 11075 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 18 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 23 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:10.958056 20208 log.cc:927] Time spent T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.115s	user 0.004s	sys 0.000s
W20260709 12:16:10.979141 20205 log.cc:927] Time spent T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:16:10.987861 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:10.995901 20050 raft_consensus.cc:697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:11.041774 20219 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:11.042017 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 54230.5us
I20260709 12:16:11.042563 19943 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:11.024616 19747 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:11.046283 20223 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:11.045922 11075 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 17 to 19, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 19 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.046758 20226 consensus_queue.cc:1059] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:16:11.047391 20215 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:11.048658 20230 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:16:11.050966 20217 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:11.031939 20206 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:16:11.054293 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
W20260709 12:16:11.075528 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:11.045789 20202 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 12:16:11.058856 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26c5b0a79b0482cb39b479148fba412" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.081533 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.082129 11928 raft_consensus.cc:2393] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:11.062932 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.070184 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:10.986809 20227 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.019351 19847 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.091003 11075 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.082999 11934 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:11.083005 11931 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
W20260709 12:16:11.079380 11222 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:11.094815 11222 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.095266 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.058946 20237 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:11.081043 11147 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.088263 20038 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.084741 20223 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:11.097586 11075 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 19 to 20, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:11.081845 20117 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.067628 19747 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.089871 20218 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.058008 20219 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.090373 20229 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:11.090620 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:11.105355 11253 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:16:11.090943 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.051810 20062 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.068542 20076 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:16:11.105701 20117 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.110862 20117 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.111881 11257 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:16:11.114952 11253 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:11.115065 20222 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.117326 20222 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.102309 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:11.100282 11222 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.097373 20223 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.120949 11222 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.118032 20229 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.126351 20227 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.133437 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.128362 19847 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 32 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.129062 20236 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:11.102676 19943 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:11.061424 20186 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:11.130384 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 674829us
I20260709 12:16:11.134506 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.140436 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:11.071658 20215 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.145339 11323 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:11.146425 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:11.146848 11222 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.147300 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.082146 20217 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.147905 11322 raft_consensus.cc:2410] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 25 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:16:11.149151 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:11.138845 19751 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: Time spent flushing consensus metadata: real 0.725s	user 0.014s	sys 0.000s
I20260709 12:16:11.154767 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:11.157522 11222 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.157290 11075 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 12 to 22, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 22 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:11.158802 20231 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:16:11.156431 20225 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.161120 20225 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.163332 20011 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:11.140952 20216 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:11.159181 19847 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:11.156363 20125 raft_consensus.cc:2804] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:11.155308 20158 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.166184 20125 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.166925 20125 raft_consensus.cc:3060] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:11.166682 20158 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.170648 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:11.171314 20191 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.171473 11931 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:11.174371 20224 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:11.175114 20224 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.177910 20076 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:11.178041 20204 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:11.103821 20237 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.132696 20223 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:11.179484 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.189548 11327 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:16:11.143316 20117 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:11.183805 11323 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 29.
I20260709 12:16:11.169481 20158 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.208037 20215 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.211285 20125 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.188352 20048 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:11.212702 20048 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:11.166306 11222 leader_election.cc:336] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.191841 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.193295 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.214270 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.214823 11324 raft_consensus.cc:2368] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 28. Current term is 29.
I20260709 12:16:11.214968 20048 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.217089 20237 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.193476 20217 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.194005 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.204304 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.219568 19747 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.178494 20076 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.165959 11147 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.222164 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.222172 19708 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:16:11.223843 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 57981.3us
I20260709 12:16:11.226166 20229 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.227483 19747 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:11.228569 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 62284.5us
I20260709 12:16:11.190824 11931 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 30.
I20260709 12:16:11.159664 19751 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.232089 19751 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.107975 20050 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.213527 20125 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.238752 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.241015 20128 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.206640 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.243425 20076 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.243975 20076 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:11.244448 20076 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.239952 11148 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:11.250931 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:11.262786 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 114076us
I20260709 12:16:11.265761 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.267474 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.264580 20235 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:11.270107 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.261873 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:11.277566 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.197160 20187 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:11.281868 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:11.278867 11900 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:11.284562 11900 leader_election.cc:336] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.285583 11223 leader_election.cc:400] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 28. Current term is 29.
I20260709 12:16:11.286026 11223 leader_election.cc:403] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:11.208256 20224 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.265177 20209 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.291792 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:11.200567 20227 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 12:16:11.264019 19708 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:11.295876 11328 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:11.285050 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.296286 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.309031 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.275239 20235 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.297940 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:11.309821 11251 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:11.304499 11147 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:11.319314 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60663.5us
W20260709 12:16:11.319518 11900 leader_election.cc:336] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.320122 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.320140 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.322394 19996 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:11.288269 20185 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:11.324728 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:11.327976 11255 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:11.326493 11147 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:11.327073 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 60002.2us
I20260709 12:16:11.328609 11075 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:11.289410 20125 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:11.278003 20022 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.291909 20131 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:11.332680 11075 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 20 to 21, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 21 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:11.323638 20187 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.323570 20126 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:11.299284 20188 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.324254 20235 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:11.350976 11255 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
W20260709 12:16:11.353884 20249 log.cc:927] Time spent T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:16:11.355544 11075 catalog_manager.cc:5697] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 12 to 17, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 17 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:11.326228 19742 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:11.335091 20128 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:16:11.364466 20138 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:11.364382 11075 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 20 to 21, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 21 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.344273 19943 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:16:11.373795 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.378495 11075 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 15 to 16, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 16 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.386236 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:11.327708 20185 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.334609 19747 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:16:11.337330 20158 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:11.340642 20131 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.394202 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.390478 20076 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.396260 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.331815 19996 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:11.420071 11222 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.426445 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.434692 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:11.310313 20209 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.437516 11250 raft_consensus.cc:2393] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:11.435328 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:11.436229 11222 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.438174 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.427776 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.338994 20125 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 29 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
W20260709 12:16:11.441290 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:11.445173 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.447041 11330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39342 (request call id 3733) took 1143 ms. Trace:
I20260709 12:16:11.454452 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.455235 11930 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:11.457710 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:11.458575 11254 raft_consensus.cc:2393] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:11.426614 20185 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:11.464982 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.465214 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.467232 11248 raft_consensus.cc:2393] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 25: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
W20260709 12:16:11.461828 11222 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.471501 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.418892 20022 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:11.472959 11147 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:11.473093 11222 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.447615 11330 rpcz_store.cc:276] 0709 12:16:10.304065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.397044 (+ 92979us) service_pool.cc:224] Handling call
0709 12:16:10.437739 (+ 40695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:10.437958 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:16:10.464713 (+ 26755us) log.cc:844] Serialized 14 byte log entry
0709 12:16:11.060662 (+595949us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:11.060789 (+   127us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:11.060863 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:11.060978 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:11.446788 (+385810us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b54002f62d0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66eb8ad 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6 00007f3ec4a2626f
0709 12:16:11.446957 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15831,"prepare.queue_time_us":26549,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":5133,"raft.queue_time_us":56288,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":135,"spinlock_wait_cycles":92822400,"thread_start_us":579156,"threads_started":2,"wal-append.queue_time_us":330209}
I20260709 12:16:11.474079 11328 raft_consensus.cc:2468] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:16:11.478904 11075 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 9 to 20. New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:11.432883 20165 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:11.478139 11222 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.484632 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.484766 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.409664 20126 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
W20260709 12:16:11.495229 20247 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:16:11.496232 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:11.497650 20243 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:16:11.500537 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.502967 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.477087 20221 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:11.506948 20165 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.510612 20204 raft_consensus.cc:2764] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:16:11.470297 20038 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.473793 20117 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:11.530490 11929 raft_consensus.cc:1217] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Deduplicated request from leader. Original: 25.17->[27.18-28.19]   Dedup: 27.18->[28.19-28.19]
I20260709 12:16:11.531198 11929 raft_consensus.cc:1240] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 28. Current term is 29. Ops: [27.18-28.19]
I20260709 12:16:11.504855 20040 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:11.485723 20054 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:16:11.528076 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55017.7us
I20260709 12:16:11.537743 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:11.538352 11926 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:16:11.538554 11222 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.535284 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.542196 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:11.547880 11222 leader_election.cc:336] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:11.548568 11222 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.557932 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.558087 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:11.562323 11223 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:11.562634 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 79460.4us
I20260709 12:16:11.558714 11934 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 24.
W20260709 12:16:11.562136 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 46591.1us
I20260709 12:16:11.567610 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.563344 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:16:11.482067 20022 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.531455 20217 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:11.537158 20143 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:11.587498 20143 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.536729 19707 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:11.591528 19707 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.578794 11901 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.594425 11901 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.598011 20054 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.598945 20054 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.549932 19751 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:11.561715 20162 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.574045 20125 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:11.585183 20248 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:16:11.585575 20209 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.529877 20038 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.580370 20229 raft_consensus.cc:2804] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:11.580601 20215 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:11.556723 20117 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.611572 11296 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:11.574687 20158 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:11.614077 19490 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:16:11.537101 20223 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.631776 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.630949 20128 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967) with higher term. Message: Invalid argument: T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e for earlier term 28. Current term is 29.
I20260709 12:16:11.589547 20143 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:11.603180 20146 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:11.621330 11926 raft_consensus.cc:2468] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 32.
W20260709 12:16:11.631050 20228 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:16:11.628159 20062 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:11.706943 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:11.701069 11248 raft_consensus.cc:3060] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:11.700176 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.639483 20191 raft_consensus.cc:2804] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:11.645356 20229 raft_consensus.cc:697] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:11.734138 20191 raft_consensus.cc:697] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:11.635177 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:11.738657 11935 raft_consensus.cc:3060] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:16:11.704702 20257 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:16:11.680948 20054 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.630451 20162 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.644644 20117 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.739751 20131 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.725332 20158 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:11.750653 20158 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.751128 20158 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:11.743040 20038 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.719498 20088 raft_consensus.cc:3055] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:16:11.754716 20088 raft_consensus.cc:740] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:11.755961 20088 consensus_queue.cc:260] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.742455 19707 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.742027 20229 consensus_queue.cc:237] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.700418 20185 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:11.760062 11327 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 27: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:11.756645 20162 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.741307 20219 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.685547 20126 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.692133 20223 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.717566 20225 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.691007 20232 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:11.779923 11252 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 29.
I20260709 12:16:11.745002 20235 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.780902 11248 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:16:11.781944 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 234102us
W20260709 12:16:11.824168 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 261899us
I20260709 12:16:11.828799 11329 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:16:11.830521 11248 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:16:11.830405 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 234553us
I20260709 12:16:11.837018 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.783114 20126 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:11.843132 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 279833us
I20260709 12:16:11.844069 11330 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:16:11.844050 11147 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:11.844542 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.847334 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.849519 11326 raft_consensus.cc:2393] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 24: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:11.854812 11324 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 29: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:11.854853 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.813446 20022 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.859498 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.865453 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.866916 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.868203 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:11.868996 11935 raft_consensus.cc:2468] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:16:11.869050 11250 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:11.869243 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.828998 20186 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.786580 20232 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.859078 20162 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:11.879866 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:11.880285 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.894629 11935 raft_consensus.cc:2393] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 26: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:11.894683 11931 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 28.
I20260709 12:16:11.877094 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:11.890908 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:11.882638 20050 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:11.874910 20048 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:11.918574 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:11.916296 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 313762us
I20260709 12:16:11.882985 20185 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:11.934069 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:11.934827 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.935731 11326 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 29.
I20260709 12:16:11.921456 20264 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:11.923614 20232 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:11.923719 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.841984 20191 consensus_queue.cc:237] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.896507 20117 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:11.945806 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.951978 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:11.958534 20260 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:16:11.889340 20186 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.966367 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.948370 20038 raft_consensus.cc:3060] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:11.939386 20224 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:11.941192 20187 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:11.972999 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cba0650205b4eb0b416158a057a90b9" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:11.976287 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:16:11.952981 20050 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:11.990244 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:12.021723 10011 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 11207)
I20260709 12:16:11.988340 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:12.029395 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 50323.2us
W20260709 12:16:12.032346 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 56177.1us
I20260709 12:16:12.035096 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:12.030835 11329 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 24.
I20260709 12:16:12.046342 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:11.983458 20224 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:12.050876 20224 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:11.987012 20273 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:12.052165 20126 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:12.054158 20260 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:16:12.056097 20277 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.056715 20277 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:11.985503 20187 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.042794 20125 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:12.062043 20217 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:16:12.062518 20217 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:12.066427 20235 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:12.060945 20050 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:12.073300 20270 log.cc:927] Time spent T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 12:16:12.078258 20271 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:16:12.090637 20162 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:16:12.062084 20125 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.092732 20125 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:16:12.113425 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 67164.2us
I20260709 12:16:12.124171 11934 consensus_queue.cc:237] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.127311 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.128382 11254 raft_consensus.cc:2393] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 28: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:12.125092 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.128723 11148 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:12.129377 11252 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 23.
I20260709 12:16:12.137905 11148 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:12.139552 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:12.063472 20217 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.158253 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:12.172171 11252 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 12:16:12.158797 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:12.176645 11296 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:12.158691 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:12.141384 20117 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:12.141171 20126 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:16:12.145784 11147 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:12.207531 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.212052 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.217031 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.219918 11932 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 29. Current term is 30. Ops: [29.17-29.17]
W20260709 12:16:12.220610 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.170657 19490 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:12.224277 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 78568.6us
I20260709 12:16:12.223733 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:12.151156 20269 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:12.232640 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.232318 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.237414 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.211410 11248 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:12.213967 11254 raft_consensus.cc:3060] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:12.240381 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:12.241156 11930 raft_consensus.cc:3060] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:12.251482 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.223742 20186 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:12.230635 19742 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:12.235075 20076 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:16:12.266150 11900 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.234032 20217 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:16:12.300866 20284 log.cc:927] Time spent T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:16:12.301716 20277 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:12.305729 20248 log.cc:927] Time spent T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:16:12.234824 19490 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.261443 19742 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.267819 20221 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:12.225900 20266 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.044s
W20260709 12:16:12.428361 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 162453us
I20260709 12:16:12.437752 11254 raft_consensus.cc:2468] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:12.434899 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:12.447093 11927 raft_consensus.cc:1275] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:16:12.436218 20278 log.cc:927] Time spent T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.196s	user 0.000s	sys 0.008s
I20260709 12:16:12.546926 20062 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:12.655611 20062 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.657781 20062 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:12.551450 20088 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:16:12.552145 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:12.659348 11222 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:12.660835 19747 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:12.661128 11222 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:12.561815 20267 log.cc:927] Time spent T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.390s	user 0.003s	sys 0.001s
I20260709 12:16:12.561671 20187 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:12.455331 20158 consensus_meta.cc:287] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.704s	user 0.005s	sys 0.015s
I20260709 12:16:12.662359 20158 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.664382 20158 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:12.575137 20283 log.cc:927] Time spent T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:16:12.577117 20038 consensus_meta.cc:287] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.005s
I20260709 12:16:12.665489 20038 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:12.584285 20275 log.cc:927] Time spent T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.361s	user 0.002s	sys 0.012s
I20260709 12:16:12.665877 11329 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 12:16:12.666242 11329 pending_rounds.cc:85] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting all ops after (but not including) 18
I20260709 12:16:12.666473 11329 pending_rounds.cc:107] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 12:16:12.666694 11329 raft_consensus.cc:2889] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:12.433830 20186 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.667434 20186 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.669891 11928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52016 (request call id 1605) took 1472 ms. Trace:
I20260709 12:16:12.670245 11928 rpcz_store.cc:276] 0709 12:16:11.197547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.198607 (+  1060us) service_pool.cc:224] Handling call
0709 12:16:11.201354 (+  2747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:11.202243 (+   889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:11.202274 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:11.205006 (+  2732us) log.cc:844] Serialized 26 byte log entry
0709 12:16:11.745474 (+540468us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18003a45c0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:16:11.963578 (+218104us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:16:11.978505 (+ 14927us) log.cc:844] Serialized 14 byte log entry
0709 12:16:11.978728 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:11.978805 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:12.583942 (+605137us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b24000311d8. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec34859c4 00007f3ec3483acb 00007f3ec363edc9 00007f3ec363ea55 00007f3ec363e09d 00007f3ec627010e 00007f3ec628788c 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221
0709 12:16:12.669702 (+ 85760us) raft_consensus.cc:1685] finished
0709 12:16:12.669724 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:12.669839 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":430,"prepare.queue_time_us":15339,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":82934,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":192,"spinlock_wait_cycles":34930304,"thread_start_us":217918,"threads_started":1,"wal-append.queue_time_us":1003154}
I20260709 12:16:12.604735 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:12.549579 20274 log.cc:927] Time spent T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.326s	user 0.010s	sys 0.000s
I20260709 12:16:12.618275 11934 raft_consensus.cc:1240] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:16:12.541689 20277 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:12.675276 20277 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:12.625862 20229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.015s	sys 0.045s
I20260709 12:16:12.627184 11248 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 29.
W20260709 12:16:12.628609 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 156123us
I20260709 12:16:12.452841 11930 raft_consensus.cc:2468] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 21.
I20260709 12:16:12.573504 20125 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.634869 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:12.649789 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 420214us
I20260709 12:16:12.669466 20186 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:12.677621 20277 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:12.699267 11296 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:12.677837 20229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.015s	sys 0.045s
I20260709 12:16:12.678593 11255 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Rejecting Update request from peer 786851d4227e4dd89b75ccdea0b1f03b for earlier term 28. Current term is 29. Ops: [28.17-28.17]
W20260709 12:16:12.700637 11296 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.680954 20125 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:12.682770 11248 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
W20260709 12:16:12.702135 11296 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.687227 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:12.703717 11253 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:16:12.704075 11296 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.689247 11249 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:16:12.692751 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:12.707463 11927 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:16:12.713307 20186 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:16:12.714259 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 53203.1us
I20260709 12:16:12.715154 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:12.716565 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:12.717696 19747 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:12.717823 20229 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:12.718387 20229 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:12.718500 19747 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:12.718919 20229 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:12.719022 19747 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.719553 19747 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:12.722899 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:12.722479 20186 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:12.732667 19747 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.736709 20186 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:12.743311 19747 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:12.742276 11296 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:12.748679 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 490340us
W20260709 12:16:12.749411 11900 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:12.750749 11900 leader_election.cc:336] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.751252 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:12.757460 11222 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:12.758144 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:12.759155 11928 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:16:12.758766 11296 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:12.653165 19742 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:12.767522 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:12.775300 11900 leader_election.cc:336] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:12.776711 11900 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.779223 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:12.749312 20186 raft_consensus.cc:2804] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:12.754156 11253 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:12.799314 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.776423 20162 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.746300 20221 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:12.784698 19490 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:12.780525 20266 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:12.795439 20191 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:12.791939 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:12.813696 11324 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:16:12.792958 20188 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:16:12.794068 20165 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:12.796823 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.838809 11932 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:12.781180 11222 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:12.799229 20214 raft_consensus.cc:2764] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:16:12.775993 19751 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.755712 20286 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:16:12.844504 20286 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:16:12.844060 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 48484.9us
I20260709 12:16:12.845597 20286 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.847079 20286 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:12.848994 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.800393 20186 raft_consensus.cc:697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:12.806442 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.774475 20273 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:16:12.850029 11933 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:12.775164 20223 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:16:12.850346 20186 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:12.815208 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.816295 11256 raft_consensus.cc:1275] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:16:12.817795 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:12.819523 20048 consensus_queue.cc:1048] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:12.827826 20221 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:12.810427 20187 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:12.839922 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 58814.3us
I20260709 12:16:12.840355 11147 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:12.857414 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.856292 11222 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:12.863466 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:12.840207 20191 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:12.842535 20128 raft_consensus.cc:2804] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:16:12.807364 20162 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.871240 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:12.871261 11222 leader_election.cc:336] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:12.781245 11927 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:12.878257 11933 raft_consensus.cc:3060] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:12.878261 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:12.855880 19751 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:12.853729 20229 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.882131 11248 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:12.885777 20277 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:12.890122 20219 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:12.787897 11928 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 30.
I20260709 12:16:12.863646 20165 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:12.866866 20209 raft_consensus.cc:2804] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:12.876140 20128 raft_consensus.cc:697] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 32 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:12.863282 20273 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:12.910318 11251 raft_consensus.cc:1275] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:16:12.914263 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:12.957463 20191 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:12.963999 11933 raft_consensus.cc:2468] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 30.
I20260709 12:16:12.965309 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:12.896456 20165 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.863765 20187 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:12.895574 20281 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:12.998749 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:12.962473 20209 raft_consensus.cc:697] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:12.964417 20141 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.019909 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.006672 11327 raft_consensus.cc:2393] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 23: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:12.976404 20165 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:12.968640 20273 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:13.026772 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:13.028717 10011 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 11207)
W20260709 12:16:13.033106 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 211658us
I20260709 12:16:13.060551 11253 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 17. (term mismatch)
I20260709 12:16:13.061197 11253 pending_rounds.cc:85] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 16
I20260709 12:16:13.061545 11253 pending_rounds.cc:107] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 29.17
I20260709 12:16:13.062021 11253 raft_consensus.cc:2889] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:13.063172 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:13.074681 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:16:13.081179 20297 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 12:16:13.086427 20162 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.056509 20128 consensus_queue.cc:237] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.093992 20048 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:13.094691 20048 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:13.095175 20048 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:13.098022 20209 consensus_queue.cc:237] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.103118 20187 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:13.105108 20298 log.cc:927] Time spent T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:16:13.067022 20088 consensus_queue.cc:1048] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
W20260709 12:16:13.109757 20294 log.cc:927] Time spent T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:16:13.125159 20229 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.025887 20141 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.140959 20141 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:13.151263 20296 log.cc:927] Time spent T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:16:13.171303 19751 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:13.176941 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.180305 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.186748 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:16:13.144903 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:13.145532 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 255095us
W20260709 12:16:13.198026 11222 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:13.200609 11222 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.201278 11222 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:13.204452 11222 leader_election.cc:336] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.205561 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:13.208048 11222 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:13.210996 11222 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:13.213471 11222 leader_election.cc:336] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.215389 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:13.218537 11222 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:13.224380 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 79572.2us
I20260709 12:16:13.217834 20299 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:13.219169 20219 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:13.225829 19975 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:13.217587 20229 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:13.266330 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:13.268193 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49669.1us
I20260709 12:16:13.220669 20216 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:13.217608 20279 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:13.147465 20273 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:13.301111 11222 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:13.309427 11222 leader_election.cc:336] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.310575 11222 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:13.319662 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 53480.3us
I20260709 12:16:13.293645 20269 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.296085 20186 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.328533 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:13.302408 20300 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.302035 20299 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 30 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:13.326610 20229 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:13.328590 20303 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:13.339651 20304 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:13.381688 20050 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:13.333331 20276 consensus_queue.cc:1048] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 12:16:13.398284 20312 log.cc:927] Time spent T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:16:13.399235 20309 log.cc:927] Time spent T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:16:13.399786 11257 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3535) took 2319 ms. Trace:
W20260709 12:16:13.406461 19742 consensus_meta.cc:287] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.000s
I20260709 12:16:13.338280 20186 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.433197 19742 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.439160 19742 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:13.441015 20307 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:13.441830 20191 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:13.419791 11257 rpcz_store.cc:276] 0709 12:16:11.075637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.112809 (+ 37172us) service_pool.cc:224] Handling call
0709 12:16:11.113195 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:11.927732 (+814537us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:16:11.949918 (+ 22186us) log.cc:844] Serialized 14 byte log entry
0709 12:16:12.127073 (+177155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:12.127132 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:12.127179 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:13.394509 (+1267330us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54000f7dd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:13.394564 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:13.394739 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":337,"prepare.queue_time_us":23740,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":3625,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":1872510848,"thread_start_us":93312,"threads_started":1,"wal-append.queue_time_us":291134}
I20260709 12:16:13.442613 20191 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.445011 20191 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:13.447002 11325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39356 (request call id 3446) took 1512 ms. Trace:
I20260709 12:16:13.447458 11325 rpcz_store.cc:276] 0709 12:16:11.934196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.936276 (+  2080us) service_pool.cc:224] Handling call
0709 12:16:11.936661 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:11.939634 (+  2973us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:16:12.234436 (+294802us) log.cc:844] Serialized 14 byte log entry
0709 12:16:12.654363 (+419927us) log.cc:844] Serialized 14 byte log entry
0709 12:16:12.654836 (+   473us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:12.654908 (+    72us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:16:12.654973 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.832287 (+177314us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b18000e9d40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:16:13.065477 (+233190us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b18000e9d40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419
0709 12:16:13.410647 (+345170us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b18000e9d40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1 00007f3ec3601779 00007f3ec3601709 00007f3ec36016d1 00007f3ec36009cd 0000000000388634 0000000000388479 00007f3ec35e283a
0709 12:16:13.446741 (+ 36094us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b54002f76d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:13.446806 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:13.446947 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14473,"prepare.queue_time_us":132788,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":92,"raft.queue_time_us":296,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":1379,"spinlock_wait_cycles":662252672,"thread_start_us":681217,"threads_started":2,"wal-append.queue_time_us":727559}
I20260709 12:16:13.449805 20191 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:13.450430 20191 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.453240 20191 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:13.456779 20191 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:13.458583 19742 raft_consensus.cc:493] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.459213 19742 raft_consensus.cc:515] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.462672 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.457482 20191 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.464442 19742 leader_election.cc:290] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.439208 20224 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:13.468540 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 1.31789e+06us
I20260709 12:16:13.468715 20224 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.472458 20224 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.477392 11148 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:13.481331 20125 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:13.481240 20191 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:13.518739 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 437449us
I20260709 12:16:13.522310 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.489163 20224 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:13.504925 19747 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:13.517189 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 47530.4us
I20260709 12:16:13.502043 20268 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.530843 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.441812 20307 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.378053 20300 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.334251 20269 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.543511 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:13.550544 11257 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:16:13.538300 20224 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.553190 11933 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:16:13.558698 20307 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.559791 20186 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.538262 19747 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.477311 20299 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.580538 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.570998 11933 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 17. (term mismatch)
I20260709 12:16:13.582302 11933 pending_rounds.cc:85] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting all ops after (but not including) 16
I20260709 12:16:13.582634 11933 pending_rounds.cc:107] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: Aborting uncommitted NO_OP operation due to leader change: 28.17
I20260709 12:16:13.582870 11933 raft_consensus.cc:2889] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:13.576996 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.586059 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.553737 20050 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:13.621577 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.619689 11251 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:16:13.538170 20268 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:13.625602 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 316212us
I20260709 12:16:13.629107 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.629962 20307 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:16:13.621579 20165 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.567s	user 0.013s	sys 0.000s
I20260709 12:16:13.635984 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:13.622879 20050 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:16:13.641768 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 276512us
W20260709 12:16:13.645820 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 44547.9us
I20260709 12:16:13.668466 11930 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:13.688576 20209 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:13.639567 20165 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.694912 11932 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:16:13.696122 11222 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:13.701340 11222 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:13.705478 11222 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:13.706103 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 163003us
W20260709 12:16:13.710192 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 166474us
I20260709 12:16:13.712978 20038 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:13.713174 20125 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.713002 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e03893c6062f45ea972113cfa57ba362" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:13.714838 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a629d78b61074c2cbd007926bd78fa29" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:13.715793 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:13.716480 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:13.717478 11330 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:16:13.719449 11222 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.719964 11222 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:13.689343 20209 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.727180 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:13.715308 20128 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:13.727957 11331 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:13.728266 20217 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:13.727226 20125 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.730904 11328 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:16:13.731381 20125 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:13.732892 20323 log.cc:927] Time spent T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 12:16:13.735590 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:13.736335 11222 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.737124 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:13.738690 11331 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:13.704738 20214 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:13.742089 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:13.709685 20229 raft_consensus.cc:2804] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:13.801322 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:13.745684 20307 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:16:13.742729 20300 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.817677 10011 tablet_replica.cc:333] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
W20260709 12:16:13.831115 20325 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:16:13.806517 20136 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.834935 10011 raft_consensus.cc:2243] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.834875 20136 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.836031 10011 raft_consensus.cc:2272] T 89068ffedf554d89af62a8b1fe0700c0 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.804944 19747 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:13.845203 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 170658us
I20260709 12:16:13.858793 11250 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 12:16:13.818342 20316 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:16:13.807036 20229 raft_consensus.cc:697] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 30 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:13.806599 19751 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:13.818549 20126 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.737838 20038 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.812091 20215 raft_consensus.cc:493] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:13.868016 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.819355 20219 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:13.872845 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:13.843290 20300 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:13.875182 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.880119 11927 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:13.860462 20191 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.883741 20191 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.879457 11251 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
I20260709 12:16:13.885174 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:13.887308 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 151492us
I20260709 12:16:13.887411 11257 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
W20260709 12:16:13.890561 11222 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.891091 11222 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:13.897536 11222 leader_election.cc:336] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:13.905498 11296 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:13.904744 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 99886.4us
I20260709 12:16:13.906008 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:13.879940 20126 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:13.892062 20128 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:13.902551 20214 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:13.907683 20319 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.213s	user 0.003s	sys 0.000s
W20260709 12:16:13.913206 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.914038 11148 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:13.883090 20215 raft_consensus.cc:515] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.885026 20219 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.919420 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:13.915834 20330 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:13.933272 10011 tablet_replica.cc:333] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98: stopping tablet replica
W20260709 12:16:13.934072 20322 log.cc:927] Time spent T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:16:13.934423 10011 raft_consensus.cc:2243] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.935202 20219 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.922371 20300 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:13.908756 20165 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:13.935263 10011 raft_consensus.cc:2272] T 5fa73285733a4a67a3e705a20155f69d P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:13.938230 20329 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:16:13.939491 11256 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33798 (request call id 3570) took 1081 ms. Trace:
I20260709 12:16:13.940608 20268 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:13.939855 11256 rpcz_store.cc:276] 0709 12:16:12.858095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:12.858356 (+   261us) service_pool.cc:224] Handling call
0709 12:16:12.858714 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:12.859234 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:12.859257 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:12.859768 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:16:13.450508 (+590740us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b18000b1c40. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35e1e07 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b 00007f3ec628bd23 00007f3ec628c0d1 00007f3ec66a7a0f
0709 12:16:13.684238 (+233730us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:13.684277 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:13.684593 (+   316us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:13.939194 (+254601us) raft_consensus.cc:1685] finished
0709 12:16:13.939217 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:13.939427 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":209,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8322,"spinlock_wait_cycles":9294208,"thread_start_us":233535,"threads_started":1,"wal-append.queue_time_us":1000691}
W20260709 12:16:13.943472 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:13.943913 20048 consensus_meta.cc:287] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.848s	user 0.013s	sys 0.002s
I20260709 12:16:13.944430 11252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33812 (request call id 2995) took 1115 ms. Trace:
I20260709 12:16:13.944350 20048 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.944823 11252 rpcz_store.cc:276] 0709 12:16:12.829065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:12.829209 (+   144us) service_pool.cc:224] Handling call
0709 12:16:12.829537 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:12.829949 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:16:13.045251 (+215302us) log.cc:844] Serialized 14 byte log entry
0709 12:16:13.485925 (+440674us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:13.493308 (+  7383us) log.cc:844] Serialized 26 byte log entry
0709 12:16:13.493568 (+   260us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:16:13.493646 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:13.493747 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:13.933391 (+439644us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec628786e 00007f3ec6286bb6 00007f3ec62867d8 00007f3ec6290221 00007f3ec62901d9 00007f3ec6290169
0709 12:16:13.944131 (+ 10740us) raft_consensus.cc:1685] finished
0709 12:16:13.944229 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:13.944368 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4223,"prepare.queue_time_us":320028,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":7739,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":6366,"spinlock_wait_cycles":38058752,"thread_start_us":605389,"threads_started":2,"wal-append.queue_time_us":672645}
I20260709 12:16:13.947217 20126 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:13.947080 20128 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.947906 10011 tablet_replica.cc:333] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:13.948920 10011 raft_consensus.cc:2243] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.949425 20038 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.950979 20334 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:13.951301 20191 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:13.953285 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:13.954660 20209 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:13.956080 20141 raft_consensus.cc:493] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:13.956740 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 43571.4us
I20260709 12:16:13.956605 20141 raft_consensus.cc:515] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.957422 11927 raft_consensus.cc:1275] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:16:13.957455 11296 leader_election.cc:336] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.958477 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:13.949510 10011 raft_consensus.cc:2272] T 9fdf86dcb01f4d81913c37b45cfc3879 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.958753 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.956761 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.960651 11934 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 30.
I20260709 12:16:13.962736 11330 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 30.
W20260709 12:16:13.964336 11146 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:13.964612 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:13.944881 20229 consensus_queue.cc:237] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:13.986357 20141 leader_election.cc:290] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:13.986382 11296 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:13.986452 20186 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:13.988070 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:13.993214 11222 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:14.003178 11222 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.003806 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:14.007731 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:14.024820 11296 leader_election.cc:336] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.025480 11296 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:14.032842 11296 leader_election.cc:336] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.033535 11296 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:14.035665 11296 leader_election.cc:336] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.036376 11296 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:14.039505 11296 leader_election.cc:336] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.039945 11296 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:14.040372 10011 tablet_replica.cc:333] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
W20260709 12:16:14.041709 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:14.046216 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.046906 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.048544 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:13.992889 20141 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:14.064343 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 61199.4us
W20260709 12:16:14.086241 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 48017.8us
W20260709 12:16:14.087393 11296 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.087894 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.090777 11148 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:14.092468 11148 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:14.082788 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:14.023449 20330 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:14.030382 20191 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:14.103068 10011 raft_consensus.cc:2243] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:14.043685 20334 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:14.117111 11148 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:16:14.106462 11222 leader_election.cc:336] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.049640 20291 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.130785 20291 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.135025 20124 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:14.135809 20124 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:14.136976 20124 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:14.142390 11148 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.001806 20209 consensus_queue.cc:1048] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.097s
I20260709 12:16:14.146062 11331 raft_consensus.cc:2468] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:14.147471 20158 raft_consensus.cc:3060] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:14.113533 20126 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:14.049950 20186 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:14.052657 20320 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:16:14.126293 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 384266us
I20260709 12:16:14.105162 19747 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:14.102249 20337 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:14.126526 20055 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:14.158192 10011 raft_consensus.cc:2272] T 0a6ec18bb8c54ece99a794e109201789 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:14.126853 20279 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:14.166378 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 79011.6us
W20260709 12:16:14.167762 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:14.169440 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:14.145906 20128 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:14.099231 20141 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:14.156584 20304 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:14.156947 20191 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:16:14.181243 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 135068us
I20260709 12:16:14.144341 20332 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:16:14.126235 20330 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:14.194173 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.194440 11325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
W20260709 12:16:14.187297 20050 consensus_meta.cc:287] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.000s
I20260709 12:16:14.195883 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:14.199461 11296 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.199184 11329 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
W20260709 12:16:14.200786 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.201213 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.180051 20128 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:14.171819 19747 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:14.208457 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:14.210769 11296 leader_election.cc:336] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.211249 11296 leader_election.cc:304] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:14.214111 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eaf18bb7a144d35b42cf5548b769c0f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:14.215565 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:14.216356 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.216722 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.217957 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.219707 10011 tablet_replica.cc:333] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:14.220185 20141 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.220613 10011 raft_consensus.cc:2243] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:14.220728 20320 raft_consensus.cc:2749] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:14.221868 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.223732 20320 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:14.224444 20334 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:14.226413 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.227535 20334 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:14.228742 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.230962 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:14.232149 20334 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:14.232522 20334 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.232858 20334 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:14.236078 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.200605 20304 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:14.249554 20343 log.cc:927] Time spent T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:16:14.244812 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.245620 20048 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:14.214684 20191 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:14.204864 20050 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.250881 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.251081 20304 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:14.252269 20333 log.cc:927] Time spent T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:16:14.243768 11251 raft_consensus.cc:2393] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 27: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:14.246510 11934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.253245 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.266127 11256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:14.267179 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.258088 11928 raft_consensus.cc:2393] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 in current term 32: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:14.261901 10011 raft_consensus.cc:2272] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:14.275134 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.287309 20330 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:14.279798 20345 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 12:16:14.290251 20331 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:14.290560 20128 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.293413 11928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39571de4248f4cfd85f9b1062a776cc8" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.299023 11926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.293923 20341 raft_consensus.cc:493] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.306043 20341 raft_consensus.cc:515] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.308439 20341 leader_election.cc:290] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:14.309464 20327 log.cc:927] Time spent T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.494s	user 0.001s	sys 0.001s
I20260709 12:16:14.281586 20141 raft_consensus.cc:493] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:14.314812 20334 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.324656 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:14.335134 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 329034us
I20260709 12:16:14.336024 20141 raft_consensus.cc:515] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.338429 20215 leader_election.cc:290] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:14.339699 20141 leader_election.cc:290] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:14.341566 20224 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:14.301549 20330 raft_consensus.cc:493] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
I20260709 12:16:14.350052 20330 raft_consensus.cc:515] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.352995 20330 leader_election.cc:290] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:14.354660 20304 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.328176 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.328214 11927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.291074 19747 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.364573 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.367679 11147 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.368427 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.370893 11327 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:16:14.379000 11147 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.370502 20301 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:14.381471 20158 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:14.382259 20136 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.371601 19747 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:14.384305 19747 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:14.385794 11900 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.370414 20124 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:14.390790 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.392071 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.391520 11249 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 29.
I20260709 12:16:14.398237 11147 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.402338 11932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52014 (request call id 1420) took 1534 ms. Trace:
I20260709 12:16:14.403025 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.402951 11932 rpcz_store.cc:276] 0709 12:16:12.867679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:12.867839 (+   160us) service_pool.cc:224] Handling call
0709 12:16:12.868118 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:13.696329 (+828211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:14.402011 (+705682us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b5400415fd4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66ece01 00007f3ec66eb83e 00007f3ecb6b0419 00007f3ec4c27bd4 00007f3ec4c27b62 00007f3ec4c27a91 00007f3ec4c27a0c 00007f3ec4c26cc2 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:14.402255 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1900971776}
I20260709 12:16:14.404181 11322 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:16:14.408802 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.394119 20125 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:14.428767 11147 raft_consensus.cc:2686] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:16:14.458017 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:14.459753 11147 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.468510 11223 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:14.474335 20126 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:14.475113 19747 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.478091 20334 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:14.479147 20332 raft_consensus.cc:3060] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:16:14.479764 11147 leader_election.cc:304] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:14.480608 20125 raft_consensus.cc:2749] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:14.482901 20269 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.490955 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.493829 10011 tablet_replica.cc:333] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:14.494987 10011 raft_consensus.cc:2243] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:14.500937 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97e23379bda4a1b9ad1e2eabbf4d82f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.501170 11328 raft_consensus.cc:1240] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:16:14.501241 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:14.502043 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71381.5us
I20260709 12:16:14.506443 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.506827 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.507710 20357 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:14.506028 20209 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:14.508481 20209 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:14.508937 20209 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:14.508307 11253 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index 4 to 13, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:14.514041 20358 consensus_queue.cc:1059] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:16:14.497593 20277 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.516151 20277 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:14.505146 20232 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.007s	sys 0.009s
I20260709 12:16:14.518618 20277 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.518759 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:14.527657 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c21b703f1a4a445a8505bb01530a8439" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.528076 11328 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:16:14.528426 11932 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:16:14.530411 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e478aac72b4a62a8ed9124187f2dec" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.537158 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c40179116e14aa2aff8e95298cbcd4f" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:14.518167 20232 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.007s	sys 0.009s
W20260709 12:16:14.502113 20088 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.005s	sys 0.009s
I20260709 12:16:14.541303 20076 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:14.508332 20357 raft_consensus.cc:493] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:14.550668 20357 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:14.580173 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.564405 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:14.577987 11250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:14.641404 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 65256.5us
I20260709 12:16:14.535620 20237 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 12:16:14.561755 11253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.645959 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:14.553896 20088 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.006s	sys 0.009s
I20260709 12:16:14.665678 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:14.679039 11254 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:16:14.540804 20277 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:16:14.673745 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 98700.3us
I20260709 12:16:14.675285 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ddcb89ab6d47cd97e8c19734f0e365" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:14.680130 11935 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:14.675360 11328 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 30.
I20260709 12:16:14.681730 19742 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 12:16:14.684032 20269 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.684566 20269 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:14.692909 20269 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.576208 20286 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:14.699169 11250 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:16:14.697132 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:14.703423 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.709589 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a634acd9ff543cca551c8a28e72d6c4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.716593 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:14.717234 11930 raft_consensus.cc:3060] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:14.553534 20209 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:14.724022 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 181457us
W20260709 12:16:14.745755 11147 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:14.757742 11254 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
W20260709 12:16:14.756261 11296 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:14.759277 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:14.764428 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:14.768293 11296 leader_election.cc:304] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:14.787268 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:14.782613 20355 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:14.791159 11935 raft_consensus.cc:2468] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:16:14.788070 20356 raft_consensus.cc:2749] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:14.803416 20380 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.838399 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89068ffedf554d89af62a8b1fe0700c0" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:14.841034 11331 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:16:14.834827 20268 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.846952 11931 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:16:14.850045 11927 raft_consensus.cc:1275] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:16:14.840013 20038 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.862730 20038 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:14.877030 20050 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:14.836323 20380 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.883574 20215 raft_consensus.cc:3055] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:16:14.884055 20215 raft_consensus.cc:740] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:14.884577 20380 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:14.764410 20304 raft_consensus.cc:2955] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Committing config change with OpId 23.13: config changed from index 4 to 13, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:14.892899 20357 raft_consensus.cc:515] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.894203 11932 raft_consensus.cc:1275] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:16:14.899472 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:14.900187 11248 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:14.901333 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25983a67e23a4c708ae496061b992207" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.906077 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458129fddd7e4b33b63d818f6af21f23" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:14.884851 20215 consensus_queue.cc:260] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.895236 20357 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.922706 11248 raft_consensus.cc:2468] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 30.
W20260709 12:16:14.928225 11900 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 133 similar messages]
I20260709 12:16:14.930989 11255 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:14.943452 11930 raft_consensus.cc:2468] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:16:14.866667 20268 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:14.952939 11900 leader_election.cc:304] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:14.959995 20340 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:14.965140 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:14.967859 20341 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:14.971364 20341 raft_consensus.cc:3055] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:16:14.971858 20341 raft_consensus.cc:740] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:14.981159 20062 raft_consensus.cc:2804] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:15.002092 11250 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:16:15.004186 20291 leader_election.cc:290] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:15.004637 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:15.005451 11930 raft_consensus.cc:2393] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 28: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:16:15.008073 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.010962 11148 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.015774 11326 raft_consensus.cc:1240] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Rejecting Update request from peer fecfc03de52a4308bc9b045cad5aac1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:16:15.016426 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:15.011258 20062 raft_consensus.cc:697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:15.018234 20062 consensus_queue.cc:237] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.008715 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.015851 20209 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:14.949380 20215 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:15.027453 20038 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:15.032437 11222 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:15.032758 20268 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:15.039021 11148 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:15.039719 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.040359 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.042104 20320 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:15.043004 20320 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:15.045822 20209 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:15.046657 20209 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:15.049098 11296 leader_election.cc:336] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.049782 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:15.051069 11296 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.051926 20356 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:15.055056 11296 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:15.056562 11296 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.057065 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.045408 20326 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:15.062265 20326 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.067986 20276 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.068735 20276 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.069950 20320 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:15.074735 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 49900us
W20260709 12:16:15.074756 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.075562 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.081853 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:15.082522 11250 raft_consensus.cc:1240] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 25. Current term is 26. Ops: []
I20260709 12:16:15.075711 11148 leader_election.cc:304] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:15.085258 11900 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:15.087072 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:15.089798 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:15.096022 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.096547 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.096985 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:15.098172 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.100296 20320 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:15.103991 11296 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:15.109310 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:15.084873 20124 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:15.102808 20268 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:15.129608 11296 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.107335 20286 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:16:15.126845 20320 raft_consensus.cc:2804] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:15.107868 20214 consensus_queue.cc:1059] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.025s
I20260709 12:16:15.126245 20326 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:15.102432 20299 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.035s
I20260709 12:16:15.127480 20276 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:15.151366 20335 raft_consensus.cc:3055] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:15.158260 20209 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:15.135787 20359 consensus_queue.cc:1059] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 12:16:15.141010 20237 raft_consensus.cc:2804] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:16:15.162631 20332 consensus_meta.cc:287] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: Time spent flushing consensus metadata: real 0.683s	user 0.012s	sys 0.003s
I20260709 12:16:15.165673 20335 raft_consensus.cc:740] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:15.171542 20332 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:16:15.167832 20320 raft_consensus.cc:697] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:15.204212 10011 raft_consensus.cc:2272] T 86c5246d860d4d559ea665671cc46b5c P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:15.175252 20335 consensus_queue.cc:260] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:15.242198 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 166380us
I20260709 12:16:15.244081 19742 raft_consensus.cc:493] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.248427 20124 raft_consensus.cc:2749] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:15.253042 20320 consensus_queue.cc:237] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.258936 20237 raft_consensus.cc:697] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:16:15.260234 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 167571us
I20260709 12:16:15.247987 20158 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.266402 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a6ec18bb8c54ece99a794e109201789" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:15.273550 20335 raft_consensus.cc:3060] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:15.276194 20165 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:16:15.277329 10011 tablet_replica.cc:333] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:15.278286 20276 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:16:15.257045 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 117778us
I20260709 12:16:15.294196 20276 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.294833 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.298493 20355 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:16:15.298375 20158 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:15.266599 20224 raft_consensus.cc:3055] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:16:15.306099 20224 raft_consensus.cc:740] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:15.307926 20224 consensus_queue.cc:260] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.309352 20224 raft_consensus.cc:3060] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:15.312304 11248 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:16:15.282229 19742 raft_consensus.cc:515] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:15.352000 19742 leader_election.cc:290] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:15.287117 20237 consensus_queue.cc:237] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.327798 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:15.331565 20276 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:15.332674 20158 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:15.340313 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:15.348238 11148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 64202.1us
W20260709 12:16:15.355502 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:15.357067 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:15.360294 10011 raft_consensus.cc:2243] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:15.360743 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:15.370415 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.371201 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.372447 20276 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:15.361963 10011 raft_consensus.cc:2272] T e03893c6062f45ea972113cfa57ba362 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:15.292482 20341 consensus_queue.cc:260] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:15.379745 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.393692 20165 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:15.394467 20165 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:15.402848 11148 leader_election.cc:304] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:15.404183 20038 raft_consensus.cc:2749] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:15.406026 20396 log.cc:927] Time spent T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.072s	user 0.004s	sys 0.000s
I20260709 12:16:15.407941 19742 consensus_queue.cc:1059] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:15.414851 20128 raft_consensus.cc:493] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:15.415812 20128 raft_consensus.cc:515] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:15.417239 20399 log.cc:927] Time spent T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 12:16:15.419824 20128 leader_election.cc:290] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:15.425030 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fdf86dcb01f4d81913c37b45cfc3879" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.442754 11929 raft_consensus.cc:1240] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 LEADER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 29. Current term is 30. Ops: []
W20260709 12:16:15.450368 11145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:15.456326 20398 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.181s	user 0.003s	sys 0.000s
I20260709 12:16:15.465845 20341 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:16:15.472926 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 304926us
W20260709 12:16:15.474344 11222 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:15.475335 11222 leader_election.cc:336] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.476821 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.478752 20124 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:15.484580 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 61348us
W20260709 12:16:15.499749 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 129396us
I20260709 12:16:15.447175 10011 tablet_replica.cc:333] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
W20260709 12:16:15.512555 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:15.485272 20165 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.526132 20038 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:15.555395 10011 raft_consensus.cc:2243] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:15.558717 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.566112 20388 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:15.564385 11147 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:15.568544 11296 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:15.587471 10011 pending_rounds.cc:70] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Trying to abort 1 pending ops.
I20260709 12:16:15.587646 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28d2d114b03e403c83f7001bc117434e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.588024 10011 pending_rounds.cc:77] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 24 index: 15 } timestamp: 7305622973558091776 op_type: NO_OP noop_request { }
I20260709 12:16:15.588485 10011 raft_consensus.cc:2889] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:15.588913 10011 raft_consensus.cc:2272] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:15.589752 11251 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:16:15.582734 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.596273 20223 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.589316 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.601039 20165 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:15.607046 20304 consensus_meta.cc:287] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.019s
I20260709 12:16:15.614822 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.619027 20038 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.619789 20038 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.597219 20223 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:15.623557 11323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:15.625187 20223 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:15.635701 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 78255.8us
W20260709 12:16:15.638342 11296 leader_election.cc:336] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:15.638811 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 84979.3us
I20260709 12:16:15.639670 20223 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:15.646160 20038 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:15.654897 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2d37a48723c42228c9717f514174704" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:15.657546 11148 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.659066 20269 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:15.659241 11148 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:15.660719 20269 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:15.667847 10011 tablet_replica.cc:333] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:15.669674 10011 raft_consensus.cc:2243] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:15.670742 10011 raft_consensus.cc:2272] T 5ad54acf641349bc8119ce4c0eae446b P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:15.751955 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 137991us
I20260709 12:16:15.759425 11929 raft_consensus.cc:1240] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Rejecting Update request from peer 016411114c6d42039bd04025378dfc98 for earlier term 30. Current term is 31. Ops: []
I20260709 12:16:15.766181 11326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.768512 20388 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:15.769474 20388 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.772677 20388 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:15.780135 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 105812us
I20260709 12:16:15.779762 10011 tablet_replica.cc:333] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:15.785460 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:15.789315 11294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:15.804674 11326 raft_consensus.cc:1275] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:16:15.818817 20299 raft_consensus.cc:493] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.819779 20299 raft_consensus.cc:515] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:15.823566 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62903fc48c140aca4aa949625fa88f6" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:15.847512 11147 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:15.856737 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 188085us
I20260709 12:16:15.868376 10011 raft_consensus.cc:2243] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:15.869437 10011 raft_consensus.cc:2272] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:15.852603 20038 consensus_queue.cc:1059] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.021s
I20260709 12:16:15.881381 20223 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:15.895998 11222 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:15.891449 20354 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:15.910981 20038 raft_consensus.cc:493] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.911067 20300 raft_consensus.cc:493] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.941144 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e4ceea03b94eb2803569189ab5fcca" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:15.926026 20128 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:16:15.984606 11327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:15.913882 20062 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.088s
I20260709 12:16:15.985038 20128 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting leader election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:16.000342 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:16.021385 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 128733us
I20260709 12:16:15.999194 20299 leader_election.cc:290] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.013726 20128 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:15.995057 20038 raft_consensus.cc:515] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:16.040138 11248 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:16:16.018795 20276 consensus_queue.cc:1048] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
W20260709 12:16:16.052058 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 187187us
W20260709 12:16:16.051278 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 203895us
I20260709 12:16:16.049315 20299 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.006732 20300 raft_consensus.cc:515] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.057751 20358 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:16.063007 20406 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:16:16.057916 20299 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.067137 20419 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:16.067590 20332 raft_consensus.cc:3055] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:16:16.068198 20332 raft_consensus.cc:740] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:16.067970 20419 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:16.070675 11221 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:16.053771 20304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41142 (request call id 12) took 4514 ms. Trace:
I20260709 12:16:16.078994 20299 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.074663 11222 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:16.097375 20419 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:16.079401 20304 rpcz_store.cc:276] 0709 12:16:11.544083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.681507 (+137424us) service_pool.cc:224] Handling call
0709 12:16:12.108455 (+426948us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5800824cd0. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec66480aa 00007f3ec66ca447 00007f3ec66e770a 00007f3ec66e876a 00007f3ec66e791b 00007f3ec66f303c 00007f3ec66f26b3 00007f3ec66ea9fa 00007f3ecb6b24c9 00007f3ec4c2ec54
0709 12:16:12.144415 (+ 35960us) log.cc:844] Serialized 501 byte log entry
0709 12:16:12.567394 (+422979us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b5400780b20. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6655571 00007f3ec6665142 00007f3ec6645eaa 00007f3ec664cd53 00007f3ec664ccd9 00007f3ec664cc69 00007f3ec664cc31 00007f3ec664bdfd 0000000000388634 0000000000388479 00007f3ec35ff475
0709 12:16:16.044845 (+3477451us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18435,"raft.queue_time_us":25209,"raft.run_cpu_time_us":7396,"raft.run_wall_time_us":167318,"spinlock_wait_cycles":344129536,"thread_start_us":267336,"threads_started":1,"wal-append.queue_time_us":608306,"wal-append.run_cpu_time_us":3394,"wal-append.run_wall_time_us":1080090}
I20260709 12:16:16.110064 10011 tablet_replica.cc:333] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.124567 20300 leader_election.cc:290] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.138507 11257 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:16:16.074262 20358 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.069288 20332 consensus_queue.cc:260] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.150025 11251 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 12:16:16.150539 11251 pending_rounds.cc:85] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Aborting all ops after (but not including) 17
I20260709 12:16:16.150828 11251 pending_rounds.cc:107] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 12:16:16.151202 11251 raft_consensus.cc:2889] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:16.158056 11935 raft_consensus.cc:1275] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:16:16.160187 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 106042us
I20260709 12:16:16.161118 11222 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:16.149860 10011 raft_consensus.cc:2243] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:16:16.173441 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 91285.4us
I20260709 12:16:16.181530 11222 leader_election.cc:304] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:16.168987 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:16.167119 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:16.174682 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 95681.6us
I20260709 12:16:16.201162 11147 leader_election.cc:304] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:16.165848 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:16.168074 20362 raft_consensus.cc:2749] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:16.205952 20268 raft_consensus.cc:2749] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:16.178895 20358 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.170006 20128 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.209053 20128 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 31 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.209797 11252 raft_consensus.cc:1240] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Rejecting Update request from peer bc92f9d2c6c5477182fa33eba4368ec1 for earlier term 30. Current term is 31. Ops: []
W20260709 12:16:16.210366 11075 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 247) took 35339 ms (client timeout 15000 ms). Trace:
I20260709 12:16:16.214315 11147 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:16.215220 11147 raft_consensus.cc:2686] T 16e4ceea03b94eb2803569189ab5fcca P 016411114c6d42039bd04025378dfc98: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:16:16.215291 10011 raft_consensus.cc:2272] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.224299 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80d10b7c9e4644e9a03edf10767df625" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.230822 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:16.234128 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.235352 11931 raft_consensus.cc:2410] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 30 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:16:16.239635 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73b50321123c433f91c5d0972791cf37" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.244211 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e235bf9f3514f629a5c26c1eac58682" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:16.210831 11075 rpcz_store.cc:269] 0709 12:15:40.890589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.890796 (+   207us) service_pool.cc:224] Handling call
0709 12:16:16.210061 (+35319265us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:11.975615 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:12.055350 (+ 79735us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:12.055393 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:16:12.055416 (+    23us) tablet.cc:662] Decoding operations
0709 12:16:12.057121 (+  1705us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:12.057220 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:12.057268 (+    48us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:16:12.106122 (+ 48854us) tablet.cc:701] Row locks acquired
0709 12:16:12.106164 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:16:12.106368 (+   204us) write_op.cc:270] Start()
0709 12:16:12.106523 (+   155us) write_op.cc:276] Timestamp: P: 1783599372106305 usec, L: 0
0709 12:16:12.106555 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:16:12.215175 (+108620us) log.cc:844] Serialized 6392 byte log entry
0709 12:16:13.693552 (+1478377us) op_driver.cc:464] REPLICATION: finished
0709 12:16:13.932701 (+239149us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec35e20ac 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3eca122ef0
0709 12:16:14.337793 (+405092us) write_op.cc:301] APPLY: starting
0709 12:16:14.337878 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:14.879890 (+542012us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:14.879920 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:15.245002 (+365082us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:15.251350 (+  6348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:16:15.277744 (+ 26394us) write_op.cc:312] APPLY: finished
0709 12:16:15.278673 (+   929us) log.cc:844] Serialized 148 byte log entry
0709 12:16:15.415181 (+136508us) spinlock_profiling.cc:243] Waited 82.8 ms on lock 0x7b4400022b4c. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec3525c85 00007f3ec35266a2 00007f3ec62981db 00007f3ec6293aaf 00007f3ec35e20ac 00007f3ecc43c9a9 00007f3ec35fe69b 00007f3ec35fc93f 00007f3ec35fec02 00007f3ec628620b
0709 12:16:15.524739 (+109558us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:15.583514 (+ 58775us) write_op.cc:454] Released schema lock
0709 12:16:15.591322 (+  7808us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:15.649964 (+ 58642us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:15.649996 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":108720,"reported_tablets":50,"spinlock_wait_cycles":18304,"child_traces":[["op",{"apply.queue_time_us":643914,"apply.run_cpu_time_us":69427,"apply.run_wall_time_us":1312068,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71135,"cfile_init":1,"lbm_read_time_us":439,"lbm_reads_lt_1ms":4,"num_ops":16,"prepare.queue_time_us":259246,"prepare.run_cpu_time_us":20957,"prepare.run_wall_time_us":693141,"raft.queue_time_us":540800,"raft.run_cpu_time_us":42482,"raft.run_wall_time_us":502126,"replication_time_us":1586780,"spinlock_wait_cycles":288534016,"thread_start_us":1646538,"threads_started":5,"wal-append.queue_time_us":1006312,"wal-append.run_cpu_time_us":38666,"wal-append.run_wall_time_us":1540790}]]}
I20260709 12:16:16.250609 20332 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:16.252995 11147 leader_election.cc:304] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:16.255225 20358 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:16.255707 20128 raft_consensus.cc:2749] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:16.257423 10011 tablet_replica.cc:333] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.255918 20362 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:16.258255 20358 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:16.254711 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 101237us
I20260709 12:16:16.167666 20038 leader_election.cc:290] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:16.265070 11147 leader_election.cc:304] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:16.258919 10011 raft_consensus.cc:2243] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.288048 20223 raft_consensus.cc:2749] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:16:16.288705 11931 raft_consensus.cc:1275] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:16:16.282639 20158 raft_consensus.cc:2749] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:16.290552 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 86283.9us
W20260709 12:16:16.293000 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:16.250615 20268 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.296604 11147 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:16.303217 10011 raft_consensus.cc:2272] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.306887 20299 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.307663 20299 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.308566 20377 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:16:16.309554 11147 leader_election.cc:304] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:16.310705 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:16.312045 20422 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:16.294368 20358 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.314082 20377 consensus_queue.cc:1059] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:16:16.315323 20377 raft_consensus.cc:3055] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:16:16.315680 20377 raft_consensus.cc:740] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:16.316332 10011 tablet_replica.cc:333] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.316648 20377 consensus_queue.cc:260] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.317605 20421 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:16.317973 20423 raft_consensus.cc:2749] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:16.318219 20377 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:16:16.317579 10011 raft_consensus.cc:2243] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.319573 10011 raft_consensus.cc:2272] T 4c40179116e14aa2aff8e95298cbcd4f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.322190 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5df07764623640ce8ec29ad695812b7c with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:16.325258 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:16.326045 11328 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:16:16.326799 20299 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.327469 20268 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:16.330207 20268 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:16.331346 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:16.339171 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:16.341588 20425 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:16.342092 20425 raft_consensus.cc:493] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:16.342489 20425 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:16.353145 10011 tablet_replica.cc:333] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.369881 20277 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:16.427579 10011 raft_consensus.cc:2243] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Raft consensus shutting down.
W20260709 12:16:16.353021 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 171582us
I20260709 12:16:16.438906 10011 raft_consensus.cc:2272] T 35c7899b1a314ae1b67824dfee6ff091 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.425429 11071 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:16.444922 10011 tablet_replica.cc:333] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.427613 20277 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.388720 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:16.446123 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 57534.2us
I20260709 12:16:16.447081 20277 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:16.447928 20427 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:16.448477 20427 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.448876 20427 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:16.355060 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f62270698314885bc0e03e0de4507ae" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:16.369809 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 45048.3us
I20260709 12:16:16.354125 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:16.394348 20425 raft_consensus.cc:515] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.440043 20428 consensus_queue.cc:1048] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:16:16.456447 20307 consensus_queue.cc:1048] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:16:16.440884 20426 consensus_queue.cc:1048] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:16:16.457525 20425 leader_election.cc:290] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:16.355927 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5a029ce312749bdaf2113d2823e993e" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:16.376308 11900 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:16.458578 11900 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:16.459365 20320 consensus_queue.cc:1048] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:16:16.460207 20425 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:16:16.461295 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9dab54d96124869a4a52f9cc08e288a" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.467751 20426 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:16.468499 20426 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.470484 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.471392 11935 raft_consensus.cc:2393] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fecfc03de52a4308bc9b045cad5aac1e in current term 25: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:16.471601 20426 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.473960 11071 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 20 to 21, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 21 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.368346 11251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5f804d45235415fad0f1442ea04aa47" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:16.484202 11147 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:16.490805 20268 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:16.492126 11071 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 20 to 22, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 22 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:16.493920 11147 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:16.495294 20332 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:16.494288 11071 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 19 to 21, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 21 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:16.499307 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.519078 11929 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:16:16.522054 11929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:16.516708 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 69398.9us
I20260709 12:16:16.503623 20427 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.527201 20427 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:16.459568 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 83350.3us
I20260709 12:16:16.511519 20320 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:16.528262 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:16.528304 20320 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.516754 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
W20260709 12:16:16.529009 11900 leader_election.cc:336] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:16.532243 20320 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:16.497470 11328 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 31.
I20260709 12:16:16.537199 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:16.537927 11328 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:16:16.539654 10011 raft_consensus.cc:2243] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.541927 10011 raft_consensus.cc:2272] T 53e6b498179b4c96b8b0be6240a36efb P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.553289 11933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.554188 11933 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:16:16.619499 20320 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:16.620496 20320 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:16.611886 11223 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:16.629822 20431 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:16:16.630501 20431 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 31 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:16.630384 11071 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 20 to 22, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 22 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.631738 20431 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.655516 11222 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:16.664824 20358 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:16.678944 10011 tablet_replica.cc:333] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.679055 20320 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:16.679869 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:16.680272 10011 raft_consensus.cc:2243] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.682744 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:16.683386 11324 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:16.691850 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:16.698266 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:16.700219 20299 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.700827 20299 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:16.703148 20299 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:16.686373 20427 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:16.731670 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:16.755856 20136 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.756520 20136 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:16.757004 20427 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:16.757555 20427 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:16.758989 20136 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:16.633169 11071 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 25 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.769582 11071 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 18 to 21, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 21 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.775259 10011 raft_consensus.cc:2272] T ddf8cca10e1244d8882abd13fa81113a P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.772704 11255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad54acf641349bc8119ce4c0eae446b" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:16.792431 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 51336.7us
I20260709 12:16:16.794919 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa73285733a4a67a3e705a20155f69d" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:16.795850 11249 raft_consensus.cc:3060] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:16.796578 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:16.799211 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:16.799180 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4958b19b61094e1db763fdb94852f994" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:16.810206 11328 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 25.
I20260709 12:16:16.823877 11324 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
I20260709 12:16:16.808355 11252 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 26.
I20260709 12:16:16.842568 11223 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:16.845515 11254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:16.843711 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:16.865054 11147 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:16.865849 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 242219us
I20260709 12:16:16.862349 20358 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:16.869143 20332 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:16:16.893011 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:16.893527 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:16.877125 20268 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:16.899062 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56261.6us
I20260709 12:16:16.773054 11071 catalog_manager.cc:5697] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:16.896313 20332 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.897938 20358 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:16.926681 11071 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 20 to 21, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 21 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:16.928526 20433 log.cc:927] Time spent T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:16:16.902596 20440 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:16.929068 11071 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:16:16.942556 20432 log.cc:927] Time spent T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:16:16.910658 20332 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:16.940572 10011 tablet_replica.cc:333] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.941665 20358 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.979872 10011 raft_consensus.cc:2243] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.980729 10011 raft_consensus.cc:2272] T 3e235bf9f3514f629a5c26c1eac58682 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.984385 10011 tablet_replica.cc:333] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:16.993124 20427 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:16.996920 20427 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:17.014290 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 63373.1us
I20260709 12:16:17.014346 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:17.015411 11329 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:16.985455 10011 raft_consensus.cc:2243] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Raft consensus shutting down.
W20260709 12:16:17.014750 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.016723 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:17.019276 11249 raft_consensus.cc:2468] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:16:17.018864 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:17.024251 11900 leader_election.cc:336] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:17.028308 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.033449 10011 raft_consensus.cc:2272] T 7f736e38db6b45bfa87e35af564bffc1 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.026665 20304 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:16.945990 11071 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 18 to 23, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 23 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.977234 20332 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.035243 11329 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
I20260709 12:16:17.037098 11900 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:17.028420 20224 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:17.056170 20388 raft_consensus.cc:2804] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:17.056810 20388 raft_consensus.cc:697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:17.057832 20388 consensus_queue.cc:237] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.061555 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:17.043164 11071 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 16 to 26. New cstate: current_term: 26 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:17.063366 11071 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 17 to 20, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 20 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:17.062294 11329 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 27
W20260709 12:16:17.078239 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 396934us
I20260709 12:16:17.086540 11329 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
W20260709 12:16:17.095904 11222 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:17.095999 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46612.6us
I20260709 12:16:17.098665 11148 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
W20260709 12:16:17.099848 11147 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 196850us
W20260709 12:16:17.100942 11222 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.082301 20332 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 27 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:17.104936 11222 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:17.109139 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:17.104103 20446 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:17.097262 20441 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.110636 11257 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:17.113723 20440 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:17.114641 20446 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:17.127580 11329 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:16:17.120031 20440 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.118080 20358 raft_consensus.cc:2749] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:17.148180 10011 tablet_replica.cc:333] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.144402 20431 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
W20260709 12:16:17.148856 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:17.149549 11296 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:17.150767 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.151265 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:17.152127 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54173.7us
W20260709 12:16:17.153031 11222 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.153621 20440 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:17.154598 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.154659 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:17.149379 10011 raft_consensus.cc:2243] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.158303 11330 raft_consensus.cc:2393] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 31: Already voted for candidate fecfc03de52a4308bc9b045cad5aac1e in this term.
I20260709 12:16:17.118124 20441 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.163616 20441 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:17.127570 20446 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.167325 20320 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:17.169117 10011 raft_consensus.cc:2272] T f62903fc48c140aca4aa949625fa88f6 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.172645 10011 tablet_replica.cc:333] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
W20260709 12:16:17.173804 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.175446 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:17.174554 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.184691 10011 raft_consensus.cc:2243] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.183427 20388 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:16:17.204689 20448 log.cc:927] Time spent T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
W20260709 12:16:17.193756 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.211784 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:17.232473 11935 raft_consensus.cc:2393] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 31: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
W20260709 12:16:17.201964 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49004.8us
W20260709 12:16:17.235205 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:17.235896 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:17.236685 11222 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.237102 11222 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
W20260709 12:16:17.236698 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.222587 10011 raft_consensus.cc:2272] T 4a634acd9ff543cca551c8a28e72d6c4 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.245095 10011 tablet_replica.cc:333] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.248140 20428 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:17.250773 10011 raft_consensus.cc:2243] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.260666 11257 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 27.
I20260709 12:16:17.274669 10011 raft_consensus.cc:2272] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:17.276633 11296 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.277154 11296 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:17.278142 20441 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:16:17.286113 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 49475.6us
I20260709 12:16:17.290079 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:17.290719 11935 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:16:17.294407 20427 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.295027 20427 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.297183 20427 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:17.306319 10011 tablet_replica.cc:333] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.307677 10011 raft_consensus.cc:2243] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.310154 20388 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:17.310523 10011 raft_consensus.cc:2272] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.312882 11932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:17.315239 11931 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:16:17.320726 20441 raft_consensus.cc:2749] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:17.325488 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:17.326203 11930 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:16:17.330008 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:17.331059 11222 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:17.332204 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.332423 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:17.352360 11935 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
I20260709 12:16:17.354951 20427 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:17.357530 11329 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
W20260709 12:16:17.365600 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:17.366674 11222 leader_election.cc:336] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.367676 11222 leader_election.cc:304] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:17.367286 20128 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.008s
I20260709 12:16:17.377802 10011 tablet_replica.cc:333] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.378791 10011 raft_consensus.cc:2243] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.379889 10011 raft_consensus.cc:2272] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.381271 11930 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
W20260709 12:16:17.402371 11222 leader_election.cc:336] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.402817 11222 leader_election.cc:304] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:17.403366 20428 raft_consensus.cc:2749] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:17.407091 11222 leader_election.cc:336] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.408360 11222 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:17.407892 20128 raft_consensus.cc:2749] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:17.415966 20426 raft_consensus.cc:2749] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:17.417799 11222 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.420284 10011 tablet_replica.cc:333] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.421423 10011 raft_consensus.cc:2243] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.422816 10011 raft_consensus.cc:2272] T 0fd765747f9648049b905c70696b206d P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.426962 10011 tablet_replica.cc:333] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.451555 10011 raft_consensus.cc:2243] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.450959 20341 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.454010 20341 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.457145 20341 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:17.458429 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:17.459406 11330 raft_consensus.cc:2410] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 016411114c6d42039bd04025378dfc98 for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:16:17.461136 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:17.461532 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.462774 11147 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:17.464389 20341 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:17.474053 10011 raft_consensus.cc:2272] T 6f62270698314885bc0e03e0de4507ae P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.477408 10011 tablet_replica.cc:333] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
W20260709 12:16:17.483044 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:17.485653 10011 raft_consensus.cc:2243] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.486994 10011 raft_consensus.cc:2272] T c519deb8010e411995230a6685947b97 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.488333 20128 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.489012 20128 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.490794 20358 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:17.492331 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:17.490859 11222 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:17.500530 20335 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:17.501116 20335 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:17.502153 20335 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.500084 20128 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:17.509471 11222 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:17.514683 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:17.524644 11222 leader_election.cc:336] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.528136 11222 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:17.530923 20128 raft_consensus.cc:2749] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:17.559446 10011 tablet_replica.cc:333] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.560436 10011 raft_consensus.cc:2243] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.561000 10011 raft_consensus.cc:2272] T 80d10b7c9e4644e9a03edf10767df625 P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.569439 11935 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:16:17.572058 20332 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.572681 20332 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.574890 20332 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:17.578153 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:17.578831 11330 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 31.
I20260709 12:16:17.581177 11148 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:17.583964 10011 tablet_replica.cc:333] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.584932 10011 raft_consensus.cc:2243] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.585592 10011 raft_consensus.cc:2272] T 02e478aac72b4a62a8ed9124187f2dec P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:17.586937 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.587852 20341 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:16:17.588291 20341 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.588640 20341 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:16:17.588887 10011 tablet_replica.cc:333] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.586815 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:17.590310 11935 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 31.
I20260709 12:16:17.595558 10011 raft_consensus.cc:2243] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.596848 10011 raft_consensus.cc:2272] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.599048 20341 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.599828 10011 tablet_replica.cc:333] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.600792 10011 raft_consensus.cc:2243] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.601380 20341 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 election: Requested vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:17.602141 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:17.602728 11330 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:16:17.603169 11222 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.603631 11222 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:17.607486 20428 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:17.607569 20128 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:16:17.611469 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:17.612071 11935 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:16:17.622288 11252 raft_consensus.cc:1275] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:16:17.623968 20165 consensus_queue.cc:1048] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:16:17.629254 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.649369 10011 raft_consensus.cc:2272] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.650569 20320 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.651309 20320 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.652482 10011 tablet_replica.cc:333] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.653462 11257 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:16:17.654361 20320 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:17.655233 20332 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:17.663727 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.664420 11249 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:17.670452 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:17.671212 11935 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 32.
I20260709 12:16:17.671352 20441 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:17.671903 20441 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.672375 20441 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:17.683363 11330 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016411114c6d42039bd04025378dfc98 in term 32.
I20260709 12:16:17.686235 10011 raft_consensus.cc:2243] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.686969 10011 raft_consensus.cc:2272] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:17.687543 11296 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.692636 11147 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b; no voters: 
I20260709 12:16:17.694067 20341 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:16:17.694419 10011 tablet_replica.cc:333] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.694911 20341 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 32 LEADER]: Becoming Leader. State: Replica: 016411114c6d42039bd04025378dfc98, State: Running, Role: LEADER
I20260709 12:16:17.695472 10011 raft_consensus.cc:2243] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.696027 20341 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.696738 10011 raft_consensus.cc:2272] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.699491 20354 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.700028 20354 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.702188 20354 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:17.714778 20441 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.717458 20441 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:17.722188 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.731487 20341 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.735261 11252 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:17.738365 20341 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.743481 20341 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:17.744032 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:17.745949 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01e75f61a948459a884630690e51c686" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.747560 11147 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:17.748960 20341 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:17.757680 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab915731e0347ae8353a7f300c44401" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:17.759975 11252 raft_consensus.cc:3060] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:16:17.762624 20468 log.cc:927] Time spent T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:16:17.736366 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.756078 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:17.770414 11252 raft_consensus.cc:2468] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:16:17.772624 20425 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.773242 20425 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.774796 11076 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:17.775586 20425 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:17.779204 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.779889 11248 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
W20260709 12:16:17.770411 11296 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.769399 20320 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:17.782573 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:17.786135 11296 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:17.794329 20355 raft_consensus.cc:2804] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:17.793879 20425 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:16:17.797062 11296 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.796285 20355 raft_consensus.cc:697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
W20260709 12:16:17.793299 11900 leader_election.cc:336] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.782752 20320 raft_consensus.cc:493] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.793776 20277 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.799762 20320 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:17.804978 20277 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.806197 20355 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.807253 20277 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:17.797497 20425 raft_consensus.cc:493] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.810078 20425 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:17.812947 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.813936 11257 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
I20260709 12:16:17.829043 11330 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:16:17.834877 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:17.836398 20332 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
W20260709 12:16:17.852378 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.853469 20304 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:17.854003 20304 raft_consensus.cc:493] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.854391 20304 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:17.854626 10011 tablet_replica.cc:333] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: stopping tablet replica
W20260709 12:16:17.859913 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:17.860136 20470 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.863763 20470 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:17.864537 10011 raft_consensus.cc:2243] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.865283 10011 pending_rounds.cc:70] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Trying to abort 1 pending ops.
I20260709 12:16:17.865530 10011 pending_rounds.cc:77] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305622924650754048 op_type: NO_OP noop_request { }
I20260709 12:16:17.865952 10011 raft_consensus.cc:2889] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:17.866331 10011 raft_consensus.cc:2272] T 5df07764623640ce8ec29ad695812b7c P 016411114c6d42039bd04025378dfc98 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.873842 10011 tablet_replica.cc:333] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.875203 10011 raft_consensus.cc:2243] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.877127 10011 raft_consensus.cc:2272] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:17.866308 11071 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 243) took 29863 ms (client timeout 15000 ms). Trace:
I20260709 12:16:17.886077 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:17.899204 20165 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.903946 20165 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.906213 20165 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:17.910699 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 77997.7us
W20260709 12:16:17.910847 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 76094.8us
I20260709 12:16:17.913461 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:17.913813 11900 leader_election.cc:336] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.914429 11257 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 23.
I20260709 12:16:17.922363 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:17.924818 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:17.927188 11900 leader_election.cc:336] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:17.932317 20320 raft_consensus.cc:515] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:17.878439 11071 rpcz_store.cc:269] 0709 12:15:47.997168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.110706 (+113538us) service_pool.cc:224] Handling call
0709 12:16:17.849627 (+29738921us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:17.266990 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:17.267750 (+   760us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:17.267806 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:16:17.267828 (+    22us) tablet.cc:662] Decoding operations
0709 12:16:17.269116 (+  1288us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:17.269233 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:17.269271 (+    38us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:16:17.271641 (+  2370us) tablet.cc:701] Row locks acquired
0709 12:16:17.271691 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:16:17.271880 (+   189us) write_op.cc:270] Start()
0709 12:16:17.272031 (+   151us) write_op.cc:276] Timestamp: P: 1783599377271811 usec, L: 0
0709 12:16:17.272054 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:16:17.339014 (+ 66960us) log.cc:844] Serialized 5246 byte log entry
0709 12:16:17.420076 (+ 81062us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b1800122f80. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35cd2a6 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ecc4c7ab1 00007f3ecc4d4281 00007f3ec35ff3f3 00007f3ec36017c1
0709 12:16:17.481799 (+ 61723us) op_driver.cc:464] REPLICATION: finished
0709 12:16:17.534013 (+ 52214us) write_op.cc:301] APPLY: starting
0709 12:16:17.534116 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:17.572050 (+ 37934us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:17.572099 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:17.636311 (+ 64212us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:17.636601 (+   290us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:16:17.636668 (+    67us) write_op.cc:312] APPLY: finished
0709 12:16:17.637033 (+   365us) log.cc:844] Serialized 121 byte log entry
0709 12:16:17.637226 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:17.637381 (+   155us) write_op.cc:454] Released schema lock
0709 12:16:17.637433 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:17.637898 (+   465us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:17.654677 (+ 16779us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16313,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":51558,"apply.run_cpu_time_us":17642,"apply.run_wall_time_us":103869,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58571,"num_ops":13,"prepare.queue_time_us":59311,"prepare.run_cpu_time_us":42405,"prepare.run_wall_time_us":142843,"raft.queue_time_us":58748,"raft.run_cpu_time_us":27387,"raft.run_wall_time_us":40272,"replication_time_us":209516,"spinlock_wait_cycles":8889344,"thread_start_us":190919,"threads_started":4,"wal-append.queue_time_us":70583}]]}
I20260709 12:16:17.921239 20470 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:17.942819 20388 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:17.943311 20388 raft_consensus.cc:493] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:17.943681 20388 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:17.951514 10011 tablet_replica.cc:333] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:17.952548 11331 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:16:17.954365 20335 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:16:17.960695 11148 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:17.962417 20470 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:17.967849 20425 raft_consensus.cc:515] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.981756 20304 raft_consensus.cc:515] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:17.984632 20304 leader_election.cc:290] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:17.982545 20320 leader_election.cc:290] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:17.987272 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddf8cca10e1244d8882abd13fa81113a" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:17.996098 11248 raft_consensus.cc:3060] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:17.989811 10011 raft_consensus.cc:2243] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:18.005779 10011 raft_consensus.cc:2272] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.009761 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49a939bed404bfcb2538b5331071d3b" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:18.010519 11249 raft_consensus.cc:3060] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:16:18.017958 11900 leader_election.cc:336] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:18.020696 11900 leader_election.cc:336] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.021588 20425 leader_election.cc:290] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.036123 11249 raft_consensus.cc:2468] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 24.
I20260709 12:16:18.037528 11935 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:16:18.039894 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35c7899b1a314ae1b67824dfee6ff091" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:18.042330 10011 tablet_replica.cc:333] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.043591 11296 leader_election.cc:304] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:18.044184 11249 raft_consensus.cc:3060] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:18.046788 10011 raft_consensus.cc:2243] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.047607 10011 raft_consensus.cc:2272] T 73b50321123c433f91c5d0972791cf37 P 016411114c6d42039bd04025378dfc98 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.054296 20476 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.055002 20473 raft_consensus.cc:2804] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:18.055612 20476 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.056133 20473 raft_consensus.cc:697] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:18.043036 11075 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 22 to 24, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 24 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.057380 20473 consensus_queue.cc:237] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.058773 11075 catalog_manager.cc:5697] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 4 to 13, term changed from 21 to 23, VOTER bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) evicted. New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.061826 20335 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
W20260709 12:16:18.063545 11076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 246) took 66462 ms (client timeout 15000 ms). Trace:
W20260709 12:16:18.072489 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:18.061121 11075 catalog_manager.cc:5697] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 25, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 25 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.079604 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.080722 11249 raft_consensus.cc:2468] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:16:18.082726 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fdc8cd3021d4b678786056187782c16" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:18.082890 20476 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:18.088917 11148 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:18.090137 20476 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:18.066421 11076 rpcz_store.cc:269] 0709 12:15:11.601618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.611488 (+  9870us) service_pool.cc:224] Handling call
0709 12:16:18.063468 (+66451980us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:17.880894 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:17.881482 (+   588us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:17.881526 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:16:17.881542 (+    16us) tablet.cc:662] Decoding operations
0709 12:16:17.881912 (+   370us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:17.881971 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:17.882001 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:16:17.882180 (+   179us) tablet.cc:701] Row locks acquired
0709 12:16:17.882192 (+    12us) write_op.cc:260] PREPARE: finished
0709 12:16:17.900318 (+ 18126us) write_op.cc:270] Start()
0709 12:16:17.903392 (+  3074us) write_op.cc:276] Timestamp: P: 1783599377899778 usec, L: 0
0709 12:16:17.903419 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:16:17.955536 (+ 52117us) log.cc:844] Serialized 541 byte log entry
0709 12:16:17.957860 (+  2324us) op_driver.cc:464] REPLICATION: finished
0709 12:16:17.958225 (+   365us) write_op.cc:301] APPLY: starting
0709 12:16:17.958305 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:17.959135 (+   830us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:17.959233 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:17.959940 (+   707us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:17.960086 (+   146us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:16:17.960134 (+    48us) write_op.cc:312] APPLY: finished
0709 12:16:17.960321 (+   187us) log.cc:844] Serialized 25 byte log entry
0709 12:16:17.960492 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:17.960575 (+    83us) write_op.cc:454] Released schema lock
0709 12:16:17.960619 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:17.961011 (+   392us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:17.961048 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":110447,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":2742,"apply.run_wall_time_us":2742,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4188,"num_ops":1,"prepare.queue_time_us":7932,"prepare.run_cpu_time_us":3149,"prepare.run_wall_time_us":75008,"replication_time_us":47752}]]}
I20260709 12:16:18.077899 11075 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 28 to 30, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 30 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.103830 11075 catalog_manager.cc:5697] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 21 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.106875 10011 tablet_replica.cc:333] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.107873 10011 raft_consensus.cc:2243] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.108527 10011 raft_consensus.cc:2272] T a629d78b61074c2cbd007926bd78fa29 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.093341 11248 raft_consensus.cc:2468] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:16:18.112597 11075 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 24 to 28, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 28 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:18.113855 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:18.102881 20388 raft_consensus.cc:515] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.127610 11900 leader_election.cc:304] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:16:18.130010 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 54802.8us
I20260709 12:16:18.130450 11075 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 25, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 25 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:18.135442 11222 consensus_peers.cc:597] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.137710 10011 tablet_replica.cc:333] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.138692 10011 raft_consensus.cc:2243] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.136023 20478 raft_consensus.cc:2804] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:18.139396 10011 raft_consensus.cc:2272] T d97e23379bda4a1b9ad1e2eabbf4d82f P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.135519 11900 leader_election.cc:304] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:16:18.143435 11296 leader_election.cc:336] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.144549 11075 catalog_manager.cc:5697] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 21 to 22, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 22 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.148185 11075 catalog_manager.cc:5697] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 21 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 25 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:18.146386 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:18.151371 11075 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 23, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 23 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.150440 10011 tablet_replica.cc:333] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.149609 20277 raft_consensus.cc:2804] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:18.156188 20277 raft_consensus.cc:697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:18.157145 20277 consensus_queue.cc:237] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:18.164429 11065 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 5df07764623640ce8ec29ad695812b7c with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 4 but the committed config has opid_index of 13
I20260709 12:16:18.143518 20478 raft_consensus.cc:697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:18.165585 10011 raft_consensus.cc:2243] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.166652 20478 consensus_queue.cc:237] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.167841 10011 raft_consensus.cc:2272] T 6c6a39d516e943a9b2c8c4213ac11902 P 016411114c6d42039bd04025378dfc98 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.176105 10011 tablet_replica.cc:333] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.185354 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6b498179b4c96b8b0be6240a36efb" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:18.187386 11248 raft_consensus.cc:3060] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:16:18.185534 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:18.193287 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:18.196125 11248 raft_consensus.cc:2468] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 24.
I20260709 12:16:18.198174 11900 leader_election.cc:304] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:18.200083 11075 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.200807 10011 raft_consensus.cc:2243] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.201660 11075 catalog_manager.cc:5697] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.203395 11075 catalog_manager.cc:5697] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 24, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 24 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.204711 11075 catalog_manager.cc:5697] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.206403 11075 catalog_manager.cc:5697] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 29. New cstate: current_term: 29 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.209347 10011 pending_rounds.cc:70] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Trying to abort 1 pending ops.
I20260709 12:16:18.183334 20388 leader_election.cc:290] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.213537 11930 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:16:18.210505 10011 pending_rounds.cc:77] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 28 index: 19 } timestamp: 7305623010072489984 op_type: NO_OP noop_request { }
I20260709 12:16:18.210253 11075 catalog_manager.cc:5697] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.215441 10011 raft_consensus.cc:2889] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:18.215895 10011 raft_consensus.cc:2272] T e5f804d45235415fad0f1442ea04aa47 P 016411114c6d42039bd04025378dfc98 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.207926 20165 raft_consensus.cc:2804] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:18.220511 20476 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 12:16:18.228349 20165 raft_consensus.cc:697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
W20260709 12:16:18.230872 11900 leader_election.cc:336] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.216439 11075 catalog_manager.cc:5697] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 24 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 25 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.229768 20165 consensus_queue.cc:237] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.236439 11075 catalog_manager.cc:5697] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.239993 11075 catalog_manager.cc:5697] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 21, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 21 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.240993 11075 catalog_manager.cc:5697] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.242378 11075 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:18.243903 11075 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 20 to 26. New cstate: current_term: 26 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:18.259755 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.268739 10011 tablet_replica.cc:333] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.269889 10011 raft_consensus.cc:2243] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.270525 10011 raft_consensus.cc:2272] T 28d2d114b03e403c83f7001bc117434e P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.274955 10011 tablet_replica.cc:333] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.275784 10011 raft_consensus.cc:2243] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:16:18.277187 11330 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 FOLLOWER]: Refusing update from remote peer 016411114c6d42039bd04025378dfc98: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:16:18.277395 10011 pending_rounds.cc:70] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Trying to abort 1 pending ops.
I20260709 12:16:18.277796 10011 pending_rounds.cc:77] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 32 index: 20 } timestamp: 7305623051087310848 op_type: NO_OP noop_request { }
I20260709 12:16:18.278152 10011 raft_consensus.cc:2889] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 32 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:18.279449 10011 raft_consensus.cc:2272] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:18.283854 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:18.288205 20476 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.290752 20476 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.292670 20358 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.293171 11248 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:16:18.293370 20476 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:18.293254 20358 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.295269 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:18.297070 20358 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:18.298444 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:18.302376 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "016411114c6d42039bd04025378dfc98" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.303645 11147 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 016411114c6d42039bd04025378dfc98; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:18.305410 20473 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:18.307622 20476 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:18.319465 10011 tablet_replica.cc:333] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.320508 10011 raft_consensus.cc:2243] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.321280 10011 raft_consensus.cc:2272] T c21b703f1a4a445a8505bb01530a8439 P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:18.329222 11222 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.329972 11222 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:18.330756 20358 raft_consensus.cc:2749] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:18.361693 20358 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:18.361924 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:18.362296 20358 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.365976 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:18.369047 20358 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:18.369678 11222 leader_election.cc:336] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.370165 10011 tablet_replica.cc:333] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.370177 11222 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:18.371212 10011 raft_consensus.cc:2243] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.371815 10011 raft_consensus.cc:2272] T d5a029ce312749bdaf2113d2823e993e P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.372786 20128 raft_consensus.cc:2749] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:18.384513 10011 tablet_replica.cc:333] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.391350 10011 raft_consensus.cc:2243] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.404289 10011 pending_rounds.cc:70] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Trying to abort 1 pending ops.
I20260709 12:16:18.404608 10011 pending_rounds.cc:77] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 24 index: 15 } timestamp: 7305622967700336640 op_type: NO_OP noop_request { }
I20260709 12:16:18.404973 10011 raft_consensus.cc:2889] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:18.405390 10011 raft_consensus.cc:2272] T 2fdc8cd3021d4b678786056187782c16 P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.420462 10011 tablet_replica.cc:333] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.442696 10011 raft_consensus.cc:2243] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.485198 10011 raft_consensus.cc:2272] T 25983a67e23a4c708ae496061b992207 P 016411114c6d42039bd04025378dfc98 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.488653 10011 tablet_replica.cc:333] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.489574 10011 raft_consensus.cc:2243] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.496297 20469 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.496917 20469 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.560761 20469 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.561540 10011 pending_rounds.cc:70] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Trying to abort 1 pending ops.
I20260709 12:16:18.561914 10011 pending_rounds.cc:77] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 23 index: 17 } timestamp: 7305622926530019328 op_type: NO_OP noop_request { }
I20260709 12:16:18.562290 10011 raft_consensus.cc:2889] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:18.562666 10011 raft_consensus.cc:2272] T 39571de4248f4cfd85f9b1062a776cc8 P 016411114c6d42039bd04025378dfc98 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:18.567866 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 60546.1us
I20260709 12:16:18.571005 20355 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.571707 20355 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.574010 20355 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:18.574704 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 73422.9us
I20260709 12:16:18.576936 11248 raft_consensus.cc:1275] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:16:18.586787 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:18.583997 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.596779 10011 tablet_replica.cc:333] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.592855 11072 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 23 to 25, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 25 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.615856 11248 raft_consensus.cc:1275] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:16:18.616696 11072 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 23 to 26, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 26 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.619067 11072 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 19 to 22, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 22 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.619412 20478 consensus_queue.cc:1048] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:16:18.622249 11072 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:18.623385 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:18.624599 11072 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.627436 11072 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.630173 11248 raft_consensus.cc:1275] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:16:18.631317 11072 catalog_manager.cc:5697] T d1db2221cc614cac91fdaa49f8d6cb61 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.632931 20478 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.633194 11072 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 23 to 28, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 28 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.633666 20478 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.635213 20477 consensus_queue.cc:1048] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:18.643663 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.648728 20469 consensus_queue.cc:1048] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:18.649909 10011 raft_consensus.cc:2243] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.650799 10011 pending_rounds.cc:70] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Trying to abort 2 pending ops.
I20260709 12:16:18.651140 10011 pending_rounds.cc:77] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 26 index: 15 } timestamp: 7305622957210656768 op_type: NO_OP noop_request { }
I20260709 12:16:18.651553 10011 raft_consensus.cc:2889] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:18.651659 11075 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 175) took 2109 ms. Trace:
I20260709 12:16:18.651959 10011 pending_rounds.cc:77] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98: Aborting op as it isn't in flight: id { term: 24 index: 14 } timestamp: 7305622927292477440 op_type: NO_OP noop_request { }
I20260709 12:16:18.652522 10011 raft_consensus.cc:2889] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:18.652992 20478 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.657464 10011 raft_consensus.cc:2272] T 01e75f61a948459a884630690e51c686 P 016411114c6d42039bd04025378dfc98 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.661186 10011 tablet_replica.cc:333] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.667618 10011 raft_consensus.cc:2243] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:18.668543 10011 raft_consensus.cc:2272] T ab7767a1cf4e4b5493ca9acc8daeabab P 016411114c6d42039bd04025378dfc98 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.652181 11075 rpcz_store.cc:276] 0709 12:16:16.532957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:16.533146 (+   189us) service_pool.cc:224] Handling call
0709 12:16:16.624765 (+ 91619us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:16.713312 (+ 88547us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b5400260110. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec3487db1 00007f3ec35c8640 00007f3ecc49f846 00007f3ecc5a990e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:18.641934 (+1928622us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:18.254455 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:18.255078 (+   623us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:18.255118 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:16:18.255135 (+    17us) tablet.cc:662] Decoding operations
0709 12:16:18.257096 (+  1961us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:18.257167 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:18.257201 (+    34us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:16:18.258876 (+  1675us) tablet.cc:701] Row locks acquired
0709 12:16:18.258904 (+    28us) write_op.cc:260] PREPARE: finished
0709 12:16:18.259061 (+   157us) write_op.cc:270] Start()
0709 12:16:18.259185 (+   124us) write_op.cc:276] Timestamp: P: 1783599378259008 usec, L: 0
0709 12:16:18.259212 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:16:18.261969 (+  2757us) log.cc:844] Serialized 8769 byte log entry
0709 12:16:18.266149 (+  4180us) op_driver.cc:464] REPLICATION: finished
0709 12:16:18.266630 (+   481us) write_op.cc:301] APPLY: starting
0709 12:16:18.266747 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:18.342883 (+ 76136us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:18.342994 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:18.409565 (+ 66571us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:18.409981 (+   416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 12:16:18.421050 (+ 11069us) write_op.cc:312] APPLY: finished
0709 12:16:18.421539 (+   489us) log.cc:844] Serialized 196 byte log entry
0709 12:16:18.421754 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:18.426470 (+  4716us) write_op.cc:454] Released schema lock
0709 12:16:18.426533 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:18.436273 (+  9740us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:18.436312 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13887,"reported_tablets":32,"spinlock_wait_cycles":67486848,"child_traces":[["op",{"apply.queue_time_us":251,"apply.run_cpu_time_us":28166,"apply.run_wall_time_us":169555,"cfile_cache_hit":88,"cfile_cache_hit_bytes":92136,"num_ops":22,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":5785,"prepare.run_wall_time_us":7822,"replication_time_us":6783}]]}
W20260709 12:16:18.641995 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:18.672494 11072 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 26 to 27, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 27 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:18.676503 11900 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.678429 11072 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 20 to 21, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 21 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.681352 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.682140 11252 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:16:18.684062 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:18.680745 20355 raft_consensus.cc:493] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:18.688537 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.679752 20358 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.691222 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:18.692612 10011 tablet_replica.cc:333] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98: stopping tablet replica
I20260709 12:16:18.688371 20388 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:16:18.699877 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.703354 11072 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 21 to 22, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 22 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:18.700419 20320 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:18.702502 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:18.690169 20355 raft_consensus.cc:515] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } attrs { replace: true } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.702669 20388 raft_consensus.cc:493] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.698734 20358 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.711350 20388 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:18.712690 20473 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:18.713048 20358 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:18.713454 20473 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.714000 20473 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.706840 10011 raft_consensus.cc:2243] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:18.727767 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:18.728394 20473 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.728819 11331 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
I20260709 12:16:18.735355 10011 raft_consensus.cc:2272] T 6d225f8a873c4abe992018eb9353664f P 016411114c6d42039bd04025378dfc98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:18.739904 20355 leader_election.cc:290] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:18.742766 11222 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.710248 11072 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:18.759994 11072 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 19 to 20, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 20 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.761505 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:18.767899 11072 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 17 to 22, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 22 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.776868 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.777519 11248 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:16:18.781195 11072 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P 016411114c6d42039bd04025378dfc98 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.782297 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
W20260709 12:16:18.783787 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:18.785097 11930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df07764623640ce8ec29ad695812b7c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:18.787498 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.776688 20128 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:18.792263 20128 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.792706 20128 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:18.807490 20355 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.808974 20355 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.815593 20355 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:18.822872 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.833160 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.838418 20493 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.838999 20493 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.850847 20493 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:18.859031 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 81284.1us
W20260709 12:16:18.858224 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.865528 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:18.872156 20388 raft_consensus.cc:515] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.874608 20388 leader_election.cc:290] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.876296 11248 raft_consensus.cc:1275] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 23 index: 20. Preceding OpId from leader: term: 24 index: 21. (index mismatch)
I20260709 12:16:18.877345 20493 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:18.877789 20493 raft_consensus.cc:493] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.878173 20493 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:18.881999 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f736e38db6b45bfa87e35af564bffc1" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:18.882845 11248 raft_consensus.cc:3060] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:18.864317 20128 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.884457 11296 leader_election.cc:304] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:18.906848 11900 leader_election.cc:336] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.909927 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df07764623640ce8ec29ad695812b7c with delete_type TABLET_DATA_TOMBSTONED (TS bc92f9d2c6c5477182fa33eba4368ec1 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:40846
I20260709 12:16:18.912986 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.913295 20473 raft_consensus.cc:2749] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:18.908344 11248 raft_consensus.cc:2468] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
W20260709 12:16:18.918449 11296 leader_election.cc:336] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.900542 20165 consensus_queue.cc:1048] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:16:18.901543 20355 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.919308 20128 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:18.919723 20355 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:18.921137 11296 leader_election.cc:336] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.922138 20355 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.902208 20431 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.922909 20431 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.925107 20431 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:18.926751 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:18.927084 20493 raft_consensus.cc:515] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.934324 11900 leader_election.cc:304] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:16:18.934096 11222 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.936107 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b8260131ab49ecb2df9631e5704096" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:18.936765 11331 raft_consensus.cc:3060] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:18.937006 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:18.937667 11330 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 29.
W20260709 12:16:18.939441 11222 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.943480 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
W20260709 12:16:18.944293 11296 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.946106 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:18.940457 20165 raft_consensus.cc:2804] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:18.947880 20128 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.951088 20128 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.952787 20358 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:18.953223 20358 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.953958 20358 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:18.953158 20128 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:18.959110 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:18.959802 11330 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
W20260709 12:16:18.960786 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.964586 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:18.961668 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:18.961987 20355 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:18.974836 20497 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:18.975260 20497 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:18.975630 20497 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:18.975803 20493 leader_election.cc:290] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:18.968797 20165 raft_consensus.cc:697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:18.979280 20165 consensus_queue.cc:237] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:18.983830 20494 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:18.984421 20497 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:18.989713 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:18.990464 11330 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:16:18.996270 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:18.996896 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fc47889c90446dd927b2ac3010d58fd" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:18.997025 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:18.997586 11248 raft_consensus.cc:3060] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:19.003340 20503 tablet_replica.cc:333] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:19.003952 20497 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.005000 20503 raft_consensus.cc:2243] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:19.009889 11330 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 28.
I20260709 12:16:19.013448 11223 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:19.014335 20501 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:19.014819 20501 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:19.015718 20501 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.017980 11248 raft_consensus.cc:2468] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 26.
I20260709 12:16:19.025195 11331 raft_consensus.cc:2468] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:16:19.026944 11223 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:19.028126 20504 raft_consensus.cc:2804] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:19.028797 20504 raft_consensus.cc:697] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:19.030073 20504 consensus_queue.cc:237] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.026583 11296 leader_election.cc:304] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:19.038103 20493 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.038715 20493 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:19.040920 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 96313.6us
I20260709 12:16:19.010939 20494 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:19.049674 20503 raft_consensus.cc:2272] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:19.050375 20493 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:19.051311 20358 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.053671 20358 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.055390 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:19.055650 20473 raft_consensus.cc:2804] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:19.056198 20473 raft_consensus.cc:697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:19.058969 20503 ts_tablet_manager.cc:1916] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:19.063540 20473 consensus_queue.cc:237] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.066571 20128 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.067278 20128 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.074120 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd765747f9648049b905c70696b206d" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:19.075220 11331 raft_consensus.cc:3060] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:19.073293 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:19.088406 20494 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.089066 20494 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:19.089917 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:19.091797 20494 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:19.097589 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:19.069502 20128 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.098272 11248 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 29.
W20260709 12:16:19.102221 11296 leader_election.cc:336] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.110901 11331 raft_consensus.cc:2468] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 30.
W20260709 12:16:19.109131 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.111887 11223 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:19.112620 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:19.123083 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:19.113787 20431 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.142091 20431 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.144394 20431 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.122506 20506 raft_consensus.cc:2804] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:19.148028 20503 ts_tablet_manager.cc:1929] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.12
I20260709 12:16:19.148653 20503 log.cc:1199] T 5df07764623640ce8ec29ad695812b7c P bc92f9d2c6c5477182fa33eba4368ec1: Deleting WAL directory at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/wals/5df07764623640ce8ec29ad695812b7c
I20260709 12:16:19.152441 20506 raft_consensus.cc:697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 30 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
W20260709 12:16:19.152698 11222 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.153579 20506 consensus_queue.cc:237] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:19.154767 11222 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.155889 20128 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:19.156217 11222 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.156534 20128 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.158906 20128 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:19.159761 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:19.166198 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 5df07764623640ce8ec29ad695812b7c (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:16:19.167043 20469 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:19.167541 20469 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.167984 20469 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:16:19.168330 11222 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.168825 11222 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:19.172024 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:19.172787 20431 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:19.173460 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:19.174292 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 25.
W20260709 12:16:19.176167 11222 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:19.176836 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:19.177838 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:19.179589 20358 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:19.179982 20358 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.180374 20358 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:19.188000 20469 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.191480 20358 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.194250 20358 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.195919 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35bce44584bf45b68e6ff125c8f30655" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:19.196729 11331 raft_consensus.cc:3060] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:19.202054 20469 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:19.201647 11222 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.224644 11331 raft_consensus.cc:2468] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
I20260709 12:16:19.236147 11223 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
W20260709 12:16:19.255731 11222 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.263554 20497 raft_consensus.cc:2804] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:16:19.264400 20497 raft_consensus.cc:697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:19.264714 20431 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.265237 20431 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.267511 20431 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.265359 20497 consensus_queue.cc:237] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.122025 20494 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:19.285454 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:19.286471 11331 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 26.
W20260709 12:16:19.289229 11222 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:19.296973 11296 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.306418 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:19.307080 11249 raft_consensus.cc:3060] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:19.308214 11330 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 22.
I20260709 12:16:19.321074 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:19.330302 20431 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.333169 20431 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.335405 20431 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.326314 20512 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:19.339031 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:19.342171 11249 raft_consensus.cc:2468] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 30.
I20260709 12:16:19.343633 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:19.342530 20512 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.346911 20128 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:19.353978 20128 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
W20260709 12:16:19.357209 20509 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.002s
I20260709 12:16:19.357959 20128 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:19.351367 20512 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:16:19.370584 20510 log.cc:927] Time spent T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
W20260709 12:16:19.373358 20518 log.cc:927] Time spent T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:16:19.381304 11296 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:19.384232 20515 raft_consensus.cc:2804] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:19.384838 20515 raft_consensus.cc:697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:19.385846 20515 consensus_queue.cc:237] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:19.391696 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:19.403127 20469 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.403818 20469 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.406046 20469 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:19.410648 11222 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:19.412456 11222 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.412942 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:19.413787 20358 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:19.421768 11248 raft_consensus.cc:1275] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:16:19.424512 20512 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:19.427945 11296 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.428033 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dedbfb6dda024d1e81b2740f108080af" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:19.428752 11329 raft_consensus.cc:3060] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:16:19.429081 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:19.445942 20128 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.447486 20512 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.447976 11248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:19.448760 11248 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 27.
I20260709 12:16:19.457393 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:19.458461 20515 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:16:19.458931 20515 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.459332 20515 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:19.462344 20128 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:19.463740 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06a9ec0400af47eeaffc85f3f1d38de4" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:19.464563 11330 raft_consensus.cc:3060] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:19.465179 11222 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:19.470646 11222 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.474340 11330 raft_consensus.cc:2468] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 23.
I20260709 12:16:19.477952 20478 consensus_queue.cc:1048] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:19.486033 11223 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:19.488129 20501 raft_consensus.cc:2804] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:19.488723 20501 raft_consensus.cc:697] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:19.489683 20501 consensus_queue.cc:237] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.506160 11329 raft_consensus.cc:2468] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fecfc03de52a4308bc9b045cad5aac1e in term 27.
I20260709 12:16:19.513605 11223 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:19.516819 20431 raft_consensus.cc:2804] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:19.517705 20431 raft_consensus.cc:697] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Becoming Leader. State: Replica: fecfc03de52a4308bc9b045cad5aac1e, State: Running, Role: LEADER
I20260709 12:16:19.518680 20431 consensus_queue.cc:237] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:19.533133 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:19.534144 11296 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:19.541908 11075 catalog_manager.cc:5697] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 24 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 25 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.544340 11075 catalog_manager.cc:5697] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 22 to 30. New cstate: current_term: 30 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.546458 11075 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 28 to 32. New cstate: current_term: 32 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.548276 11075 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 30 to 31, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 31 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.550196 11075 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 25 to 29, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 29 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.478722 20515 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:19.558389 11900 connection.cc:769] client connection to 127.9.198.193:45115 send error: Network error: sendmsg error: Connection reset by peer (error 104)
W20260709 12:16:19.569078 11296 connection.cc:769] client connection to 127.9.198.193:45115 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 12:16:19.571398 11222 connection.cc:573] client connection to 127.9.198.193:45115 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:16:19.574961 20515 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:19.575770 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: sendmsg error: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:19.576380 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: sendmsg error: Connection reset by peer (error 104). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:19.580551 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:19.599771 11075 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 28 to 29, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 29 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.602113 11075 catalog_manager.cc:5697] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 29 to 32, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 32 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.609383 11075 catalog_manager.cc:5697] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 24, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 24 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.614764 11249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:19.615541 11249 raft_consensus.cc:3060] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:16:19.617157 11075 catalog_manager.cc:5697] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 28, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.619113 11075 catalog_manager.cc:5697] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 23 to 26, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 26 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.620608 11075 catalog_manager.cc:5697] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 27 to 30, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 30 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:19.622634 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:19.623167 11222 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:19.622737 11075 catalog_manager.cc:5697] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 25 to 26, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 26 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.624785 11249 raft_consensus.cc:2468] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 28.
I20260709 12:16:19.624505 11075 catalog_manager.cc:5697] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 16 to 24, leader changed from 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 24 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:19.626040 11075 catalog_manager.cc:5697] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 26 to 27, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 27 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:19.627403 11072 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 263) took 21561 ms (client timeout 15000 ms). Trace:
W20260709 12:16:19.629947 11072 rpcz_store.cc:269] 0709 12:15:58.072918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.088034 (+ 15116us) service_pool.cc:224] Handling call
0709 12:15:59.600246 (+1512212us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:19.627105 (+20026859us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:18.918028 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:18.918596 (+   568us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:18.918632 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:16:18.918650 (+    18us) tablet.cc:662] Decoding operations
0709 12:16:18.957148 (+ 38498us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:18.957234 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:18.957261 (+    27us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:16:18.958523 (+  1262us) tablet.cc:701] Row locks acquired
0709 12:16:18.958542 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:16:18.958688 (+   146us) write_op.cc:270] Start()
0709 12:16:18.958803 (+   115us) write_op.cc:276] Timestamp: P: 1783599378958639 usec, L: 0
0709 12:16:18.958824 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:16:18.991709 (+ 32885us) log.cc:844] Serialized 5969 byte log entry
0709 12:16:19.146036 (+154327us) op_driver.cc:464] REPLICATION: finished
0709 12:16:19.290616 (+144580us) write_op.cc:301] APPLY: starting
0709 12:16:19.290706 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:19.320588 (+ 29882us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:19.320691 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:19.428627 (+107936us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:19.428911 (+   284us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:16:19.428957 (+    46us) write_op.cc:312] APPLY: finished
0709 12:16:19.429329 (+   372us) log.cc:844] Serialized 139 byte log entry
0709 12:16:19.429502 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:19.429683 (+   181us) write_op.cc:454] Released schema lock
0709 12:16:19.429740 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:19.430125 (+   385us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:19.430166 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":50,"spinlock_wait_cycles":108811648,"child_traces":[["op",{"apply.queue_time_us":144298,"apply.run_cpu_time_us":21048,"apply.run_wall_time_us":139491,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66947,"num_ops":15,"prepare.queue_time_us":100753,"prepare.run_cpu_time_us":5366,"prepare.run_wall_time_us":74039,"replication_time_us":165172,"thread_start_us":169220,"threads_started":2}]]}
W20260709 12:16:19.634701 11066 connection.cc:466] server connection from 127.0.0.1:50196 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 263) could send its response
W20260709 12:16:19.636744 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:19.638108 11296 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:19.638909 20515 raft_consensus.cc:2804] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:16:19.639513 20515 raft_consensus.cc:697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:19.644472 11249 raft_consensus.cc:1275] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:16:19.649231 20469 consensus_queue.cc:1048] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:19.649338 20515 consensus_queue.cc:237] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:19.650353 11328 raft_consensus.cc:1275] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:16:19.654384 20501 consensus_queue.cc:1048] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:19.660593 11329 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:16:19.669893 20501 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
W20260709 12:16:19.674840 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:19.675487 11222 consensus_peers.cc:597] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:19.680698 11296 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:19.681707 11222 consensus_peers.cc:597] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:19.683866 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:19.684684 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:19.721414 11331 raft_consensus.cc:1275] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:16:19.723199 11330 raft_consensus.cc:1275] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:16:19.728338 20478 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.729445 20497 consensus_queue.cc:1048] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:16:19.729290 20478 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:19.728226 20431 consensus_queue.cc:1048] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:19.739711 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:19.740613 11324 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 32.
I20260709 12:16:19.742230 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:19.749184 20498 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:16:19.750613 20498 raft_consensus.cc:493] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.751101 20498 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:16:19.760293 11900 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:19.744824 20478 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:19.819610 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:19.822384 20128 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:19.823020 20128 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:19.832852 20128 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:19.842689 11324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:19.842260 11222 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:19.809691 20498 raft_consensus.cc:515] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:19.856217 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:19.871028 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:19.861460 20358 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:19.875475 20530 log.cc:927] Time spent T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 12:16:19.924031 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 56007.8us
W20260709 12:16:19.938412 11222 consensus_peers.cc:597] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:19.948215 11900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111) [suppressed 320 similar messages]
I20260709 12:16:19.954723 11249 raft_consensus.cc:1275] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 29 index: 14. Preceding OpId from leader: term: 30 index: 15. (index mismatch)
I20260709 12:16:19.965857 11330 raft_consensus.cc:1275] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:16:19.998164 20497 consensus_queue.cc:1048] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:16:20.028113 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:20.035061 20494 consensus_queue.cc:1048] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:20.043730 20498 leader_election.cc:290] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:20.046387 11331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99332ff05c3e46579ef9ff825e42a628" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:20.047336 11331 raft_consensus.cc:3060] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:16:20.049151 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:20.049853 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:20.054119 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:20.059466 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:20.065950 11900 leader_election.cc:336] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:20.074944 20498 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:20.075588 20498 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:20.078337 20320 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:20.079031 20320 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:20.098821 20498 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:20.099865 20320 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:20.102073 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:20.102838 11322 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
W20260709 12:16:20.105890 11222 consensus_peers.cc:597] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:20.106494 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:20.108631 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:20.110539 11257 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
W20260709 12:16:20.110988 11222 consensus_peers.cc:597] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:20.113402 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:20.114199 11931 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 25.
I20260709 12:16:20.134253 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:20.135273 11257 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 25.
W20260709 12:16:20.147372 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:20.147929 11296 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:20.154417 11257 raft_consensus.cc:1275] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:16:20.158169 11328 raft_consensus.cc:1275] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:16:20.154342 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:20.161255 20498 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:20.161734 20498 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:20.162142 20498 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:20.166407 20536 consensus_queue.cc:1048] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:16:20.165338 11900 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:20.166853 11331 raft_consensus.cc:2468] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 33.
I20260709 12:16:20.172482 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:20.175858 20473 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:20.176291 20473 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 016411114c6d42039bd04025378dfc98)
I20260709 12:16:20.176656 20473 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:20.178216 20320 consensus_queue.cc:1048] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:16:20.188521 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:20.191847 20498 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:20.197754 11901 leader_election.cc:304] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:20.203828 20533 raft_consensus.cc:2804] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:16:20.204566 20533 raft_consensus.cc:697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 33 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:20.205646 20533 consensus_queue.cc:237] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:20.220919 20473 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:20.223893 20498 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:20.225153 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:20.226387 11257 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:20.230726 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:20.231415 11252 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:20.250813 11328 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:16:20.255687 11252 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:16:20.257436 11329 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 786851d4227e4dd89b75ccdea0b1f03b in current term 26: Already voted for candidate bc92f9d2c6c5477182fa33eba4368ec1 in this term.
I20260709 12:16:20.260636 20536 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:16:20.261991 11222 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c519deb8010e411995230a6685947b97. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:20.267784 20473 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:20.268508 11222 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:20.272176 11900 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:20.272642 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:20.280208 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:20.281080 11931 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:20.285986 11257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:20.287201 11257 raft_consensus.cc:2393] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate bc92f9d2c6c5477182fa33eba4368ec1 in current term 26: Already voted for candidate 786851d4227e4dd89b75ccdea0b1f03b in this term.
I20260709 12:16:20.289165 11329 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:16:20.288601 20431 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 LEADER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:16:20.308362 11296 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:20.309012 11296 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:20.309722 20494 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:16:20.303856 20533 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:16:20.338446 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:20.344647 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:20.356957 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:20.369493 11127 auto_rebalancer.cc:629] Failed to schedule move for tablet 7cba0650205b4eb0b416158a057a90b9: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:20.381102 11127 auto_rebalancer.cc:629] Failed to schedule move for tablet d97e23379bda4a1b9ad1e2eabbf4d82f: Illegal state: Replica fecfc03de52a4308bc9b045cad5aac1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:16:20.389570 11127 auto_rebalancer.cc:629] Failed to schedule move for tablet fab915731e0347ae8353a7f300c44401: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:20.582516 10011 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11203)
I20260709 12:16:20.617349 11075 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56652 (request call id 176) took 1641 ms. Trace:
I20260709 12:16:20.617966 11075 rpcz_store.cc:276] 0709 12:16:18.976325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:18.976504 (+   179us) service_pool.cc:224] Handling call
0709 12:16:18.998198 (+ 21694us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:20.473759 (+1475561us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:19.791374 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:19.822878 (+ 31504us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:19.822910 (+    32us) write_op.cc:435] Acquired schema lock
0709 12:16:19.822926 (+    16us) tablet.cc:662] Decoding operations
0709 12:16:19.833049 (+ 10123us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:19.833146 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:19.833175 (+    29us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:16:19.834289 (+  1114us) tablet.cc:701] Row locks acquired
0709 12:16:19.834308 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:16:19.834465 (+   157us) write_op.cc:270] Start()
0709 12:16:19.834593 (+   128us) write_op.cc:276] Timestamp: P: 1783599379834410 usec, L: 0
0709 12:16:19.834612 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:16:19.905258 (+ 70646us) log.cc:844] Serialized 5579 byte log entry
0709 12:16:20.055699 (+150441us) op_driver.cc:464] REPLICATION: finished
0709 12:16:20.274280 (+218581us) write_op.cc:301] APPLY: starting
0709 12:16:20.274369 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:20.351365 (+ 76996us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:20.351495 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:20.380876 (+ 29381us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:20.381226 (+   350us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 12:16:20.381277 (+    51us) write_op.cc:312] APPLY: finished
0709 12:16:20.381589 (+   312us) log.cc:844] Serialized 129 byte log entry
0709 12:16:20.381779 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:20.381932 (+   153us) write_op.cc:454] Released schema lock
0709 12:16:20.381983 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:20.382363 (+   380us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:20.382397 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":99924,"reported_tablets":20,"spinlock_wait_cycles":34006400,"child_traces":[["op",{"apply.queue_time_us":218339,"apply.run_cpu_time_us":19539,"apply.run_wall_time_us":108071,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62759,"num_ops":14,"prepare.queue_time_us":131787,"prepare.run_cpu_time_us":5495,"prepare.run_wall_time_us":131423,"replication_time_us":220929,"thread_start_us":260706,"threads_started":2}]]}
W20260709 12:16:20.650388 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:20.704548 11070 catalog_manager.cc:5697] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 23 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.707437 11070 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 24 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 25 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:20.710139 11070 catalog_manager.cc:5697] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 23 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:20.744134 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:20.746819 20558 ts_tablet_manager.cc:933] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:20.756642 20558 tablet_copy_client.cc:323] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.194:37243
I20260709 12:16:20.753799 11070 catalog_manager.cc:5697] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 27, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 27 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.767177 11070 catalog_manager.cc:5697] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 25, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 25 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.772922 11267 tablet_copy_service.cc:140] P fecfc03de52a4308bc9b045cad5aac1e: Received BeginTabletCopySession request for tablet c519deb8010e411995230a6685947b97 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:52330)
I20260709 12:16:20.779518 11267 tablet_copy_service.cc:161] P fecfc03de52a4308bc9b045cad5aac1e: Beginning new tablet copy session on tablet c519deb8010e411995230a6685947b97 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:52330: session id = 786851d4227e4dd89b75ccdea0b1f03b-c519deb8010e411995230a6685947b97
W20260709 12:16:20.777444 11222 consensus_peers.cc:597] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:20.786259 11070 catalog_manager.cc:5697] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 18 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 23 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:20.789798 11070 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 25, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 25 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.792482 11267 tablet_copy_source_session.cc:215] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:20.793664 11070 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 23, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 23 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.795646 11070 catalog_manager.cc:5697] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 23 to 27. New cstate: current_term: 27 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.798700 20558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c519deb8010e411995230a6685947b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:20.797978 11070 catalog_manager.cc:5697] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.800071 11070 catalog_manager.cc:5697] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 17 to 20, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 20 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:20.806332 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:20.803355 11070 catalog_manager.cc:5697] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:20.809141 11070 catalog_manager.cc:5697] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 23 to 24, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 24 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.810876 11070 catalog_manager.cc:5697] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:20.815267 11328 raft_consensus.cc:1275] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 33 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:16:20.817392 20165 consensus_queue.cc:1048] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:16:20.822643 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:20.861079 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:20.874038 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:20.881892 20558 tablet_copy_client.cc:806] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:20.882798 20558 tablet_copy_client.cc:670] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
W20260709 12:16:20.889715 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:20.892426 20558 tablet_copy_client.cc:538] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:20.942508 20558 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
W20260709 12:16:21.165107 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:21.166617 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:21.203773 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:21.208491 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:21.250733 11075 catalog_manager.cc:5697] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 32 to 33, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 33 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.257424 11075 catalog_manager.cc:5697] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.296169 20558 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=1 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:21.313509 20558 tablet_bootstrap.cc:492] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:16:21.320854 20558 ts_tablet_manager.cc:1403] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.379s	user 0.082s	sys 0.033s
I20260709 12:16:21.338143 20558 raft_consensus.cc:359] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:21.349484 20558 raft_consensus.cc:740] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:16:21.354211 20558 consensus_queue.cc:260] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:21.357316 20558 ts_tablet_manager.cc:1434] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:16:21.362695 11267 tablet_copy_service.cc:342] P fecfc03de52a4308bc9b045cad5aac1e: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-c519deb8010e411995230a6685947b97 received from {username='slave'} at 127.0.0.1:52330
I20260709 12:16:21.363200 11267 tablet_copy_service.cc:434] P fecfc03de52a4308bc9b045cad5aac1e: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-c519deb8010e411995230a6685947b97 on tablet c519deb8010e411995230a6685947b97 with peer 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:21.376720 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:21.398307 11078 catalog_manager.cc:5697] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:21.409076 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:21.409732 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:21.418190 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df07764623640ce8ec29ad695812b7c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:40846
I20260709 12:16:21.418387 11078 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:21.422897 11078 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 23 to 25, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 25 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:21.427824 11078 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: term changed from 25 to 27, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to 016411114c6d42039bd04025378dfc98 (127.9.198.193). New cstate: current_term: 27 leader_uuid: "016411114c6d42039bd04025378dfc98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:16:21.429317 11070 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 244) took 12252 ms (client timeout 15000 ms). Trace:
I20260709 12:16:21.434123 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 5df07764623640ce8ec29ad695812b7c (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
W20260709 12:16:21.430990 11070 rpcz_store.cc:269] 0709 12:16:09.176452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:09.211335 (+ 34883us) service_pool.cc:224] Handling call
0709 12:16:21.429230 (+12217895us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:20.946304 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:20.946859 (+   555us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:20.946892 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:16:20.946909 (+    17us) tablet.cc:662] Decoding operations
0709 12:16:20.948129 (+  1220us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:20.948207 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:20.948234 (+    27us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:16:20.949266 (+  1032us) tablet.cc:701] Row locks acquired
0709 12:16:20.949285 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:16:20.949425 (+   140us) write_op.cc:270] Start()
0709 12:16:20.949543 (+   118us) write_op.cc:276] Timestamp: P: 1783599380949377 usec, L: 0
0709 12:16:20.949562 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:16:20.962364 (+ 12802us) log.cc:844] Serialized 5578 byte log entry
0709 12:16:21.030699 (+ 68335us) op_driver.cc:464] REPLICATION: finished
0709 12:16:21.047934 (+ 17235us) write_op.cc:301] APPLY: starting
0709 12:16:21.048026 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:21.095107 (+ 47081us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:21.095207 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:21.246418 (+151211us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:21.246680 (+   262us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 12:16:21.246743 (+    63us) write_op.cc:312] APPLY: finished
0709 12:16:21.247056 (+   313us) log.cc:844] Serialized 129 byte log entry
0709 12:16:21.247240 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:21.247393 (+   153us) write_op.cc:454] Released schema lock
0709 12:16:21.247442 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:21.247783 (+   341us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:21.247810 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":196757,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":16916,"apply.run_cpu_time_us":20974,"apply.run_wall_time_us":199854,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62759,"num_ops":14,"prepare.queue_time_us":65926,"prepare.run_cpu_time_us":4422,"prepare.run_wall_time_us":22420,"replication_time_us":77540,"thread_start_us":93305,"threads_started":2}]]}
I20260709 12:16:21.475121 11075 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index -1 to 17, term changed from 25 to 28, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to <none>, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 28 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:16:21.573081 11078 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50208 (request call id 248) took 12645 ms (client timeout 15000 ms). Trace:
W20260709 12:16:21.584893 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:21.573518 11078 rpcz_store.cc:269] 0709 12:16:08.927922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:08.928337 (+   415us) service_pool.cc:224] Handling call
0709 12:16:21.573014 (+12644677us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:21.438355 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:21.438910 (+   555us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:21.438955 (+    45us) write_op.cc:435] Acquired schema lock
0709 12:16:21.438972 (+    17us) tablet.cc:662] Decoding operations
0709 12:16:21.439616 (+   644us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:21.439677 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:21.439715 (+    38us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:16:21.440120 (+   405us) tablet.cc:701] Row locks acquired
0709 12:16:21.440136 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:16:21.440329 (+   193us) write_op.cc:270] Start()
0709 12:16:21.440441 (+   112us) write_op.cc:276] Timestamp: P: 1783599381440272 usec, L: 0
0709 12:16:21.440456 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:16:21.441067 (+   611us) log.cc:844] Serialized 1706 byte log entry
0709 12:16:21.443454 (+  2387us) op_driver.cc:464] REPLICATION: finished
0709 12:16:21.443899 (+   445us) write_op.cc:301] APPLY: starting
0709 12:16:21.443984 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:21.446270 (+  2286us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:21.446359 (+    89us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:21.448727 (+  2368us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:21.448896 (+   169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:16:21.448943 (+    47us) write_op.cc:312] APPLY: finished
0709 12:16:21.449187 (+   244us) log.cc:844] Serialized 49 byte log entry
0709 12:16:21.451212 (+  2025us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:21.451314 (+   102us) write_op.cc:454] Released schema lock
0709 12:16:21.451356 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:21.451660 (+   304us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:21.451686 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":317104,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":262,"apply.run_cpu_time_us":5910,"apply.run_wall_time_us":7732,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16752,"num_ops":4,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":3011,"prepare.run_wall_time_us":4884,"replication_time_us":2870}]]}
W20260709 12:16:21.614030 10011 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:21.620452 11079 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 261) took 84589 ms (client timeout 15000 ms). Trace:
W20260709 12:16:21.624686 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:21.625260 11079 rpcz_store.cc:269] 0709 12:14:57.031261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.031477 (+   216us) service_pool.cc:224] Handling call
0709 12:16:21.620245 (+84588768us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":222839,"reported_tablets":50,"spinlock_wait_cycles":24448}
W20260709 12:16:21.626390 11066 connection.cc:466] server connection from 127.0.0.1:50196 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50196 (request call id 261) could send its response
I20260709 12:16:21.705181 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df07764623640ce8ec29ad695812b7c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:40846
I20260709 12:16:21.714939 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 5df07764623640ce8ec29ad695812b7c (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
W20260709 12:16:21.723006 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:21.736224 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:21.765306 11077 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 240) took 105672 ms (client timeout 15000 ms). Trace:
W20260709 12:16:21.765802 11077 rpcz_store.cc:269] 0709 12:14:36.070105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.070472 (+   367us) service_pool.cc:224] Handling call
0709 12:16:21.742746 (+105672274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8417,"reported_tablets":49,"spinlock_wait_cycles":3328}
I20260709 12:16:21.808564 11075 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.810230 11075 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 25 to 28, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.826197 11930 raft_consensus.cc:1217] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Deduplicated request from leader. Original: 28.16->[28.17-28.17]   Dedup: 28.17->[]
I20260709 12:16:21.831779 11075 catalog_manager.cc:5697] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 26 to 27, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 27 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:21.833472 11075 catalog_manager.cc:5697] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 20 to 23, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 23 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:21.835570 11075 catalog_manager.cc:5697] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 21 to 25, leader changed from bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 25 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
W20260709 12:16:21.839318 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:21.855840 11075 catalog_manager.cc:5697] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 21 to 24, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 24 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.866030 11075 catalog_manager.cc:5697] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 27 to 30, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 30 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:21.882658 11075 catalog_manager.cc:5697] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 23 to 28, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.885946 11318 tablet_service.cc:1558] Processing DeleteTablet for tablet 5df07764623640ce8ec29ad695812b7c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:40846
I20260709 12:16:21.891999 11066 catalog_manager.cc:5028] TS bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): tablet 5df07764623640ce8ec29ad695812b7c (table test-workload [id=3e3271307e694b2281edeb86f889d4e9]) successfully deleted
I20260709 12:16:21.908267 11075 catalog_manager.cc:5697] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 25 to 26, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 26 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:21.909729 11074 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50220 (request call id 242) took 56506 ms (client timeout 15000 ms). Trace:
W20260709 12:16:21.919706 11074 rpcz_store.cc:269] 0709 12:15:25.403206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.403441 (+   235us) service_pool.cc:224] Handling call
0709 12:15:27.011656 (+1608215us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b54003d09d4. stack: 00007f3ec35cd343 00007f3ec35cd240 00007f3ec2f5c1d4 00007f3eccc9a304 00007f3eccc9a299 00007f3ec6656090 00007f3ec66f4f3a 00007f3ecc4aae9c 00007f3ecc5a904e 00007f3ecaa6a2a4 00007f3ecaa6a232 00007f3ecaa6a161 00007f3ecaa6a0dc 00007f3ecaa69392 00007f3ec4a2748c 00007f3ec4a268c6
0709 12:16:21.909496 (+54897840us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35726,"reported_tablets":49,"spinlock_wait_cycles":71751424}
I20260709 12:16:21.920892 11075 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 25 to 28, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:21.924626 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:21.924182 11075 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: leader changed from <none> to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 28 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:16:21.929000 11075 catalog_manager.cc:5697] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 23 to 26, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 26 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:21.942853 11075 catalog_manager.cc:5697] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 23 to 26, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194). New cstate: current_term: 26 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } }
I20260709 12:16:21.950557 11075 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 21 to 28, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.954388 11075 catalog_manager.cc:5697] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 27 to 30, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 30 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.955971 20165 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:21.977862 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:21.977519 20165 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:21.980556 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:21.981999 20165 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:21.983174 11328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:21.984218 11328 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:16:21.986346 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e" is_pre_election: true
I20260709 12:16:21.987124 11252 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 26.
I20260709 12:16:21.991333 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 
I20260709 12:16:21.995613 20165 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:16:21.996011 20165 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:21.996306 20165 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:16:22.030536 11900 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:22.034407 20165 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:22.044950 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:22.054322 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:22.070348 20165 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Requested vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:22.071404 11329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:22.076486 11329 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:22.089504 11252 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "fecfc03de52a4308bc9b045cad5aac1e"
I20260709 12:16:22.090229 11252 raft_consensus.cc:3060] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:16:22.109122 11329 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
W20260709 12:16:22.106999 11900 leader_election.cc:336] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:22.124012 11252 raft_consensus.cc:2468] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 27.
I20260709 12:16:22.125797 11900 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98
I20260709 12:16:22.128181 20165 raft_consensus.cc:2804] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:16:22.128743 20165 raft_consensus.cc:697] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:22.146071 20165 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:22.170689 11222 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:22.208974 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:22.215051 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:22.221278 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.227012 11222 consensus_peers.cc:597] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.227588 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:22.254948 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:22.313006 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.345295 11222 consensus_peers.cc:597] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:22.360636 11077 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 25 to 27, leader changed from 016411114c6d42039bd04025378dfc98 (127.9.198.193) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 27 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:22.392256 20512 raft_consensus.cc:1064] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
W20260709 12:16:22.406998 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:22.401207 20512 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 15, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:22.430886 11257 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:22.467109 11930 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:16:22.468974 20576 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:16:22.487881 11222 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:22.491704 11930 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:16:22.498750 11330 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:16:22.509868 20571 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:16:22.510867 11328 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:16:22.511929 20576 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:16:22.542704 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.554131 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:22.533083 20571 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:16:22.587813 11257 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:16:22.589744 20165 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:22.594777 11331 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Committing config change with OpId 29.19: config changed from index 6 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.595393 20536 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Committing config change with OpId 29.19: config changed from index 6 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:22.614553 10011 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 11203)
I20260709 12:16:22.614054 11931 raft_consensus.cc:2955] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Committing config change with OpId 29.19: config changed from index 6 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.615728 11079 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 6 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 29 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.638718 20512 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.646445 11329 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:22.674769 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.686230 11222 consensus_peers.cc:597] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.702538 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:22.705854 11931 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:22.720175 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:22.723903 11329 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:16:22.728029 20573 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:16:22.736795 11222 consensus_peers.cc:597] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.752938 11065 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 53350.4us
I20260709 12:16:22.753823 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 86c5246d860d4d559ea665671cc46b5c with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:16:22.764526 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:22.768069 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 86c5246d860d4d559ea665671cc46b5c: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:22.770309 11079 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:22.784782 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:22.807798 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:22.815577 11252 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:22.816593 11222 consensus_peers.cc:597] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:22.817453 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:22.830111 11330 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:16:22.831926 11931 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Refusing update from remote peer fecfc03de52a4308bc9b045cad5aac1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:16:22.833402 20576 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:16:22.835832 20576 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:16:22.845688 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:22.849889 20512 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 LEADER]: Committing config change with OpId 28.19: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.862812 11931 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.869390 11330 raft_consensus.cc:2955] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.888574 11079 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 28 leader_uuid: "fecfc03de52a4308bc9b045cad5aac1e" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:22.914564 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c519deb8010e411995230a6685947b97 with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:16:23.122464 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:23.181834 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:23.185806 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:23.197120 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:23.312043 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:23.344340 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:23.405176 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:23.536175 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:23.615015 10011 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:23.837270 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 133586us
W20260709 12:16:23.865836 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 147296us
W20260709 12:16:23.887240 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:23.968852 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 86c5246d860d4d559ea665671cc46b5c: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:23.976157 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:23.986033 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 81721.7us
W20260709 12:16:24.029759 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 76391.6us
W20260709 12:16:24.040906 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 60662.5us
W20260709 12:16:24.060410 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:24.069872 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:24.083258 11901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 93439.8us
W20260709 12:16:24.106734 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:24.129894 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:24.134358 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:24.192222 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:24.217921 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:24.285688 11331 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:24.321419 11248 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:16:24.323863 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab7767a1cf4e4b5493ca9acc8daeabab. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:24.325402 20603 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:16:24.327534 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:24.348064 20568 raft_consensus.cc:2955] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Committing config change with OpId 28.20: config changed from index -1 to 20, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:16:24.355285 11248 raft_consensus.cc:2955] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index -1 to 20, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:16:24.379981 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:24.386065 11331 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:24.385658 11079 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 20, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:24.470191 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:24.473719 11248 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:16:24.475159 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d97e23379bda4a1b9ad1e2eabbf4d82f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:24.484592 20596 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:16:24.497532 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:24.542938 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:24.558205 20603 raft_consensus.cc:2955] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:16:24.570521 11248 raft_consensus.cc:2955] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:16:24.590480 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:24.597679 11331 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:16:24.615743 10011 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 11203)
I20260709 12:16:24.635370 11075 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:24.644093 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8eaf18bb7a144d35b42cf5548b769c0f. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:24.650063 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:24.650859 11248 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:16:24.652496 20603 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:16:24.671039 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:24.700906 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:24.703389 20603 raft_consensus.cc:2955] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 LEADER]: Committing config change with OpId 27.15: config changed from index -1 to 15, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:16:24.713402 11252 raft_consensus.cc:2955] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Committing config change with OpId 27.15: config changed from index -1 to 15, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:16:24.805998 11222 consensus_peers.cc:597] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:24.815500 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 62050.7us
W20260709 12:16:24.825416 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:24.826098 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:24.870153 11075 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 15, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 27 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:16:24.876068 11331 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:24.877115 20619 ts_tablet_manager.cc:933] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:24.898319 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:24.909981 20619 tablet_copy_client.cc:323] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:16:24.912973 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet ab7767a1cf4e4b5493ca9acc8daeabab from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:16:24.913856 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet ab7767a1cf4e4b5493ca9acc8daeabab from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-ab7767a1cf4e4b5493ca9acc8daeabab
I20260709 12:16:24.927625 11341 tablet_copy_source_session.cc:215] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:16:24.929283 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:24.931739 20619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7767a1cf4e4b5493ca9acc8daeabab. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:24.937297 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:24.940421 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): Couldn't send request to peer 786851d4227e4dd89b75ccdea0b1f03b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fab915731e0347ae8353a7f300c44401. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:24.942081 11252 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:16:24.976315 20603 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:16:24.978763 11296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111) [suppressed 475 similar messages]
I20260709 12:16:24.983837 20619 tablet_copy_client.cc:806] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:24.993014 20619 tablet_copy_client.cc:670] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
W20260709 12:16:24.999614 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:25.005793 20619 tablet_copy_client.cc:538] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:16:25.007397 11222 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:25.028489 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:25.042872 20596 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:16:25.046972 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:25.062361 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:25.058460 11252 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
I20260709 12:16:25.104596 20623 ts_tablet_manager.cc:933] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:25.114137 20619 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
W20260709 12:16:25.135962 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:25.137195 20623 tablet_copy_client.cc:323] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:16:25.139461 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet d97e23379bda4a1b9ad1e2eabbf4d82f from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:16:25.139976 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet d97e23379bda4a1b9ad1e2eabbf4d82f from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-d97e23379bda4a1b9ad1e2eabbf4d82f
I20260709 12:16:25.148101 11341 tablet_copy_source_session.cc:215] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:16:25.152280 11065 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 69057.9us
I20260709 12:16:25.170569 20623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97e23379bda4a1b9ad1e2eabbf4d82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:25.181169 11079 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e reported cstate change: config changed from index -1 to 17, NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) added. New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } } }
W20260709 12:16:25.197191 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c519deb8010e411995230a6685947b97: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:25.206197 11222 consensus_peers.cc:597] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:25.213739 11222 consensus_peers.cc:597] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:25.215338 20627 ts_tablet_manager.cc:933] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:25.256556 20623 tablet_copy_client.cc:806] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:25.257308 20623 tablet_copy_client.cc:670] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:25.262598 20623 tablet_copy_client.cc:538] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:16:25.268505 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:25.269775 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:25.302309 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:25.317040 20623 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:16:25.322748 20627 tablet_copy_client.cc:323] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:16:25.324868 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet 8eaf18bb7a144d35b42cf5548b769c0f from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:16:25.325453 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet 8eaf18bb7a144d35b42cf5548b769c0f from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-8eaf18bb7a144d35b42cf5548b769c0f
I20260709 12:16:25.336776 11341 tablet_copy_source_session.cc:215] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:25.340940 20627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eaf18bb7a144d35b42cf5548b769c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:25.388103 20627 tablet_copy_client.cc:806] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:25.388948 20627 tablet_copy_client.cc:670] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:25.481184 20627 tablet_copy_client.cc:538] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:16:25.515410 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 45073.3us
W20260709 12:16:25.582266 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:25.613987 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:25.616434 10011 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:25.620824 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:25.627287 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:25.675323 20638 ts_tablet_manager.cc:933] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: Initiating tablet copy from peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:25.683255 20627 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:16:25.723013 20638 tablet_copy_client.cc:323] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.198.195:42967
I20260709 12:16:25.748730 11341 tablet_copy_service.cc:140] P bc92f9d2c6c5477182fa33eba4368ec1: Received BeginTabletCopySession request for tablet fab915731e0347ae8353a7f300c44401 from peer 786851d4227e4dd89b75ccdea0b1f03b ({username='slave'} at 127.0.0.1:59670)
I20260709 12:16:25.749568 11341 tablet_copy_service.cc:161] P bc92f9d2c6c5477182fa33eba4368ec1: Beginning new tablet copy session on tablet fab915731e0347ae8353a7f300c44401 from peer 786851d4227e4dd89b75ccdea0b1f03b at {username='slave'} at 127.0.0.1:59670: session id = 786851d4227e4dd89b75ccdea0b1f03b-fab915731e0347ae8353a7f300c44401
W20260709 12:16:25.771898 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:25.781433 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 68738.8us
I20260709 12:16:25.757601 20619 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=22 overwritten=2 applied=20 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:25.784409 20619 tablet_bootstrap.cc:492] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:16:25.787160 20619 ts_tablet_manager.cc:1403] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.673s	user 0.070s	sys 0.009s
I20260709 12:16:25.815564 11341 tablet_copy_source_session.cc:215] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:25.820963 20638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab915731e0347ae8353a7f300c44401. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:25.831307 11222 consensus_peers.cc:597] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:25.857251 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:25.864198 20638 tablet_copy_client.cc:806] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:25.864923 20638 tablet_copy_client.cc:670] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:25.872406 20619 raft_consensus.cc:359] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
W20260709 12:16:25.889273 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:25.889207 11222 consensus_peers.cc:597] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:25.901327 20638 tablet_copy_client.cc:538] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:25.905748 20619 raft_consensus.cc:740] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:16:25.914713 20638 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap starting.
I20260709 12:16:25.914224 20619 consensus_queue.cc:260] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:25.926625 20623 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=1 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:25.927908 20623 tablet_bootstrap.cc:492] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:16:25.928923 20623 ts_tablet_manager.cc:1403] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.612s	user 0.063s	sys 0.010s
I20260709 12:16:25.943778 20623 raft_consensus.cc:359] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:25.944785 20623 raft_consensus.cc:740] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:16:25.945490 20623 consensus_queue.cc:260] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:25.947685 20623 ts_tablet_manager.cc:1434] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:16:25.949649 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-d97e23379bda4a1b9ad1e2eabbf4d82f received from {username='slave'} at 127.0.0.1:59670
I20260709 12:16:25.950882 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-d97e23379bda4a1b9ad1e2eabbf4d82f on tablet d97e23379bda4a1b9ad1e2eabbf4d82f with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:25.977579 20619 ts_tablet_manager.cc:1434] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.190s	user 0.009s	sys 0.001s
I20260709 12:16:26.026760 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-ab7767a1cf4e4b5493ca9acc8daeabab received from {username='slave'} at 127.0.0.1:59670
I20260709 12:16:26.027268 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-ab7767a1cf4e4b5493ca9acc8daeabab on tablet ab7767a1cf4e4b5493ca9acc8daeabab with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:26.028270 11935 raft_consensus.cc:1217] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Deduplicated request from leader. Original: 28.16->[28.17-28.17]   Dedup: 28.17->[]
I20260709 12:16:26.207955 11935 raft_consensus.cc:1217] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Deduplicated request from leader. Original: 28.19->[28.20-28.20]   Dedup: 28.20->[]
I20260709 12:16:26.220933 20627 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=2 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:26.221973 20627 tablet_bootstrap.cc:492] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:16:26.222782 20627 ts_tablet_manager.cc:1403] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.540s	user 0.075s	sys 0.058s
I20260709 12:16:26.240551 20627 raft_consensus.cc:359] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:26.252285 20627 raft_consensus.cc:740] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:16:26.255169 20627 consensus_queue.cc:260] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:26.259272 20627 ts_tablet_manager.cc:1434] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.004s
I20260709 12:16:26.261861 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-8eaf18bb7a144d35b42cf5548b769c0f received from {username='slave'} at 127.0.0.1:59670
I20260709 12:16:26.262308 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-8eaf18bb7a144d35b42cf5548b769c0f on tablet 8eaf18bb7a144d35b42cf5548b769c0f with peer 786851d4227e4dd89b75ccdea0b1f03b
I20260709 12:16:26.267475 20638 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=6 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:26.268615 20638 tablet_bootstrap.cc:492] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: Bootstrap complete.
I20260709 12:16:26.269829 20638 ts_tablet_manager.cc:1403] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent bootstrapping tablet: real 0.355s	user 0.090s	sys 0.034s
I20260709 12:16:26.273795 20638 raft_consensus.cc:359] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:26.274811 20638 raft_consensus.cc:740] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Initialized, Role: LEARNER
I20260709 12:16:26.275993 20638 consensus_queue.cc:260] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: NON_VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: true } }
I20260709 12:16:26.279902 20638 ts_tablet_manager.cc:1434] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:26.283824 11341 tablet_copy_service.cc:342] P bc92f9d2c6c5477182fa33eba4368ec1: Request end of tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-fab915731e0347ae8353a7f300c44401 received from {username='slave'} at 127.0.0.1:59670
I20260709 12:16:26.284274 11341 tablet_copy_service.cc:434] P bc92f9d2c6c5477182fa33eba4368ec1: ending tablet copy session 786851d4227e4dd89b75ccdea0b1f03b-fab915731e0347ae8353a7f300c44401 on tablet fab915731e0347ae8353a7f300c44401 with peer 786851d4227e4dd89b75ccdea0b1f03b
W20260709 12:16:26.285238 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:26.350221 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:26.366536 11935 raft_consensus.cc:1217] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEARNER]: Deduplicated request from leader. Original: 27.14->[27.15-27.15]   Dedup: 27.15->[]
W20260709 12:16:26.410265 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:26.475658 20642 raft_consensus.cc:1064] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:16:26.480248 20642 consensus_queue.cc:237] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 15, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:26.521770 11935 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:16:26.526926 20596 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:26.540522 11252 raft_consensus.cc:1275] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:16:26.542176 20596 consensus_queue.cc:1048] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:16:26.550629 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:26.617018 10011 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 11203)
I20260709 12:16:26.652009 11931 raft_consensus.cc:1217] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Deduplicated request from leader. Original: 28.16->[28.17-28.17]   Dedup: 28.17->[]
I20260709 12:16:26.665658 20618 raft_consensus.cc:2955] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:26.666664 11935 raft_consensus.cc:2955] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:26.716450 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:26.750018 11248 raft_consensus.cc:2955] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:26.777959 11079 catalog_manager.cc:5697] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:26.819813 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:26.842603 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:26.851950 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:26.867730 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:26.877619 20568 raft_consensus.cc:1064] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
W20260709 12:16:26.904685 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:26.912283 20568 consensus_queue.cc:237] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 18, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:26.917819 20642 raft_consensus.cc:1064] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:16:26.923461 20642 consensus_queue.cc:237] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 13, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:26.950117 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:26.960350 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:26.963665 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:26.970520 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:26.999182 11935 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:16:27.018857 11248 raft_consensus.cc:1275] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:16:27.031448 20618 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:16:27.054544 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.065438 20642 consensus_queue.cc:1048] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:27.075407 11935 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:16:27.085289 20642 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:16:27.087291 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:27.095112 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:27.097620 11248 raft_consensus.cc:1275] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:16:27.118583 20642 consensus_queue.cc:1048] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20260709 12:16:27.138558 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:27.139132 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.168735 20618 raft_consensus.cc:2955] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 LEADER]: Committing config change with OpId 27.16: config changed from index 15 to 16, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.171363 11935 raft_consensus.cc:2955] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index 15 to 16, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:27.195015 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:27.195994 20568 raft_consensus.cc:1064] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: attempting to promote NON_VOTER 786851d4227e4dd89b75ccdea0b1f03b to VOTER
I20260709 12:16:27.201575 20568 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 15, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:27.221892 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:27.229568 20596 raft_consensus.cc:2955] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Committing config change with OpId 28.21: config changed from index 20 to 21, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.231384 11248 raft_consensus.cc:2955] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index 15 to 16, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.237769 11249 raft_consensus.cc:2955] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Committing config change with OpId 28.21: config changed from index 20 to 21, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.243847 11077 catalog_manager.cc:5697] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 15 to 16, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:27.248766 11931 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEARNER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:16:27.248791 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.250298 11257 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:16:27.252640 20618 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 12:16:27.261535 20654 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:16:27.287981 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:27.313971 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:27.346313 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.348366 11931 raft_consensus.cc:2955] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Committing config change with OpId 28.21: config changed from index 20 to 21, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.374203 11077 catalog_manager.cc:5697] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: config changed from index 20 to 21, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:27.381737 20663 log.cc:927] Time spent T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:16:27.391069 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c519deb8010e411995230a6685947b97: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:27.398965 11252 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.395701 20596 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:27.410816 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.480338 11931 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:27.502645 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:27.512760 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:27.532298 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:27.542843 11222 consensus_peers.cc:597] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:27.535300 11077 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 17 to 18, 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:27.606101 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:27.617544 10011 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:27.642882 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 50133.1us
I20260709 12:16:27.659235 11330 consensus_queue.cc:237] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:27.693682 11931 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:16:27.696285 11252 raft_consensus.cc:1275] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:16:27.698794 20642 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:16:27.701813 20642 consensus_queue.cc:1048] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:16:27.712334 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:27.762523 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.768831 20642 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Committing config change with OpId 28.19: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
W20260709 12:16:27.804006 11222 consensus_peers.cc:597] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:27.805416 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.810978 11931 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.818856 11254 raft_consensus.cc:2955] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:27.822830 11066 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fab915731e0347ae8353a7f300c44401 with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:16:27.826602 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:27.827185 11222 consensus_peers.cc:597] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:27.858393 11222 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:27.859444 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:27.860659 11077 catalog_manager.cc:5697] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 18 to 19, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 28 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:27.882043 11222 consensus_peers.cc:597] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:27.936967 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:27.962360 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:27.996690 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:28.004248 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:28.027377 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:28.354944 11222 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 113632us
W20260709 12:16:28.437031 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:28.499904 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:28.636735 11222 consensus_peers.cc:597] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:28.643754 10011 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:28.648442 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fab915731e0347ae8353a7f300c44401: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:28.970108 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:28.976830 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.050009 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:29.056543 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:29.257015 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.290627 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.291733 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:29.338152 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:29.341696 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:29.381325 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:29.398622 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.454983 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.470126 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:29.575273 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:29.580152 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:29.604985 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:29.644320 10011 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:29.687870 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.688593 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:29.724054 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.741600 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.750690 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:29.862605 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:29.885450 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:29.898175 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.929880 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:29.985397 11900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111) [suppressed 485 similar messages]
W20260709 12:16:29.997695 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:29.990705 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:30.019097 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.104799 11222 consensus_peers.cc:597] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:30.113402 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.119341 11222 consensus_peers.cc:597] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:30.170253 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.188966 11222 consensus_peers.cc:597] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:30.191609 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.192782 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.230463 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:30.282460 11222 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:30.337205 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fab915731e0347ae8353a7f300c44401: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:30.377698 11222 consensus_peers.cc:597] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:30.543067 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.602566 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.610252 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:30.627830 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:30.631809 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:30.644853 10011 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:30.649125 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:30.715741 11935 consensus_queue.cc:237] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:30.745903 11331 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:16:30.747542 20686 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:30.752936 11254 raft_consensus.cc:1275] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:16:30.756546 20687 consensus_queue.cc:1048] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:30.773162 20687 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Committing config change with OpId 27.14: config changed from index 4 to 14, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:30.778538 11329 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Committing config change with OpId 27.14: config changed from index 4 to 14, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:30.790752 11254 raft_consensus.cc:2955] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Committing config change with OpId 27.14: config changed from index 4 to 14, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:30.851598 11075 catalog_manager.cc:5697] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: config changed from index 4 to 14, VOTER 016411114c6d42039bd04025378dfc98 (127.9.198.193) evicted. New cstate: current_term: 27 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } } }
I20260709 12:16:30.890686 11065 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6d225f8a873c4abe992018eb9353664f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
W20260709 12:16:30.978957 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:31.074549 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:31.241289 11222 consensus_peers.cc:597] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:31.353004 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6d225f8a873c4abe992018eb9353664f: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:31.466355 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:31.630326 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:31.639359 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:31.645367 10011 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:31.675580 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:31.724239 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:31.744230 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:31.785251 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:31.831535 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:31.846454 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:31.854984 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:31.900605 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:31.902119 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:31.972247 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:31.993425 11222 consensus_peers.cc:597] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:32.021332 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:32.044559 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:32.127408 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:32.141304 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:32.315069 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:32.325052 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:32.388821 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:32.414554 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:32.433179 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:32.451076 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fab915731e0347ae8353a7f300c44401: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:32.530061 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:39119, user_credentials={real_user=slave}} blocked reactor thread for 44989.5us
W20260709 12:16:32.645982 10011 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:33.154412 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 534507us
W20260709 12:16:33.158375 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:33.170557 11294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.035s	sys 0.038s
W20260709 12:16:33.171010 11294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.035s	sys 0.039s
W20260709 12:16:33.227429 11900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 67809.3us
W20260709 12:16:33.252846 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 56159.7us
W20260709 12:16:33.320255 11222 consensus_peers.cc:597] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:33.321913 11222 consensus_peers.cc:597] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.322312 11222 consensus_peers.cc:597] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:33.322723 11222 consensus_peers.cc:597] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:33.323961 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:33.353868 11222 consensus_peers.cc:597] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.354642 11222 consensus_peers.cc:597] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:33.355451 11222 consensus_peers.cc:597] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:33.365852 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.367326 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:33.408651 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:33.409425 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:33.409996 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.410491 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:33.415759 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.627977 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.630206 11065 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 81097.2us
W20260709 12:16:33.636703 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6d225f8a873c4abe992018eb9353664f: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:33.647758 10011 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 11203)
W20260709 12:16:33.711992 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.728820 11222 consensus_peers.cc:597] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.779385 11222 consensus_peers.cc:597] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:33.955976 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:34.076221 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002d47e0 after lost signal to thread 10014
W20260709 12:16:34.077654 11956 debug-util.cc:398] Leaking SignalData structure 0x7b080033c440 after lost signal to thread 11113
W20260709 12:16:34.079226 11956 debug-util.cc:398] Leaking SignalData structure 0x7b0800256520 after lost signal to thread 11281
W20260709 12:16:34.081049 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002ac120 after lost signal to thread 11355
W20260709 12:16:34.082428 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2260 after lost signal to thread 11959
W20260709 12:16:34.282888 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:34.355942 11222 consensus_peers.cc:597] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:34.377187 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:34.404798 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:34.514993 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:34.577698 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:34.635771 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
W20260709 12:16:34.716761 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:34.732729 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:34.795500 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:34.825858 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:34.875403 11222 consensus_peers.cc:597] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:34.884502 11222 consensus_peers.cc:597] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:34.911279 11222 consensus_peers.cc:597] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:34.930136 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:34.988135 11900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111) [suppressed 424 similar messages]
W20260709 12:16:34.992272 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:34.993988 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
W20260709 12:16:34.999619 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.010694 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.030021 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.039006 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:35.044602 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.076713 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:35.089394 11296 consensus_peers.cc:597] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.106832 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.113770 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.116072 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.123579 11900 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.141765 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.143740 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.149899 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.165469 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.167222 11222 consensus_peers.cc:597] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:35.188355 11222 consensus_peers.cc:597] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:35.200213 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.210723 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.222530 11296 consensus_peers.cc:597] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.229480 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.230310 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:35.239149 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:16:35.242130 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:35.240325 10011 tablet_replica.cc:333] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.244554 10011 raft_consensus.cc:2243] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.246255 10011 raft_consensus.cc:2272] T 89068ffedf554d89af62a8b1fe0700c0 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.250037 10011 tablet_replica.cc:333] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.255661 11296 consensus_peers.cc:597] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.256606 10011 raft_consensus.cc:2243] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.257541 10011 raft_consensus.cc:2272] T 9fdf86dcb01f4d81913c37b45cfc3879 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:35.259407 11222 consensus_peers.cc:597] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:35.261497 10011 tablet_replica.cc:333] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.264508 11222 consensus_peers.cc:597] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:35.276783 10011 raft_consensus.cc:2243] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.277572 10011 raft_consensus.cc:2272] T 5fa73285733a4a67a3e705a20155f69d P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.281188 10011 tablet_replica.cc:333] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.286573 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.295696 10011 raft_consensus.cc:2243] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.309904 10011 raft_consensus.cc:2272] T 0a6ec18bb8c54ece99a794e109201789 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:35.332739 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:35.337365 11222 consensus_peers.cc:597] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:35.356222 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:35.357144 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.359297 10011 tablet_replica.cc:333] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.362833 10011 raft_consensus.cc:2243] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.364200 10011 raft_consensus.cc:2272] T 458129fddd7e4b33b63d818f6af21f23 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.368300 10011 tablet_replica.cc:333] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.377056 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.379271 10011 raft_consensus.cc:2243] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.385056 10011 raft_consensus.cc:2272] T 8eaf18bb7a144d35b42cf5548b769c0f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.387959 10011 tablet_replica.cc:333] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.383972 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:35.391055 11296 consensus_peers.cc:597] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.392277 10011 raft_consensus.cc:2243] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.393426 10011 raft_consensus.cc:2272] T 86c5246d860d4d559ea665671cc46b5c P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.396610 10011 tablet_replica.cc:333] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.402750 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.397622 10011 raft_consensus.cc:2243] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.407124 10011 raft_consensus.cc:2272] T e03893c6062f45ea972113cfa57ba362 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.426579 10011 tablet_replica.cc:333] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.427662 10011 raft_consensus.cc:2243] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.428959 10011 raft_consensus.cc:2272] T 7cba0650205b4eb0b416158a057a90b9 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.432025 10011 tablet_replica.cc:333] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.433099 10011 raft_consensus.cc:2243] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.434852 10011 raft_consensus.cc:2272] T 5ad54acf641349bc8119ce4c0eae446b P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.438055 10011 tablet_replica.cc:333] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.439112 10011 raft_consensus.cc:2243] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.440362 10011 raft_consensus.cc:2272] T 16e4ceea03b94eb2803569189ab5fcca P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.444232 10011 tablet_replica.cc:333] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.444983 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.445676 10011 raft_consensus.cc:2243] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.447141 10011 raft_consensus.cc:2272] T fab915731e0347ae8353a7f300c44401 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.501748 10011 tablet_replica.cc:333] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.505512 10011 raft_consensus.cc:2243] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.506896 10011 raft_consensus.cc:2272] T b9dab54d96124869a4a52f9cc08e288a P fecfc03de52a4308bc9b045cad5aac1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.511677 10011 tablet_replica.cc:333] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.513837 10011 raft_consensus.cc:2243] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.515444 10011 raft_consensus.cc:2272] T d49a939bed404bfcb2538b5331071d3b P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.520063 10011 tablet_replica.cc:333] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.522454 10011 raft_consensus.cc:2243] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.524204 10011 raft_consensus.cc:2272] T 35c7899b1a314ae1b67824dfee6ff091 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.545959 10011 tablet_replica.cc:333] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.572919 10011 raft_consensus.cc:2243] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.576545 10011 raft_consensus.cc:2272] T 53e6b498179b4c96b8b0be6240a36efb P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:35.591444 11222 consensus_peers.cc:597] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:35.610134 10011 tablet_replica.cc:333] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.611666 10011 raft_consensus.cc:2243] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.613072 10011 raft_consensus.cc:2272] T 4a634acd9ff543cca551c8a28e72d6c4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:35.613339 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:45115, user_credentials={real_user=slave}} blocked reactor thread for 55145.2us
I20260709 12:16:35.644867 10011 tablet_replica.cc:333] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.667186 10011 raft_consensus.cc:2243] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.671888 10011 raft_consensus.cc:2272] T ddf8cca10e1244d8882abd13fa81113a P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.676395 10011 tablet_replica.cc:333] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.670434 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:35.695957 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:35.715965 11223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42967, user_credentials={real_user=slave}} blocked reactor thread for 71010.1us
I20260709 12:16:35.718549 10011 raft_consensus.cc:2243] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.719769 10011 raft_consensus.cc:2272] T 3e235bf9f3514f629a5c26c1eac58682 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:35.735888 11222 consensus_peers.cc:597] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:35.742129 10011 tablet_replica.cc:333] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.770790 10011 raft_consensus.cc:2243] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.771847 10011 raft_consensus.cc:2272] T 7f736e38db6b45bfa87e35af564bffc1 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.789232 10011 tablet_replica.cc:333] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.789490 11296 consensus_peers.cc:597] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.798653 10011 raft_consensus.cc:2243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.801810 10011 raft_consensus.cc:2272] T 9424adcaacdf40f3bbd2f639ef2a48c2 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.806502 10011 tablet_replica.cc:333] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
W20260709 12:16:35.816594 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:35.830413 10011 raft_consensus.cc:2243] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:16:35.833048 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6d225f8a873c4abe992018eb9353664f: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:35.833518 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.837220 10011 raft_consensus.cc:2272] T 4958b19b61094e1db763fdb94852f994 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.841265 10011 tablet_replica.cc:333] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.852571 10011 raft_consensus.cc:2243] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:16:35.853955 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:35.860378 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.862628 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:35.854396 10011 raft_consensus.cc:2272] T c519deb8010e411995230a6685947b97 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.890264 10011 tablet_replica.cc:333] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.900628 10011 raft_consensus.cc:2243] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.909039 10011 raft_consensus.cc:2272] T 5fc47889c90446dd927b2ac3010d58fd P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:35.908013 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:35.926403 10011 tablet_replica.cc:333] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.935985 10011 raft_consensus.cc:2243] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.937583 10011 raft_consensus.cc:2272] T 00b8260131ab49ecb2df9631e5704096 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.953328 10011 tablet_replica.cc:333] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:35.978849 10011 raft_consensus.cc:2243] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:16:35.983111 10011 raft_consensus.cc:2272] T 0fd765747f9648049b905c70696b206d P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.991667 10011 tablet_replica.cc:333] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.003242 10011 raft_consensus.cc:2243] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.004900 10011 raft_consensus.cc:2272] T f2d37a48723c42228c9717f514174704 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:36.008742 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:36.010762 11222 consensus_peers.cc:597] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:36.021657 10011 tablet_replica.cc:333] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.022780 10011 raft_consensus.cc:2243] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.024346 10011 raft_consensus.cc:2272] T 06a9ec0400af47eeaffc85f3f1d38de4 P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.051921 20739 raft_consensus.cc:493] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:36.052665 20739 raft_consensus.cc:515] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:36.056079 10011 tablet_replica.cc:333] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.069589 20739 leader_election.cc:290] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:36.072686 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1db2221cc614cac91fdaa49f8d6cb61" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:36.073611 10011 raft_consensus.cc:2243] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.085129 10011 raft_consensus.cc:2272] T 5df07764623640ce8ec29ad695812b7c P fecfc03de52a4308bc9b045cad5aac1e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:36.088253 11222 leader_election.cc:336] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:36.091885 11222 leader_election.cc:304] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:36.093370 20739 raft_consensus.cc:2749] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:36.095263 10011 tablet_replica.cc:333] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.096293 10011 raft_consensus.cc:2243] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.097851 10011 raft_consensus.cc:2272] T dedbfb6dda024d1e81b2740f108080af P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.100173 20739 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:36.100929 20739 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.105199 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:36.119051 11222 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:36.121135 10011 tablet_replica.cc:333] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.128548 10011 raft_consensus.cc:2243] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.125867 11222 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:36.137317 20715 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:36.140793 20739 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:36.125409 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:36.142198 10011 raft_consensus.cc:2272] T 88ddcb89ab6d47cd97e8c19734f0e365 P fecfc03de52a4308bc9b045cad5aac1e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.142879 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7767a1cf4e4b5493ca9acc8daeabab" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:36.161353 20739 raft_consensus.cc:493] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:36.166249 20739 raft_consensus.cc:515] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:36.169075 10011 tablet_replica.cc:333] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.179265 20739 leader_election.cc:290] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:36.180962 20715 raft_consensus.cc:493] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:36.185519 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f9c5be972b3443eb3211fc2eb29d575" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:36.188053 11222 leader_election.cc:336] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:36.193434 10011 raft_consensus.cc:2243] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.194617 20715 raft_consensus.cc:515] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:36.203612 11223 leader_election.cc:304] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:36.202664 20715 leader_election.cc:290] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:36.206380 20715 raft_consensus.cc:2749] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:36.208438 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6a39d516e943a9b2c8c4213ac11902" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:36.210985 11222 leader_election.cc:336] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:36.213088 20715 raft_consensus.cc:493] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 786851d4227e4dd89b75ccdea0b1f03b)
W20260709 12:16:36.204200 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:36.214038 20715 raft_consensus.cc:515] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.218034 20715 leader_election.cc:290] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:36.222550 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:36.224218 10011 raft_consensus.cc:2272] T d97e23379bda4a1b9ad1e2eabbf4d82f P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.230129 11223 leader_election.cc:304] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:36.231513 20715 raft_consensus.cc:2749] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:36.233878 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d225f8a873c4abe992018eb9353664f" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:36.253425 11223 leader_election.cc:304] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:36.262701 10011 tablet_replica.cc:333] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.263682 10011 raft_consensus.cc:2243] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.264694 10011 raft_consensus.cc:2272] T 6c6a39d516e943a9b2c8c4213ac11902 P fecfc03de52a4308bc9b045cad5aac1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.268162 10011 tablet_replica.cc:333] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.269071 10011 raft_consensus.cc:2243] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.269970 20715 raft_consensus.cc:2749] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:36.276657 10011 raft_consensus.cc:2272] T e5f804d45235415fad0f1442ea04aa47 P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.291082 10011 tablet_replica.cc:333] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.297799 10011 raft_consensus.cc:2243] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.299250 10011 raft_consensus.cc:2272] T 2fdc8cd3021d4b678786056187782c16 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.303764 10011 tablet_replica.cc:333] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.304658 10011 raft_consensus.cc:2243] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.305768 10011 raft_consensus.cc:2272] T 1fb796253655415ba47e989105d7c807 P fecfc03de52a4308bc9b045cad5aac1e [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.309214 10011 tablet_replica.cc:333] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.310127 10011 raft_consensus.cc:2243] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.311475 10011 raft_consensus.cc:2272] T d5a029ce312749bdaf2113d2823e993e P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.343046 20715 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:36.343657 20715 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.347020 20715 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:36.347225 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:36.349053 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa8154ca19364187a13db1be3e0b4f53" candidate_uuid: "fecfc03de52a4308bc9b045cad5aac1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:36.349884 10011 tablet_replica.cc:333] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.351516 11223 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fecfc03de52a4308bc9b045cad5aac1e; no voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:36.354440 20715 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:36.355973 10011 raft_consensus.cc:2243] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.357353 10011 raft_consensus.cc:2272] T 38d626794db04be2b9c36cc76fcf2183 P fecfc03de52a4308bc9b045cad5aac1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.362916 10011 tablet_replica.cc:333] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.363919 10011 raft_consensus.cc:2243] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.364755 10011 raft_consensus.cc:2272] T 1f9c5be972b3443eb3211fc2eb29d575 P fecfc03de52a4308bc9b045cad5aac1e [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.373817 10011 tablet_replica.cc:333] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.376730 10011 raft_consensus.cc:2243] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.378371 10011 raft_consensus.cc:2272] T 35bce44584bf45b68e6ff125c8f30655 P fecfc03de52a4308bc9b045cad5aac1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.393980 10011 tablet_replica.cc:333] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.394836 10011 raft_consensus.cc:2243] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.396070 10011 raft_consensus.cc:2272] T e26c5b0a79b0482cb39b479148fba412 P fecfc03de52a4308bc9b045cad5aac1e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.399487 10011 tablet_replica.cc:333] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.400295 10011 raft_consensus.cc:2243] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.401672 10011 raft_consensus.cc:2272] T c21b703f1a4a445a8505bb01530a8439 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.409300 10011 tablet_replica.cc:333] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.441845 10011 raft_consensus.cc:2243] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.443133 10011 raft_consensus.cc:2272] T bbf57c68621b4e90a77c76eb3974b80e P fecfc03de52a4308bc9b045cad5aac1e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.450754 20732 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.451488 20732 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.454970 11330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:36.456802 20732 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:36.467569 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:36.472771 10011 tablet_replica.cc:333] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.473868 10011 raft_consensus.cc:2243] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.474875 10011 raft_consensus.cc:2272] T d1db2221cc614cac91fdaa49f8d6cb61 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:36.477562 11900 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:36.480068 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:36.481698 20731 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:36.493466 10011 tablet_replica.cc:333] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.500007 10011 raft_consensus.cc:2243] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.501806 10011 raft_consensus.cc:2272] T ab7767a1cf4e4b5493ca9acc8daeabab P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.515590 10011 tablet_replica.cc:333] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.516546 10011 raft_consensus.cc:2243] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.518006 10011 raft_consensus.cc:2272] T 6d225f8a873c4abe992018eb9353664f P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.529461 10011 tablet_replica.cc:333] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.531838 10011 raft_consensus.cc:2243] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.536968 10011 raft_consensus.cc:2272] T aa8154ca19364187a13db1be3e0b4f53 P fecfc03de52a4308bc9b045cad5aac1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.540035 10011 tablet_replica.cc:333] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.545914 10011 raft_consensus.cc:2243] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.548734 10011 raft_consensus.cc:2272] T 28d2d114b03e403c83f7001bc117434e P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.558470 10011 tablet_replica.cc:333] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.559273 10011 raft_consensus.cc:2243] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.565644 10011 raft_consensus.cc:2272] T 25983a67e23a4c708ae496061b992207 P fecfc03de52a4308bc9b045cad5aac1e [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.569584 10011 tablet_replica.cc:333] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e: stopping tablet replica
I20260709 12:16:36.570384 10011 raft_consensus.cc:2243] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:36.571787 10011 raft_consensus.cc:2272] T 01e75f61a948459a884630690e51c686 P fecfc03de52a4308bc9b045cad5aac1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.572767 20731 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.573522 20731 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.576099 20731 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:36.600485 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:36.601892 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:36.607067 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:36.609064 20742 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:36.687666 20730 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.688426 20730 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.691401 20730 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:36.693612 20708 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.694222 20708 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:36.696547 20708 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:36.701043 11935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:36.701785 11935 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 29.
I20260709 12:16:36.713035 20730 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.713780 20730 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:36.715179 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:36.720094 20730 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:36.720805 20729 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:36.726773 20729 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.727383 20729 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:16:36.723789 11296 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:36.724920 20708 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.770210 20708 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:36.770247 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 46498us
I20260709 12:16:36.772581 20708 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:36.772862 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:36.778949 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:36.779469 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:36.781225 20747 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:36.796275 11296 leader_election.cc:336] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:36.797008 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:36.798466 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:36.798897 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:36.799685 20708 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:36.810137 20729 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.812567 20729 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:36.813894 20731 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:16:36.814477 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:16:36.814594 20731 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:36.819772 20731 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:36.822266 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c5246d860d4d559ea665671cc46b5c" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:36.822968 11931 raft_consensus.cc:3060] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:36.831614 11931 raft_consensus.cc:2468] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 30.
W20260709 12:16:36.833173 11296 leader_election.cc:336] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:36.833616 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:36.834442 20708 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:36.839011 11296 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:36.843019 20708 raft_consensus.cc:2804] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:36.844264 20708 raft_consensus.cc:697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:36.845403 20708 consensus_queue.cc:237] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:36.855015 11296 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:36.878348 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:36.879316 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:36.879729 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:36.880453 20732 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:36.899024 11077 catalog_manager.cc:5697] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 29 to 30, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 30 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.964412 20729 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.965250 20729 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:36.966202 20708 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:36.969183 20708 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:36.971936 20729 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:36.977557 20708 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:36.997046 11296 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:36.998353 11296 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:36.998782 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:36.999648 20729 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:37.016067 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.016973 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.017407 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.018273 20729 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:16:37.196246 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.200260 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:37.203639 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:16:37.224059 20729 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.224674 20729 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.227293 20729 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:37.231424 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.235312 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:37.236562 20729 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:16:37.237599 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.237227 20729 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.238154 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.239311 20749 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:37.239900 20729 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:37.240065 20749 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.240707 20749 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.243184 20749 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:37.260274 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
W20260709 12:16:37.272620 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:37.274251 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.280531 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:37.282673 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.283172 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.284026 20745 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:37.285339 11296 consensus_peers.cc:597] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:37.285861 11296 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.287735 11296 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 786851d4227e4dd89b75ccdea0b1f03b, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.288625 20745 raft_consensus.cc:2749] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:37.308861 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:16:37.309978 20732 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.310608 20732 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.312788 20732 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:37.314460 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:37.315310 11322 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 28.
I20260709 12:16:37.317941 11901 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:37.318879 20731 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:16:37.319275 20731 raft_consensus.cc:493] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.319669 20731 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:16:37.326586 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:37.339064 11900 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.349095 20745 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.349867 20745 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.352470 20745 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:37.358297 11296 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.359109 11296 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.359551 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.360409 20729 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:37.357057 20731 raft_consensus.cc:515] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting leader election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:37.360934 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:16:37.364109 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c519deb8010e411995230a6685947b97" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1"
I20260709 12:16:37.364796 11322 raft_consensus.cc:3060] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:16:37.368666 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.369740 20731 leader_election.cc:290] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:37.375224 11322 raft_consensus.cc:2468] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 786851d4227e4dd89b75ccdea0b1f03b in term 29.
W20260709 12:16:37.375959 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.388098 11901 leader_election.cc:304] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:37.388931 20748 raft_consensus.cc:2804] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:16:37.389434 20748 raft_consensus.cc:697] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 LEADER]: Becoming Leader. State: Replica: 786851d4227e4dd89b75ccdea0b1f03b, State: Running, Role: LEADER
I20260709 12:16:37.390489 20748 consensus_queue.cc:237] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.379016 20731 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.392635 20731 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.395052 20731 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:37.397087 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
W20260709 12:16:37.402184 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:37.403191 11900 leader_election.cc:336] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.405210 20731 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.405792 20731 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.409216 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:37.408986 20731 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:37.416613 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.417207 11900 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.417588 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.421867 20731 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:16:37.426041 11900 leader_election.cc:336] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.426504 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.429451 11077 catalog_manager.cc:5697] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b reported cstate change: term changed from 28 to 29, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196). New cstate: current_term: 29 leader_uuid: "786851d4227e4dd89b75ccdea0b1f03b" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:37.440089 20731 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:37.444240 20729 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.445240 20729 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.448423 20729 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:37.453081 11931 raft_consensus.cc:1275] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
W20260709 12:16:37.455354 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:37.456189 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:37.456630 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.457688 20729 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:37.460578 20745 consensus_queue.cc:1048] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:16:37.474948 11296 consensus_peers.cc:597] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.520383 20729 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.521003 20729 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.523398 20729 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:37.527096 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:37.529506 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.533146 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:37.533507 20729 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
W20260709 12:16:37.533877 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.534384 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.534161 20729 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.536571 20729 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:37.547292 20745 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:37.559514 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.565104 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.565611 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.566035 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.566953 20745 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:37.575836 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:37.579396 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:37.594012 11296 consensus_peers.cc:597] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.615682 20745 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.616348 20745 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.618559 20745 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
I20260709 12:16:37.629319 20729 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.629926 20729 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:37.631736 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:37.632674 20729 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:37.662727 11296 leader_election.cc:336] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.665160 11296 leader_election.cc:336] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.665612 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.666421 20749 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:37.673928 20749 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.674546 20749 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:37.676872 20749 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:37.672151 11296 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.678668 11296 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.679123 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.682341 20749 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:37.695940 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:37.703358 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.703955 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:37.704649 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.705025 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.705994 20749 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:37.707161 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.708755 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.715844 20748 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.716569 20748 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.719012 20748 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:37.728698 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.730736 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.731302 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.732156 20748 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:16:37.733579 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.748631 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.770315 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.781093 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:37.789860 11296 consensus_peers.cc:597] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.806974 20749 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.807648 20749 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.810596 20749 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:37.813901 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:37.814594 11931 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 31.
W20260709 12:16:37.818401 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.818933 11900 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:37.818938 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:37.820128 20749 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:16:37.820503 20749 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.820889 20749 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:16:37.825986 11296 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.832541 20731 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.835597 20731 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.837939 20731 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:37.841109 11322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "786851d4227e4dd89b75ccdea0b1f03b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" is_pre_election: true
I20260709 12:16:37.844332 20729 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.844920 20729 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.847249 20729 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:37.850332 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.851529 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:37.852283 11931 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 22.
I20260709 12:16:37.853884 20731 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.854924 20731 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.857475 20731 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:37.863270 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.872195 11900 leader_election.cc:336] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.875932 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.878070 20731 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:37.875298 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:37.876880 20749 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:37.878310 11900 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:37.879012 20729 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:37.882495 20729 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.882920 20729 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:37.881722 20749 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 32 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:37.897578 20749 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:37.898202 20749 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.900349 20749 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
I20260709 12:16:37.891966 20729 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:37.908874 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb796253655415ba47e989105d7c807" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:37.909518 11931 raft_consensus.cc:3060] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:16:37.912124 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:37.912900 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.913300 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:37.914042 20731 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:16:37.918684 11931 raft_consensus.cc:2468] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 32.
I20260709 12:16:37.920751 20729 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:37.922207 11296 leader_election.cc:336] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:37.936300 11322 raft_consensus.cc:1275] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Refusing update from remote peer 786851d4227e4dd89b75ccdea0b1f03b: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:16:37.939698 20731 consensus_queue.cc:1048] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:16:37.939872 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b" is_pre_election: true
I20260709 12:16:37.940672 11931 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 29.
I20260709 12:16:37.959478 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38d626794db04be2b9c36cc76fcf2183" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:37.960304 11931 raft_consensus.cc:3060] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:37.960721 11296 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:37.964861 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:38.109269 10011 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11278)
W20260709 12:16:37.961396 11296 leader_election.cc:336] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:38.133831 11296 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:37243, user_credentials={real_user=slave}} blocked reactor thread for 172445us
I20260709 12:16:38.135753 11296 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.136140 11931 raft_consensus.cc:2468] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 23.
I20260709 12:16:38.136709 20749 raft_consensus.cc:2804] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:16:38.137323 20749 raft_consensus.cc:697] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:38.138056 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.138880 20745 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:16:38.138765 20749 consensus_queue.cc:237] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:38.139328 20745 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.139777 20745 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:16:38.140153 11296 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:38.140893 20729 raft_consensus.cc:2804] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:38.141368 20729 raft_consensus.cc:697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:38.142349 20729 consensus_queue.cc:237] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:38.147117 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:38.152458 20745 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:38.153158 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:38.161309 20765 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.162032 20765 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:38.165156 20765 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:38.249003 20745 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Requested vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119)
W20260709 12:16:38.263290 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:38.301923 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.302531 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.303032 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:38.303424 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.304244 20765 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:38.321935 11931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf57c68621b4e90a77c76eb3974b80e" candidate_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "786851d4227e4dd89b75ccdea0b1f03b"
I20260709 12:16:38.322541 11931 raft_consensus.cc:3060] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:16:38.328389 11296 consensus_peers.cc:597] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.328919 11296 leader_election.cc:336] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:38.335368 11931 raft_consensus.cc:2468] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc92f9d2c6c5477182fa33eba4368ec1 in term 30.
I20260709 12:16:38.397878 11296 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b, bc92f9d2c6c5477182fa33eba4368ec1; no voters: 
I20260709 12:16:38.401763 20745 raft_consensus.cc:2804] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:16:38.402519 20745 raft_consensus.cc:697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Becoming Leader. State: Replica: bc92f9d2c6c5477182fa33eba4368ec1, State: Running, Role: LEADER
I20260709 12:16:38.403565 20745 consensus_queue.cc:237] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:38.414500 20729 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.415134 20729 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:38.417313 20729 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:38.435662 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:38.442752 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.443435 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.444056 11296 consensus_peers.cc:597] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.444552 11296 leader_election.cc:336] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:38.445578 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.447244 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:38.447805 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:38.468679 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:38.492364 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.501061 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:38.506147 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:38.506587 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:38.507570 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:38.510601 20745 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:38.511070 11079 catalog_manager.cc:5697] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 31 to 32, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 32 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:38.513115 11079 catalog_manager.cc:5697] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 22 to 23, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 23 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:38.515079 11079 catalog_manager.cc:5697] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 reported cstate change: term changed from 29 to 30, leader changed from fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194) to bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195). New cstate: current_term: 30 leader_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:38.546446 20765 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.547137 20765 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:38.549505 20765 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:38.554363 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:38.555660 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:38.556125 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.557397 20765 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:38.565315 20745 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.566074 20745 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:38.574743 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:16:38.577558 20745 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:38.591096 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:38.591841 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:38.592254 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.592978 20745 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:38.599524 20745 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.600075 20745 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:38.602182 20745 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:38.616601 11296 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:38.617285 11296 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:38.617699 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.618058 11931 raft_consensus.cc:1275] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:16:38.621829 20745 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:38.628660 20770 consensus_queue.cc:1048] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:16:38.641182 11935 raft_consensus.cc:1275] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:16:38.648725 20770 consensus_queue.cc:1048] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:16:38.708650 11296 consensus_peers.cc:597] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.736446 11296 consensus_peers.cc:597] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.757946 20749 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.758550 20749 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:38.760982 20749 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:38.795984 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:38.796458 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:38.798708 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:38.799118 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.800352 20749 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:16:38.841005 11296 consensus_peers.cc:597] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.855726 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:38.858322 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:16:38.913542 20770 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:38.914268 20770 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:38.924177 11296 leader_election.cc:336] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:38.930260 20770 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:38.936354 11296 leader_election.cc:336] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:38.936899 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:38.937682 20770 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:38.957652 11931 raft_consensus.cc:1275] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Refusing update from remote peer bc92f9d2c6c5477182fa33eba4368ec1: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:16:38.959280 20770 consensus_queue.cc:1048] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:16:39.059027 20745 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.059674 20745 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.062222 20745 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.071586 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:39.072727 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:39.073271 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.074136 20745 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:39.109740 10011 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 11278)
W20260709 12:16:39.117376 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:39.155646 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4740 after lost signal to thread 10014
W20260709 12:16:39.156910 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08003e11e0 after lost signal to thread 11113
W20260709 12:16:39.158381 11956 debug-util.cc:398] Leaking SignalData structure 0x7b0800386960 after lost signal to thread 11355
W20260709 12:16:39.159793 11956 debug-util.cc:398] Leaking SignalData structure 0x7b08003ec320 after lost signal to thread 11959
I20260709 12:16:39.281970 20745 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.282588 20745 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.284650 20745 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:39.285454 20745 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.286010 20745 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.288139 20745 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.295547 11296 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:39.298130 11296 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:39.301748 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.302652 20745 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:39.302850 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:39.309762 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:39.319821 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.320798 20745 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:39.336196 20745 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.336757 20745 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.338905 20745 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.385221 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:39.403849 20745 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.404433 20745 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:39.405014 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:39.405514 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.406239 20729 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:39.406921 20745 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.438387 11296 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:39.444970 11296 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:39.445669 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.449034 20765 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.454943 20729 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:39.454710 20765 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:39.457106 20765 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.466602 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:39.469998 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:39.470558 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.471884 20765 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:39.502144 20765 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.502784 20765 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:39.505235 20765 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.512921 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:39.514853 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:39.515362 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.516671 20765 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:16:39.565208 20729 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.565847 20729 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.568032 20729 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.584371 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:39.589170 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:39.594210 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.597844 20729 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:39.628022 20729 raft_consensus.cc:493] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.628612 20729 raft_consensus.cc:515] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.630770 20729 leader_election.cc:290] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115)
W20260709 12:16:39.647138 11296 leader_election.cc:336] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:39.659368 11296 leader_election.cc:336] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:39.659860 11296 leader_election.cc:304] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.660671 20729 raft_consensus.cc:2749] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:39.674615 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.693674 20745 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.694260 20745 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.696512 20745 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.702236 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.716322 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:39.741580 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:39.755735 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:39.769729 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 86c5246d860d4d559ea665671cc46b5c: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:39.771277 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:39.777841 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.781944 20729 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:39.812868 11296 consensus_peers.cc:597] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:39.842872 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.863453 11296 consensus_peers.cc:597] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.868512 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:39.882540 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.886137 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:39.921799 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.922531 20729 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:39.923092 20729 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:39.925100 20729 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:39.950093 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:39.950877 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:39.951267 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:39.953361 20788 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:39.971592 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:39.979553 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:39.988873 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:16:39.999724 11296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111) [suppressed 696 similar messages]
W20260709 12:16:40.002876 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:40.016714 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.017508 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:40.030611 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:40.090538 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.110275 10011 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 11278)
W20260709 12:16:40.171324 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:40.172021 20765 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:40.172724 20765 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:40.175262 20765 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:40.185387 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:40.188187 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:40.188762 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:40.189682 20765 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:40.217298 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:40.234243 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.237000 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:40.257299 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.275764 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:40.297712 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:40.321496 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.324664 11296 consensus_peers.cc:597] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.327741 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.337613 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:40.394810 11296 consensus_peers.cc:597] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:40.402937 11296 consensus_peers.cc:597] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.455775 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:16:40.464283 20788 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:40.464980 20788 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:40.469251 11900 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:40.468847 20788 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:40.474781 11296 consensus_peers.cc:597] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.478463 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:40.510212 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:40.513486 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:40.515496 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:40.514513 20788 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:40.528098 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:40.544643 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.549760 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.564035 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.570785 11900 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.582618 11296 consensus_peers.cc:597] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.590535 11296 consensus_peers.cc:597] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.591351 11296 consensus_peers.cc:597] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:40.631156 11296 consensus_peers.cc:597] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.666046 11296 consensus_peers.cc:597] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.726434 11296 consensus_peers.cc:597] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:40.872989 11296 consensus_peers.cc:597] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:40.877661 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:40.879592 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:40.899993 11296 consensus_peers.cc:597] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:40.952499 11296 consensus_peers.cc:597] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:40.969949 20788 raft_consensus.cc:493] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:40.970557 20788 raft_consensus.cc:515] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:40.973229 20788 leader_election.cc:290] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:40.979182 11296 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:40.980146 11296 leader_election.cc:336] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:40.980628 11296 leader_election.cc:304] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:40.981467 20729 raft_consensus.cc:2749] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:41.054736 11296 consensus_peers.cc:597] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:41.055954 20791 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.056689 20791 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:41.065407 20791 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.069386 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:41.072412 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.074208 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.075276 20791 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:16:41.092648 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:41.112665 10011 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 11278)
I20260709 12:16:41.149377 20788 raft_consensus.cc:493] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.150089 20788 raft_consensus.cc:515] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:41.151688 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:16:41.152372 20788 leader_election.cc:290] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.174858 11296 consensus_peers.cc:597] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:41.175530 11296 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.176805 20729 raft_consensus.cc:493] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.177354 20729 raft_consensus.cc:515] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:41.179569 20729 leader_election.cc:290] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.180840 11296 leader_election.cc:336] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:41.181383 11296 leader_election.cc:304] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.182255 20729 raft_consensus.cc:2749] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:41.182483 11296 consensus_peers.cc:597] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:16:41.191675 20788 raft_consensus.cc:493] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.192256 20788 raft_consensus.cc:515] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:41.194578 20788 leader_election.cc:290] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.197578 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:41.200820 11296 leader_election.cc:336] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:41.201565 11296 leader_election.cc:304] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.214733 20791 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.222638 20791 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:41.225809 20791 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:41.227203 20788 raft_consensus.cc:2749] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:41.230243 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:41.231112 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.232146 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:41.232685 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.233307 20791 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:16:41.245991 11296 leader_election.cc:336] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:41.246846 11296 leader_election.cc:304] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.247701 20788 raft_consensus.cc:2749] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:41.279771 20788 raft_consensus.cc:493] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.280369 20788 raft_consensus.cc:515] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:41.282648 20788 leader_election.cc:290] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.283627 11296 consensus_peers.cc:597] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:41.329281 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:41.340390 11296 leader_election.cc:336] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.340884 11296 leader_election.cc:304] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.344702 20788 raft_consensus.cc:2749] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:41.406090 20788 raft_consensus.cc:493] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.406664 20788 raft_consensus.cc:515] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:41.408850 20788 leader_election.cc:290] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.427937 11296 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:41.428864 11296 leader_election.cc:336] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.429248 11296 leader_election.cc:304] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.430063 20788 raft_consensus.cc:2749] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:41.478636 20729 raft_consensus.cc:493] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.479195 20729 raft_consensus.cc:515] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:41.481269 20729 leader_election.cc:290] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.489522 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:41.492112 11296 leader_election.cc:336] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.492594 11296 leader_election.cc:304] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.495766 20729 raft_consensus.cc:2749] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:41.523499 11296 consensus_peers.cc:597] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:16:41.565033 20788 raft_consensus.cc:493] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.565608 20788 raft_consensus.cc:515] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:41.567878 20788 leader_election.cc:290] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.574849 11296 leader_election.cc:336] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:41.577821 11296 leader_election.cc:336] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.578291 11296 leader_election.cc:304] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.579101 20788 raft_consensus.cc:2749] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:41.855628 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:16:41.868290 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
I20260709 12:16:41.965366 20793 raft_consensus.cc:493] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:41.966013 20793 raft_consensus.cc:515] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:41.968150 20793 leader_election.cc:290] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:41.976173 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:41.976847 11296 leader_election.cc:336] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:41.977242 11296 leader_election.cc:304] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:41.978024 20793 raft_consensus.cc:2749] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:42.010354 20793 raft_consensus.cc:493] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:42.010874 20793 raft_consensus.cc:515] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:42.014263 20793 leader_election.cc:290] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:42.024294 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:42.025204 11296 leader_election.cc:336] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:42.025688 11296 leader_election.cc:304] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:42.026564 20793 raft_consensus.cc:2749] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:42.072292 20793 raft_consensus.cc:493] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:42.072893 20793 raft_consensus.cc:515] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:42.074357 20791 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:42.074908 20791 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:42.077114 20791 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:42.079684 20793 leader_election.cc:290] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:42.086087 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:42.088095 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:42.088660 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:42.091156 20792 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:16:42.141780 20793 raft_consensus.cc:493] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:42.142470 20793 raft_consensus.cc:515] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
W20260709 12:16:42.143996 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:42.145205 20793 leader_election.cc:290] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:42.147264 11296 leader_election.cc:336] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:42.154904 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:42.159212 11296 leader_election.cc:304] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:42.161149 20788 raft_consensus.cc:2749] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:42.164579 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:42.175627 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:42.183192 11296 leader_election.cc:336] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:42.184640 11296 leader_election.cc:304] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:42.186157 20788 raft_consensus.cc:2749] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:42.185350 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.232152 11296 consensus_peers.cc:597] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:42.241851 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
W20260709 12:16:42.251363 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:42.263891 11901 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:42.269254 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:16:42.273236 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:16:42.273417 11901 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:42.273926 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:16:42.323297 11296 consensus_peers.cc:597] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.326646 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:16:42.330360 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:42.361867 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:42.363103 10011 tablet_replica.cc:333] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.364203 10011 raft_consensus.cc:2243] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.373935 10011 raft_consensus.cc:2272] T 9fdf86dcb01f4d81913c37b45cfc3879 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.374889 11901 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.377715 10011 tablet_replica.cc:333] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.378851 10011 raft_consensus.cc:2243] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.380124 10011 raft_consensus.cc:2272] T 458129fddd7e4b33b63d818f6af21f23 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.384166 10011 tablet_replica.cc:333] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.385249 10011 raft_consensus.cc:2243] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.387336 10011 raft_consensus.cc:2272] T 8eaf18bb7a144d35b42cf5548b769c0f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.390571 10011 tablet_replica.cc:333] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.392195 10011 raft_consensus.cc:2243] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.393106 10011 raft_consensus.cc:2272] T e03893c6062f45ea972113cfa57ba362 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.403136 10011 tablet_replica.cc:333] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.404036 10011 raft_consensus.cc:2243] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.405903 10011 raft_consensus.cc:2272] T 86c5246d860d4d559ea665671cc46b5c P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.408872 10011 tablet_replica.cc:333] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.410497 10011 raft_consensus.cc:2243] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.411414 10011 raft_consensus.cc:2272] T 4c40179116e14aa2aff8e95298cbcd4f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.415134 10011 tablet_replica.cc:333] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
W20260709 12:16:42.415843 11901 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.416057 10011 raft_consensus.cc:2243] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.417405 10011 raft_consensus.cc:2272] T fab915731e0347ae8353a7f300c44401 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.421407 10011 tablet_replica.cc:333] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.422329 10011 raft_consensus.cc:2243] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.423199 10011 raft_consensus.cc:2272] T 7cba0650205b4eb0b416158a057a90b9 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.432650 10011 tablet_replica.cc:333] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.433684 10011 raft_consensus.cc:2243] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.434752 10011 raft_consensus.cc:2272] T 16e4ceea03b94eb2803569189ab5fcca P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.439079 20793 raft_consensus.cc:493] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:42.439702 20793 raft_consensus.cc:515] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } }
I20260709 12:16:42.443018 20793 leader_election.cc:290] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:42.443706 11901 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:42.447602 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:42.437867 10011 tablet_replica.cc:333] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
W20260709 12:16:42.448308 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:42.449095 11296 leader_election.cc:336] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:42.449218 10011 raft_consensus.cc:2243] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.449550 11296 leader_election.cc:304] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc92f9d2c6c5477182fa33eba4368ec1; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:42.450840 10011 raft_consensus.cc:2272] T b9dab54d96124869a4a52f9cc08e288a P bc92f9d2c6c5477182fa33eba4368ec1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.450881 20793 raft_consensus.cc:2749] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:42.454176 10011 tablet_replica.cc:333] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
W20260709 12:16:42.463166 11901 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.464512 10011 raft_consensus.cc:2243] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.466251 10011 raft_consensus.cc:2272] T d49a939bed404bfcb2538b5331071d3b P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.469313 10011 tablet_replica.cc:333] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.470527 10011 raft_consensus.cc:2243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.471764 10011 raft_consensus.cc:2272] T 9424adcaacdf40f3bbd2f639ef2a48c2 P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.475443 10011 tablet_replica.cc:333] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.477053 10011 raft_consensus.cc:2243] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.478809 10011 raft_consensus.cc:2272] T f62903fc48c140aca4aa949625fa88f6 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.482105 10011 tablet_replica.cc:333] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.482982 10011 raft_consensus.cc:2243] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.484941 10011 raft_consensus.cc:2272] T 3e235bf9f3514f629a5c26c1eac58682 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.495767 10011 tablet_replica.cc:333] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.496676 10011 raft_consensus.cc:2243] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.498106 10011 raft_consensus.cc:2272] T 4a634acd9ff543cca551c8a28e72d6c4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.501751 10011 tablet_replica.cc:333] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.502604 10011 raft_consensus.cc:2243] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.503525 10011 raft_consensus.cc:2272] T 4958b19b61094e1db763fdb94852f994 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.504869 11901 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.506510 10011 tablet_replica.cc:333] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.507513 10011 raft_consensus.cc:2243] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.508929 10011 raft_consensus.cc:2272] T 1f9c5be972b3443eb3211fc2eb29d575 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.512131 10011 tablet_replica.cc:333] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.513129 10011 raft_consensus.cc:2243] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.514745 10011 raft_consensus.cc:2272] T 35bce44584bf45b68e6ff125c8f30655 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.518515 10011 tablet_replica.cc:333] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.519500 10011 raft_consensus.cc:2243] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.521111 10011 raft_consensus.cc:2272] T 38d626794db04be2b9c36cc76fcf2183 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.524247 10011 tablet_replica.cc:333] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.525579 10011 raft_consensus.cc:2243] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.536243 10011 raft_consensus.cc:2272] T 5fc47889c90446dd927b2ac3010d58fd P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.539980 10011 tablet_replica.cc:333] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.540903 10011 raft_consensus.cc:2243] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.542177 10011 raft_consensus.cc:2272] T 00b8260131ab49ecb2df9631e5704096 P bc92f9d2c6c5477182fa33eba4368ec1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.551345 10011 tablet_replica.cc:333] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
W20260709 12:16:42.551931 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:42.552270 10011 raft_consensus.cc:2243] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.553125 10011 raft_consensus.cc:2272] T 0fd765747f9648049b905c70696b206d P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.556296 10011 tablet_replica.cc:333] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.557178 10011 raft_consensus.cc:2243] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.558892 10011 raft_consensus.cc:2272] T 6f62270698314885bc0e03e0de4507ae P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.561986 10011 tablet_replica.cc:333] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.562924 10011 raft_consensus.cc:2243] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.563558 10011 raft_consensus.cc:2272] T c519deb8010e411995230a6685947b97 P bc92f9d2c6c5477182fa33eba4368ec1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.566504 10011 tablet_replica.cc:333] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.567378 10011 raft_consensus.cc:2243] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.568216 10011 raft_consensus.cc:2272] T 80d10b7c9e4644e9a03edf10767df625 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.571803 10011 tablet_replica.cc:333] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.572633 10011 raft_consensus.cc:2243] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.573346 10011 raft_consensus.cc:2272] T 02e478aac72b4a62a8ed9124187f2dec P bc92f9d2c6c5477182fa33eba4368ec1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.576133 10011 tablet_replica.cc:333] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.576867 10011 raft_consensus.cc:2243] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.577445 10011 raft_consensus.cc:2272] T 2fdc8cd3021d4b678786056187782c16 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.580191 10011 tablet_replica.cc:333] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.580911 10011 raft_consensus.cc:2243] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.582036 10011 raft_consensus.cc:2272] T 1fb796253655415ba47e989105d7c807 P bc92f9d2c6c5477182fa33eba4368ec1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.584704 10011 tablet_replica.cc:333] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.585460 10011 raft_consensus.cc:2243] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.586436 10011 raft_consensus.cc:2272] T d5a029ce312749bdaf2113d2823e993e P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.589603 10011 tablet_replica.cc:333] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.590560 10011 raft_consensus.cc:2243] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.593287 10011 raft_consensus.cc:2272] T d97e23379bda4a1b9ad1e2eabbf4d82f P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.593622 11296 consensus_peers.cc:597] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:42.597218 10011 tablet_replica.cc:333] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.598225 10011 raft_consensus.cc:2243] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.599805 10011 raft_consensus.cc:2272] T 6c6a39d516e943a9b2c8c4213ac11902 P bc92f9d2c6c5477182fa33eba4368ec1 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.608736 11296 consensus_peers.cc:597] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:16:42.609368 11900 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:42.612200 10011 tablet_replica.cc:333] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.613126 10011 raft_consensus.cc:2243] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:42.613889 11901 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.614324 10011 raft_consensus.cc:2272] T a629d78b61074c2cbd007926bd78fa29 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.618432 10011 tablet_replica.cc:333] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.619335 10011 raft_consensus.cc:2243] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.619978 10011 raft_consensus.cc:2272] T 73b50321123c433f91c5d0972791cf37 P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.622916 10011 tablet_replica.cc:333] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.623764 10011 raft_consensus.cc:2243] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.624984 10011 raft_consensus.cc:2272] T 88ddcb89ab6d47cd97e8c19734f0e365 P bc92f9d2c6c5477182fa33eba4368ec1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.627900 10011 tablet_replica.cc:333] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.628733 10011 raft_consensus.cc:2243] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.630494 10011 raft_consensus.cc:2272] T f2d37a48723c42228c9717f514174704 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.630929 11901 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.635457 10011 tablet_replica.cc:333] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.644618 10011 raft_consensus.cc:2243] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.646001 10011 raft_consensus.cc:2272] T 06a9ec0400af47eeaffc85f3f1d38de4 P bc92f9d2c6c5477182fa33eba4368ec1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.649017 10011 tablet_replica.cc:333] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.649909 10011 raft_consensus.cc:2243] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.650905 10011 raft_consensus.cc:2272] T dedbfb6dda024d1e81b2740f108080af P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.653838 10011 tablet_replica.cc:333] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.654625 10011 raft_consensus.cc:2243] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.655648 10011 raft_consensus.cc:2272] T 39571de4248f4cfd85f9b1062a776cc8 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.667333 10011 tablet_replica.cc:333] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.668167 10011 raft_consensus.cc:2243] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.668805 10011 raft_consensus.cc:2272] T 01e75f61a948459a884630690e51c686 P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.671681 10011 tablet_replica.cc:333] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.672443 10011 raft_consensus.cc:2243] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.673674 10011 raft_consensus.cc:2272] T d1db2221cc614cac91fdaa49f8d6cb61 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.676815 10011 tablet_replica.cc:333] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.677623 10011 raft_consensus.cc:2243] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.679318 10011 raft_consensus.cc:2272] T bbf57c68621b4e90a77c76eb3974b80e P bc92f9d2c6c5477182fa33eba4368ec1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.682345 10011 tablet_replica.cc:333] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.684427 10011 raft_consensus.cc:2243] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.685673 10011 raft_consensus.cc:2272] T aa8154ca19364187a13db1be3e0b4f53 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.694051 10011 tablet_replica.cc:333] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.694844 10011 raft_consensus.cc:2243] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.695598 10011 raft_consensus.cc:2272] T 28d2d114b03e403c83f7001bc117434e P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.696321 11901 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.698401 10011 tablet_replica.cc:333] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.699296 10011 raft_consensus.cc:2243] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.700865 10011 raft_consensus.cc:2272] T 99332ff05c3e46579ef9ff825e42a628 P bc92f9d2c6c5477182fa33eba4368ec1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.709393 10011 tablet_replica.cc:333] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.710256 10011 raft_consensus.cc:2243] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.711172 10011 raft_consensus.cc:2272] T c21b703f1a4a445a8505bb01530a8439 P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.714210 10011 tablet_replica.cc:333] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.714986 10011 raft_consensus.cc:2243] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.716240 10011 raft_consensus.cc:2272] T e26c5b0a79b0482cb39b479148fba412 P bc92f9d2c6c5477182fa33eba4368ec1 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.723264 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:16:42.723945 10011 tablet_replica.cc:333] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.724704 10011 raft_consensus.cc:2243] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.726686 10011 raft_consensus.cc:2272] T ab7767a1cf4e4b5493ca9acc8daeabab P bc92f9d2c6c5477182fa33eba4368ec1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.729461 10011 tablet_replica.cc:333] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1: stopping tablet replica
I20260709 12:16:42.730137 10011 raft_consensus.cc:2243] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:42.730928 11901 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.731424 10011 raft_consensus.cc:2272] T 6d225f8a873c4abe992018eb9353664f P bc92f9d2c6c5477182fa33eba4368ec1 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.780655 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.785317 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:16:42.805011 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.819159 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:42.829277 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.847230 11900 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.923127 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:43.196617 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:43.203568 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:43.309269 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:16:43.457377 20798 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.458137 20798 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.460887 20798 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.465790 11900 leader_election.cc:336] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:43.467489 11901 leader_election.cc:336] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:43.468083 11901 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:43.469007 20798 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:43.565672 20792 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.566388 20792 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.569164 20792 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.574406 11901 leader_election.cc:336] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.576288 11900 leader_election.cc:336] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:43.576902 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:43.577780 20792 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:43.579756 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:16:43.653448 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:16:43.664598 20792 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.665386 20792 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.668782 20792 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.675953 11901 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:43.679903 20797 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.680534 20797 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.682822 20797 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:43.682862 20792 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.683599 20792 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.686023 20792 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.690024 11900 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:43.690589 11900 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:43.694830 20792 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:43.695420 11901 leader_election.cc:336] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.696169 11901 leader_election.cc:336] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.698684 11900 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:43.704725 11900 leader_election.cc:336] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:43.705207 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:43.706050 20792 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:43.706801 11900 leader_election.cc:336] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.707381 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:43.708218 20792 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:43.718277 20792 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.718976 20792 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.723393 20792 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.727046 11901 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:43.728045 20797 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.728621 20797 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.730881 20797 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.737393 11900 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:43.737891 11900 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:43.739342 20798 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:43.741026 11900 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:43.743657 11901 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.746402 11900 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.746876 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:43.747704 20792 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:16:43.769075 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:16:43.778199 20792 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.778839 20792 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.781551 20792 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:43.783910 20798 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.784536 20798 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.787043 20798 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.789505 11901 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.793941 11900 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.794898 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:43.796343 11901 leader_election.cc:336] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:43.796769 20798 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:16:43.799736 11900 leader_election.cc:336] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.800221 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:43.801036 20798 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:43.851264 20798 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.851989 20798 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.857878 20798 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.864584 11901 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.871088 11900 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:43.871618 11900 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:43.872428 20798 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:16:43.872560 11900 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.885586 20798 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.886642 20797 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.886164 20798 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.887941 20792 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.888494 20792 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.890705 20792 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:43.887166 20797 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.893287 20797 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:43.897337 20798 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.900207 11901 leader_election.cc:336] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.900961 11901 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.904402 11901 leader_election.cc:336] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:43.906116 11900 leader_election.cc:336] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.906656 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:43.907589 11900 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.908044 11900 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:43.908861 20802 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:43.909615 20802 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:43.912740 11900 leader_election.cc:336] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:43.913240 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:43.914316 20798 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:43.933965 11352 debug-util.cc:398] Leaking SignalData structure 0x7b0800303840 after lost signal to thread 10014
W20260709 12:16:43.935145 11352 debug-util.cc:398] Leaking SignalData structure 0x7b08003588a0 after lost signal to thread 11113
W20260709 12:16:43.936684 11352 debug-util.cc:398] Leaking SignalData structure 0x7b0800037420 after lost signal to thread 11959
W20260709 12:16:43.958900 10011 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11352)
I20260709 12:16:43.968521 20802 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:43.969182 20802 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:43.971650 20802 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:43.981287 11900 leader_election.cc:336] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:43.989758 11901 leader_election.cc:336] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:43.990334 11901 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:43.992015 20798 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:16:44.133924 20798 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:44.134625 20798 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:44.138978 20798 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:44.145505 11901 leader_election.cc:336] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:44.154198 11900 leader_election.cc:336] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:44.154791 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:44.155673 20798 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:44.532850 20798 raft_consensus.cc:493] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:44.535377 20798 raft_consensus.cc:515] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:44.538437 20798 leader_election.cc:290] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:44.539446 11900 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:16:44.551326 11900 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:16:44.552067 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:44.552825 11900 leader_election.cc:336] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:44.553267 11900 leader_election.cc:304] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:44.554126 20798 raft_consensus.cc:2749] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:16:44.570423 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:44.646517 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:44.691557 11900 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 12:16:44.708362 20798 raft_consensus.cc:493] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:44.708923 20798 raft_consensus.cc:515] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:44.711104 20798 leader_election.cc:290] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:44.719285 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:44.720866 11900 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:44.728358 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:44.731163 11900 leader_election.cc:336] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:44.732865 11900 leader_election.cc:304] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:44.733683 20798 raft_consensus.cc:2749] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:16:44.739827 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:44.782965 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:16:44.835476 11900 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:16:44.871279 11901 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:44.876298 11901 consensus_peers.cc:597] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:44.907934 11901 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:44.939577 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fab915731e0347ae8353a7f300c44401: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:44.940635 11901 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:44.949441 11901 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:44.959496 10011 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 11352)
W20260709 12:16:45.006035 11901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111) [suppressed 566 similar messages]
W20260709 12:16:45.009845 11901 consensus_peers.cc:597] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:45.046841 11901 consensus_peers.cc:597] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:45.083096 11901 consensus_peers.cc:597] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:45.123914 11900 consensus_peers.cc:597] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.145473 11900 consensus_peers.cc:597] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:16:45.242910 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.249117 11900 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.285167 11900 consensus_peers.cc:597] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.289134 11900 consensus_peers.cc:597] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.309860 11900 consensus_peers.cc:597] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 12:16:45.340540 20798 raft_consensus.cc:493] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:45.341102 20798 raft_consensus.cc:515] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.343139 20798 leader_election.cc:290] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
I20260709 12:16:45.344154 20798 raft_consensus.cc:493] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.344688 20798 raft_consensus.cc:515] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:45.352005 11901 leader_election.cc:336] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.352850 20798 leader_election.cc:290] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.362213 11900 consensus_peers.cc:597] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:45.368068 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:45.373083 11900 leader_election.cc:336] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.373651 11900 leader_election.cc:304] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.374459 20798 raft_consensus.cc:2749] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:16:45.377127 11900 leader_election.cc:336] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.377657 11900 leader_election.cc:304] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.379730 20798 raft_consensus.cc:2749] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:45.380883 11900 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:45.405875 20798 raft_consensus.cc:493] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.406436 20798 raft_consensus.cc:515] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.408733 20798 leader_election.cc:290] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.414027 11901 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:45.427383 11900 leader_election.cc:336] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.428035 11900 leader_election.cc:304] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.428829 20798 raft_consensus.cc:2749] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:16:45.441226 11901 consensus_peers.cc:597] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:45.477587 20798 raft_consensus.cc:493] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader fecfc03de52a4308bc9b045cad5aac1e)
I20260709 12:16:45.478190 20798 raft_consensus.cc:515] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.480573 20798 leader_election.cc:290] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243)
W20260709 12:16:45.481832 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.493381 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:45.495316 11900 leader_election.cc:336] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.495890 11900 leader_election.cc:304] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.496743 20798 raft_consensus.cc:2749] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:16:45.508118 11901 consensus_peers.cc:597] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:45.516891 20798 raft_consensus.cc:493] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.517551 20798 raft_consensus.cc:515] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.521236 20798 leader_election.cc:290] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.521863 11901 consensus_peers.cc:597] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Couldn't send request to peer bc92f9d2c6c5477182fa33eba4368ec1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:45.527807 11901 leader_election.cc:336] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:45.535316 11900 leader_election.cc:336] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.535990 11900 leader_election.cc:304] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.536945 20798 raft_consensus.cc:2749] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:16:45.587762 11900 consensus_peers.cc:597] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:16:45.598016 20798 raft_consensus.cc:493] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.598560 20798 raft_consensus.cc:515] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.600741 20798 leader_election.cc:290] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.606294 11901 leader_election.cc:336] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.607474 20798 raft_consensus.cc:493] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.608116 20798 raft_consensus.cc:515] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.611110 20798 leader_election.cc:290] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.619095 11901 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.620728 20798 raft_consensus.cc:493] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.621340 20798 raft_consensus.cc:515] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.626529 20798 leader_election.cc:290] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.671285 11900 leader_election.cc:336] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:45.642057 11901 leader_election.cc:336] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.676626 11900 leader_election.cc:304] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:45.679270 11900 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.680001 11900 leader_election.cc:304] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.688150 20808 raft_consensus.cc:2749] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:45.691926 20808 raft_consensus.cc:2749] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:45.697839 11900 leader_election.cc:336] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:45.699997 11900 leader_election.cc:336] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:45.700585 11900 leader_election.cc:304] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:45.704780 20807 raft_consensus.cc:2749] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:16:45.706504 20807 raft_consensus.cc:493] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.707463 20807 raft_consensus.cc:515] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.710803 20807 leader_election.cc:290] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.718405 11900 consensus_peers.cc:597] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:16:45.720377 11901 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:45.721490 11900 leader_election.cc:336] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.721944 11900 leader_election.cc:304] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.722828 20809 raft_consensus.cc:2749] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:16:45.733510 20809 raft_consensus.cc:493] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.734442 20809 raft_consensus.cc:515] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.737510 20809 leader_election.cc:290] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.743404 11901 leader_election.cc:336] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.748163 20809 raft_consensus.cc:493] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
W20260709 12:16:45.751056 11900 leader_election.cc:336] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.749145 20809 raft_consensus.cc:515] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.754184 20809 leader_election.cc:290] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:45.751649 11900 leader_election.cc:304] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.755115 20809 raft_consensus.cc:2749] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:45.759218 11901 leader_election.cc:336] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.782521 20809 raft_consensus.cc:493] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.783640 20809 raft_consensus.cc:515] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:45.794327 11901 leader_election.cc:336] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.795315 20809 leader_election.cc:290] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.798274 11900 leader_election.cc:336] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.800657 11900 leader_election.cc:304] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.801476 20809 raft_consensus.cc:2749] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:16:45.817986 11900 leader_election.cc:336] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.818522 11900 leader_election.cc:304] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:45.819399 20809 raft_consensus.cc:493] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.820267 20809 raft_consensus.cc:515] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:45.826015 20808 raft_consensus.cc:2749] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:45.830628 11901 leader_election.cc:336] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.831627 20809 leader_election.cc:290] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.832604 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Couldn't send request to peer fecfc03de52a4308bc9b045cad5aac1e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.845743 11900 leader_election.cc:336] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:45.846324 11900 leader_election.cc:304] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, bc92f9d2c6c5477182fa33eba4368ec1
I20260709 12:16:45.847216 20809 raft_consensus.cc:2749] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:45.928256 20809 raft_consensus.cc:493] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:45.929266 20809 raft_consensus.cc:515] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
W20260709 12:16:45.937974 11900 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:45.939857 20809 leader_election.cc:290] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:45.952106 11900 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:45.952598 11900 leader_election.cc:304] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:45.953411 11901 leader_election.cc:336] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:45.955916 20808 raft_consensus.cc:2749] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:16:46.099994 20808 raft_consensus.cc:493] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:46.101073 20808 raft_consensus.cc:515] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "016411114c6d42039bd04025378dfc98" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45115 } } peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } attrs { replace: true } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:46.104302 20808 leader_election.cc:290] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115), fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:46.292872 20808 raft_consensus.cc:493] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:46.293735 20808 raft_consensus.cc:515] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:46.296969 20808 leader_election.cc:290] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
I20260709 12:16:46.320511 20807 raft_consensus.cc:493] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader bc92f9d2c6c5477182fa33eba4368ec1)
I20260709 12:16:46.321316 20807 raft_consensus.cc:515] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fecfc03de52a4308bc9b045cad5aac1e" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 37243 } } peers { permanent_uuid: "bc92f9d2c6c5477182fa33eba4368ec1" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42967 } } peers { permanent_uuid: "786851d4227e4dd89b75ccdea0b1f03b" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 39119 } attrs { promote: false } }
I20260709 12:16:46.323925 20807 leader_election.cc:290] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243), bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967)
W20260709 12:16:46.325796 11900 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:46.326507 11900 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
W20260709 12:16:46.328872 11900 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
I20260709 12:16:46.329270 11900 leader_election.cc:304] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: 016411114c6d42039bd04025378dfc98, fecfc03de52a4308bc9b045cad5aac1e
W20260709 12:16:46.332773 11901 leader_election.cc:336] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:46.333240 11901 leader_election.cc:336] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
I20260709 12:16:46.333591 11901 leader_election.cc:304] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:46.334067 20813 raft_consensus.cc:2749] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:16:46.334548 20798 raft_consensus.cc:2749] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:16:46.334757 11900 consensus_peers.cc:597] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:16:46.336164 11901 leader_election.cc:336] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer bc92f9d2c6c5477182fa33eba4368ec1 (127.9.198.195:42967): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42967: connect: Connection refused (error 111)
W20260709 12:16:46.343335 11900 consensus_peers.cc:597] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:16:46.347499 11900 leader_election.cc:336] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer fecfc03de52a4308bc9b045cad5aac1e (127.9.198.194:37243): Network error: Client connection negotiation failed: client connection to 127.9.198.194:37243: connect: Connection refused (error 111)
I20260709 12:16:46.348084 11900 leader_election.cc:304] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 786851d4227e4dd89b75ccdea0b1f03b; no voters: bc92f9d2c6c5477182fa33eba4368ec1, fecfc03de52a4308bc9b045cad5aac1e
I20260709 12:16:46.349610 20798 raft_consensus.cc:2749] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:46.350350 11900 consensus_peers.cc:597] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b -> Peer 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): Couldn't send request to peer 016411114c6d42039bd04025378dfc98. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 12:16:46.406562 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:16:46.568589 10011 tablet_server.cc:179] TabletServer@127.9.198.196:0 shutting down...
I20260709 12:16:46.635527 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:46.638877 10011 tablet_replica.cc:333] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.640415 10011 raft_consensus.cc:2243] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.642247 10011 raft_consensus.cc:2272] T fab915731e0347ae8353a7f300c44401 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.647393 10011 tablet_replica.cc:333] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.648651 10011 raft_consensus.cc:2243] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.650177 10011 raft_consensus.cc:2272] T d97e23379bda4a1b9ad1e2eabbf4d82f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.654798 10011 tablet_replica.cc:333] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.655805 10011 raft_consensus.cc:2243] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.656777 10011 raft_consensus.cc:2272] T 8eaf18bb7a144d35b42cf5548b769c0f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.659962 10011 tablet_replica.cc:333] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.660820 10011 raft_consensus.cc:2243] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.661765 10011 raft_consensus.cc:2272] T ab7767a1cf4e4b5493ca9acc8daeabab P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.665096 10011 tablet_replica.cc:333] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.665936 10011 raft_consensus.cc:2243] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.667958 10011 raft_consensus.cc:2272] T c519deb8010e411995230a6685947b97 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.671314 10011 tablet_replica.cc:333] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.672040 10011 raft_consensus.cc:2243] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.673364 10011 raft_consensus.cc:2272] T 5df07764623640ce8ec29ad695812b7c P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.676298 10011 tablet_replica.cc:333] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.677134 10011 raft_consensus.cc:2243] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.678540 10011 raft_consensus.cc:2272] T 6d225f8a873c4abe992018eb9353664f P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.681267 10011 tablet_replica.cc:333] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.682076 10011 raft_consensus.cc:2243] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.683216 10011 raft_consensus.cc:2272] T 86c5246d860d4d559ea665671cc46b5c P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.686295 10011 tablet_replica.cc:333] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.687161 10011 raft_consensus.cc:2243] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.688663 10011 raft_consensus.cc:2272] T 4a634acd9ff543cca551c8a28e72d6c4 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.691260 10011 tablet_replica.cc:333] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.691953 10011 raft_consensus.cc:2243] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.692533 10011 raft_consensus.cc:2272] T e26c5b0a79b0482cb39b479148fba412 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.695750 10011 tablet_replica.cc:333] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.696527 10011 raft_consensus.cc:2243] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.697592 10011 raft_consensus.cc:2272] T 0a6ec18bb8c54ece99a794e109201789 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.701184 10011 tablet_replica.cc:333] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.701995 10011 raft_consensus.cc:2243] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.703182 10011 raft_consensus.cc:2272] T 25983a67e23a4c708ae496061b992207 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.706212 10011 tablet_replica.cc:333] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.707093 10011 raft_consensus.cc:2243] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.708724 10011 raft_consensus.cc:2272] T 89068ffedf554d89af62a8b1fe0700c0 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.711912 10011 tablet_replica.cc:333] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.712571 10011 raft_consensus.cc:2243] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.713789 10011 raft_consensus.cc:2272] T 7f736e38db6b45bfa87e35af564bffc1 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.716372 10011 tablet_replica.cc:333] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.717283 10011 raft_consensus.cc:2243] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.719022 10011 raft_consensus.cc:2272] T ddf8cca10e1244d8882abd13fa81113a P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.722806 10011 tablet_replica.cc:333] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.723523 10011 raft_consensus.cc:2243] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.724277 10011 raft_consensus.cc:2272] T 9424adcaacdf40f3bbd2f639ef2a48c2 P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.727165 10011 tablet_replica.cc:333] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.727792 10011 raft_consensus.cc:2243] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.729020 10011 raft_consensus.cc:2272] T e5f804d45235415fad0f1442ea04aa47 P 786851d4227e4dd89b75ccdea0b1f03b [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.731609 10011 tablet_replica.cc:333] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.732342 10011 raft_consensus.cc:2243] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.733106 10011 raft_consensus.cc:2272] T 38d626794db04be2b9c36cc76fcf2183 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.736019 10011 tablet_replica.cc:333] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.736636 10011 raft_consensus.cc:2243] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.737408 10011 raft_consensus.cc:2272] T 6f62270698314885bc0e03e0de4507ae P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.739748 10011 tablet_replica.cc:333] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.740363 10011 raft_consensus.cc:2243] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.741652 10011 raft_consensus.cc:2272] T 02e478aac72b4a62a8ed9124187f2dec P 786851d4227e4dd89b75ccdea0b1f03b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.744357 10011 tablet_replica.cc:333] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.744963 10011 raft_consensus.cc:2243] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.745939 10011 raft_consensus.cc:2272] T f62903fc48c140aca4aa949625fa88f6 P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.749085 10011 tablet_replica.cc:333] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.749949 10011 raft_consensus.cc:2243] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.751541 10011 raft_consensus.cc:2272] T 80d10b7c9e4644e9a03edf10767df625 P 786851d4227e4dd89b75ccdea0b1f03b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.754626 10011 tablet_replica.cc:333] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.755404 10011 raft_consensus.cc:2243] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.756107 10011 raft_consensus.cc:2272] T 5ad54acf641349bc8119ce4c0eae446b P 786851d4227e4dd89b75ccdea0b1f03b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.758828 10011 tablet_replica.cc:333] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.759470 10011 raft_consensus.cc:2243] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.760705 10011 raft_consensus.cc:2272] T 4c40179116e14aa2aff8e95298cbcd4f P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.763113 10011 tablet_replica.cc:333] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.763770 10011 raft_consensus.cc:2243] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.764824 10011 raft_consensus.cc:2272] T a629d78b61074c2cbd007926bd78fa29 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.767251 10011 tablet_replica.cc:333] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.767863 10011 raft_consensus.cc:2243] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.768502 10011 raft_consensus.cc:2272] T bbf57c68621b4e90a77c76eb3974b80e P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.771085 10011 tablet_replica.cc:333] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.771698 10011 raft_consensus.cc:2243] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.772445 10011 raft_consensus.cc:2272] T aa8154ca19364187a13db1be3e0b4f53 P 786851d4227e4dd89b75ccdea0b1f03b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.774885 10011 tablet_replica.cc:333] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.775449 10011 raft_consensus.cc:2243] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.776609 10011 raft_consensus.cc:2272] T 5fa73285733a4a67a3e705a20155f69d P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.779130 10011 tablet_replica.cc:333] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.779744 10011 raft_consensus.cc:2243] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.780974 10011 raft_consensus.cc:2272] T 458129fddd7e4b33b63d818f6af21f23 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.783440 10011 tablet_replica.cc:333] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.784061 10011 raft_consensus.cc:2243] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.784668 10011 raft_consensus.cc:2272] T 1fb796253655415ba47e989105d7c807 P 786851d4227e4dd89b75ccdea0b1f03b [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.787284 10011 tablet_replica.cc:333] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.787884 10011 raft_consensus.cc:2243] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.789006 10011 raft_consensus.cc:2272] T 39571de4248f4cfd85f9b1062a776cc8 P 786851d4227e4dd89b75ccdea0b1f03b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.791479 10011 tablet_replica.cc:333] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.792057 10011 raft_consensus.cc:2243] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.793370 10011 raft_consensus.cc:2272] T 53e6b498179b4c96b8b0be6240a36efb P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.795853 10011 tablet_replica.cc:333] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.796425 10011 raft_consensus.cc:2243] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.797736 10011 raft_consensus.cc:2272] T 99332ff05c3e46579ef9ff825e42a628 P 786851d4227e4dd89b75ccdea0b1f03b [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.800148 10011 tablet_replica.cc:333] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.800710 10011 raft_consensus.cc:2243] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.801252 10011 raft_consensus.cc:2272] T 88ddcb89ab6d47cd97e8c19734f0e365 P 786851d4227e4dd89b75ccdea0b1f03b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.803622 10011 tablet_replica.cc:333] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.804178 10011 raft_consensus.cc:2243] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.805190 10011 raft_consensus.cc:2272] T 73b50321123c433f91c5d0972791cf37 P 786851d4227e4dd89b75ccdea0b1f03b [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.808285 10011 tablet_replica.cc:333] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.808864 10011 raft_consensus.cc:2243] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:16:46.810282 10011 raft_consensus.cc:2272] T 35c7899b1a314ae1b67824dfee6ff091 P 786851d4227e4dd89b75ccdea0b1f03b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.812474 10011 tablet_replica.cc:333] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b: stopping tablet replica
I20260709 12:16:46.812995 10011 raft_consensus.cc:2243] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.813585 10011 raft_consensus.cc:2272] T b9dab54d96124869a4a52f9cc08e288a P 786851d4227e4dd89b75ccdea0b1f03b [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.047226 11110 debug-util.cc:398] Leaking SignalData structure 0x7b080000b3c0 after lost signal to thread 10014
W20260709 12:16:47.048472 11110 debug-util.cc:398] Leaking SignalData structure 0x7b08003991e0 after lost signal to thread 11113
W20260709 12:16:47.049971 11110 debug-util.cc:398] Leaking SignalData structure 0x7b080022cd00 after lost signal to thread 11959
I20260709 12:16:47.239984 10011 tablet_server.cc:196] TabletServer@127.9.198.196:0 shutdown complete.
I20260709 12:16:47.325223 10011 master.cc:562] Master@127.9.198.254:36219 shutting down...
W20260709 12:16:47.375308 11127 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:16:47.375664 11127 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:16:47.375936 11127 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:16:48.158573 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6d225f8a873c4abe992018eb9353664f: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:16:56.166877 11065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111) [suppressed 153 similar messages]
W20260709 12:16:56.170961 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 86c5246d860d4d559ea665671cc46b5c: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:17:01.377501 11065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260709 12:17:01.380620 11065 catalog_manager.cc:4755] TS 016411114c6d42039bd04025378dfc98 (127.9.198.193:45115): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fab915731e0347ae8353a7f300c44401: Network error: Client connection negotiation failed: client connection to 127.9.198.193:45115: connect: Connection refused (error 111)
W20260709 12:17:07.834853 11065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.196:39119: connect: Connection refused (error 111)
W20260709 12:17:07.838661 11065 catalog_manager.cc:4755] TS 786851d4227e4dd89b75ccdea0b1f03b (127.9.198.196:39119): ChangeConfig:REMOVE_PEER RPC failed for tablet 4a634acd9ff543cca551c8a28e72d6c4: Network error: Client connection negotiation failed: client connection to 127.9.198.196:39119: connect: Connection refused (error 111)
I20260709 12:17:08.631850 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:08.632886 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:08.633296 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P bbfb3415e272477dac9ba599cc40a89d: stopping tablet replica
I20260709 12:17:08.845016 10011 master.cc:584] Master@127.9.198.254:36219 shutdown complete.
I20260709 12:17:08.897434 10011 test_util.cc:182] -----------------------------------------------
I20260709 12:17:08.897744 10011 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599049974374-10011-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (318686 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:17:08.901290 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:35489
I20260709 12:17:08.902566 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:08.908605 20824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:08.908699 20825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:08.910334 20827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:08.911759 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:08.915737 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:08.915958 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:08.916074 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599428916063 us; error 0 us; skew 500 ppm
I20260709 12:17:08.916553 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:08.919245 10011 webserver.cc:533] Webserver started at http://127.9.198.254:40693/ using document root <none> and password file <none>
I20260709 12:17:08.919724 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:08.919896 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:08.920125 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:08.921415 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "83b03a0385cf48c2bd37621ea07c4cac"
format_stamp: "Formatted at 2026-07-09 12:17:08 on dist-test-slave-glnm"
I20260709 12:17:08.926331 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:17:08.930222 20832 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:08.931066 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:17:08.931414 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "83b03a0385cf48c2bd37621ea07c4cac"
format_stamp: "Formatted at 2026-07-09 12:17:08 on dist-test-slave-glnm"
I20260709 12:17:08.931716 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:08.956741 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:08.958345 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:09.003522 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:35489
I20260709 12:17:09.003659 20883 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:35489 every 8 connection(s)
I20260709 12:17:09.007938 20884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.020648 20884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac: Bootstrap starting.
I20260709 12:17:09.025835 20884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.030596 20884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac: No bootstrap required, opened a new log
I20260709 12:17:09.033026 20884 raft_consensus.cc:359] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b03a0385cf48c2bd37621ea07c4cac" member_type: VOTER }
I20260709 12:17:09.033537 20884 raft_consensus.cc:385] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.033800 20884 raft_consensus.cc:740] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b03a0385cf48c2bd37621ea07c4cac, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.034489 20884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b03a0385cf48c2bd37621ea07c4cac" member_type: VOTER }
I20260709 12:17:09.034955 20884 raft_consensus.cc:399] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:09.035166 20884 raft_consensus.cc:493] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:09.035393 20884 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.040405 20884 raft_consensus.cc:515] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b03a0385cf48c2bd37621ea07c4cac" member_type: VOTER }
I20260709 12:17:09.040984 20884 leader_election.cc:304] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83b03a0385cf48c2bd37621ea07c4cac; no voters: 
I20260709 12:17:09.042253 20884 leader_election.cc:290] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:09.042634 20887 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.044158 20887 raft_consensus.cc:697] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 1 LEADER]: Becoming Leader. State: Replica: 83b03a0385cf48c2bd37621ea07c4cac, State: Running, Role: LEADER
I20260709 12:17:09.044847 20887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b03a0385cf48c2bd37621ea07c4cac" member_type: VOTER }
I20260709 12:17:09.045478 20884 sys_catalog.cc:565] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:09.052744 20889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83b03a0385cf48c2bd37621ea07c4cac. Latest consensus state: current_term: 1 leader_uuid: "83b03a0385cf48c2bd37621ea07c4cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b03a0385cf48c2bd37621ea07c4cac" member_type: VOTER } }
I20260709 12:17:09.053673 20889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:09.055410 20888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83b03a0385cf48c2bd37621ea07c4cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b03a0385cf48c2bd37621ea07c4cac" member_type: VOTER } }
I20260709 12:17:09.056247 20888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:09.060774 20895 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:09.069209 20895 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:09.070425 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:17:09.079134 20895 catalog_manager.cc:1383] Generated new cluster ID: 3b00e60b8fea46b3a5a37e17cd8b7ed9
I20260709 12:17:09.079492 20895 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:09.127487 20895 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:09.128829 20895 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:09.147903 20895 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac: Generated new TSK 0
I20260709 12:17:09.148593 20895 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:09.201853 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:09.208600 20905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.210336 20906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.212087 20908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:09.214098 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:09.216894 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:09.217118 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:09.217365 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599429217342 us; error 0 us; skew 500 ppm
I20260709 12:17:09.218102 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:09.220733 10011 webserver.cc:533] Webserver started at http://127.9.198.193:43527/ using document root <none> and password file <none>
I20260709 12:17:09.221309 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:09.221539 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:09.221889 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:09.223382 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "c8997852a67a4e678dfad30c76a76964"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.228720 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:17:09.232563 20913 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.233530 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:17:09.233970 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "c8997852a67a4e678dfad30c76a76964"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.234313 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:09.250960 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:09.252351 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:09.254125 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:09.256744 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:09.257000 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.257275 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:09.257491 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.303826 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:41243
I20260709 12:17:09.303923 20975 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:41243 every 8 connection(s)
I20260709 12:17:09.309389 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:09.318683 20980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.320793 20981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.324338 20983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:09.325126 20976 heartbeater.cc:344] Connected to a master server at 127.9.198.254:35489
I20260709 12:17:09.325619 20976 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:09.326259 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:09.326645 20976 heartbeater.cc:507] Master 127.9.198.254:35489 requested a full tablet report, sending...
I20260709 12:17:09.328976 20849 ts_manager.cc:194] Registered new tserver with Master: c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:09.329378 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:09.329686 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:09.329892 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599429329872 us; error 0 us; skew 500 ppm
I20260709 12:17:09.330610 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:09.331151 20849 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53100
I20260709 12:17:09.334038 10011 webserver.cc:533] Webserver started at http://127.9.198.194:43931/ using document root <none> and password file <none>
I20260709 12:17:09.334726 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:09.334946 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:09.335186 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:09.336721 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "7f62eca1a43e41f2b41f0d621002ac36"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.342593 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:17:09.346550 20988 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.347486 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:17:09.347775 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "7f62eca1a43e41f2b41f0d621002ac36"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.348049 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:09.366751 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:09.368041 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:09.369575 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:09.371919 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:09.372149 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.372429 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:09.372578 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.415866 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:43813
I20260709 12:17:09.415979 21050 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:43813 every 8 connection(s)
I20260709 12:17:09.421263 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:09.429666 21054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.432878 21055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:09.434219 10011 server_base.cc:1061] running on GCE node
W20260709 12:17:09.434280 21057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:09.435638 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:09.435896 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:09.436142 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599429436123 us; error 0 us; skew 500 ppm
I20260709 12:17:09.436491 21051 heartbeater.cc:344] Connected to a master server at 127.9.198.254:35489
I20260709 12:17:09.436942 21051 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:09.436998 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:09.438035 21051 heartbeater.cc:507] Master 127.9.198.254:35489 requested a full tablet report, sending...
I20260709 12:17:09.440059 10011 webserver.cc:533] Webserver started at http://127.9.198.195:41213/ using document root <none> and password file <none>
I20260709 12:17:09.440635 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:09.440595 20849 ts_manager.cc:194] Registered new tserver with Master: 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:09.441059 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:09.441373 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:09.442301 20849 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53102
I20260709 12:17:09.442611 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "4262e2b320ed4812be271437718fd86e"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.448186 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:17:09.451879 21062 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.452725 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:17:09.453100 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "4262e2b320ed4812be271437718fd86e"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.453397 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:09.475299 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:09.476632 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:09.478256 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:09.480682 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:09.480912 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.481191 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:09.481360 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.524501 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:42875
I20260709 12:17:09.524611 21124 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:42875 every 8 connection(s)
I20260709 12:17:09.529752 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:09.537297 21128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.538066 21129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.545926 21131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:09.546136 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:09.547500 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:09.547816 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:09.548035 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599429548018 us; error 0 us; skew 500 ppm
I20260709 12:17:09.548717 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:09.549355 21125 heartbeater.cc:344] Connected to a master server at 127.9.198.254:35489
I20260709 12:17:09.549788 21125 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:09.550745 21125 heartbeater.cc:507] Master 127.9.198.254:35489 requested a full tablet report, sending...
I20260709 12:17:09.551708 10011 webserver.cc:533] Webserver started at http://127.9.198.196:39921/ using document root <none> and password file <none>
I20260709 12:17:09.552498 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:09.552850 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:09.553224 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:09.553321 20849 ts_manager.cc:194] Registered new tserver with Master: 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:09.555286 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-3-root/instance:
uuid: "eb45c1a4fe5341368530840de4608989"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.555457 20849 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53114
I20260709 12:17:09.561476 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:17:09.565536 21136 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.566488 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:17:09.566888 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-3-root
uuid: "eb45c1a4fe5341368530840de4608989"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:09.567196 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:09.579939 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:09.581290 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:09.583025 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:09.585709 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:09.585992 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.586283 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:09.586484 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.632961 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.196:38535
I20260709 12:17:09.633219 21198 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.196:38535 every 8 connection(s)
I20260709 12:17:09.648938 21199 heartbeater.cc:344] Connected to a master server at 127.9.198.254:35489
I20260709 12:17:09.649397 21199 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:09.650302 21199 heartbeater.cc:507] Master 127.9.198.254:35489 requested a full tablet report, sending...
I20260709 12:17:09.652393 20849 ts_manager.cc:194] Registered new tserver with Master: eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:09.653359 10011 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016639172s
I20260709 12:17:09.654122 20849 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53122
I20260709 12:17:09.681264 20849 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53126:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:17:09.940896 20941 tablet_service.cc:1511] Processing CreateTablet for tablet 3778fc4a90d9419cbd028360957cbc4e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:17:09.942495 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3778fc4a90d9419cbd028360957cbc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.946955 20940 tablet_service.cc:1511] Processing CreateTablet for tablet 89fda386048b4838a8b10fbe2d602568 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:17:09.948676 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89fda386048b4838a8b10fbe2d602568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.948556 20937 tablet_service.cc:1511] Processing CreateTablet for tablet 6181c8aa4b7a4d2fa8163b486fec512b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:17:09.949707 20938 tablet_service.cc:1511] Processing CreateTablet for tablet 17ab161162c04ac8b68e594a1fbabe36 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:17:09.950397 20937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6181c8aa4b7a4d2fa8163b486fec512b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.951187 20938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17ab161162c04ac8b68e594a1fbabe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.956269 20936 tablet_service.cc:1511] Processing CreateTablet for tablet a8c82b37b2b14fcfb1d8c88765a4ba24 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:17:09.956734 21016 tablet_service.cc:1511] Processing CreateTablet for tablet 3778fc4a90d9419cbd028360957cbc4e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:17:09.948807 20939 tablet_service.cc:1511] Processing CreateTablet for tablet c7a254b1018240fd8b49a2776fb01a2b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:17:09.957902 20936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c82b37b2b14fcfb1d8c88765a4ba24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.958261 21016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3778fc4a90d9419cbd028360957cbc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.958582 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a254b1018240fd8b49a2776fb01a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.962713 21015 tablet_service.cc:1511] Processing CreateTablet for tablet 89fda386048b4838a8b10fbe2d602568 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:17:09.964373 21015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89fda386048b4838a8b10fbe2d602568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.966742 21014 tablet_service.cc:1511] Processing CreateTablet for tablet aaf04654bfa04b2d8d24ec0581483a23 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:17:09.968255 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf04654bfa04b2d8d24ec0581483a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.971067 21013 tablet_service.cc:1511] Processing CreateTablet for tablet 17ab161162c04ac8b68e594a1fbabe36 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:17:09.972990 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17ab161162c04ac8b68e594a1fbabe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.975337 21012 tablet_service.cc:1511] Processing CreateTablet for tablet 6181c8aa4b7a4d2fa8163b486fec512b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:17:09.977144 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6181c8aa4b7a4d2fa8163b486fec512b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.979707 21090 tablet_service.cc:1511] Processing CreateTablet for tablet 89fda386048b4838a8b10fbe2d602568 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:17:09.996074 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89fda386048b4838a8b10fbe2d602568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.996079 21161 tablet_service.cc:1511] Processing CreateTablet for tablet 6181c8aa4b7a4d2fa8163b486fec512b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:17:10.003440 21161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6181c8aa4b7a4d2fa8163b486fec512b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.985468 21089 tablet_service.cc:1511] Processing CreateTablet for tablet c7a254b1018240fd8b49a2776fb01a2b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:17:09.988276 21088 tablet_service.cc:1511] Processing CreateTablet for tablet aaf04654bfa04b2d8d24ec0581483a23 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:17:09.990260 21164 tablet_service.cc:1511] Processing CreateTablet for tablet 3778fc4a90d9419cbd028360957cbc4e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:17:09.991592 21086 tablet_service.cc:1511] Processing CreateTablet for tablet a8c82b37b2b14fcfb1d8c88765a4ba24 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:17:09.991839 21160 tablet_service.cc:1511] Processing CreateTablet for tablet a8c82b37b2b14fcfb1d8c88765a4ba24 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:17:09.992189 21163 tablet_service.cc:1511] Processing CreateTablet for tablet c7a254b1018240fd8b49a2776fb01a2b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:17:09.994048 21162 tablet_service.cc:1511] Processing CreateTablet for tablet aaf04654bfa04b2d8d24ec0581483a23 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:17:09.993922 21087 tablet_service.cc:1511] Processing CreateTablet for tablet 17ab161162c04ac8b68e594a1fbabe36 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:17:10.007951 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf04654bfa04b2d8d24ec0581483a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.008311 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3778fc4a90d9419cbd028360957cbc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.014626 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c82b37b2b14fcfb1d8c88765a4ba24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.018682 21162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf04654bfa04b2d8d24ec0581483a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.018952 21219 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:10.022272 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a254b1018240fd8b49a2776fb01a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.026571 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7a254b1018240fd8b49a2776fb01a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.048223 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17ab161162c04ac8b68e594a1fbabe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.054612 21086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c82b37b2b14fcfb1d8c88765a4ba24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.065797 21011 tablet_service.cc:1511] Processing CreateTablet for tablet 1b836c6e345b4404b5fbc1f5d42b5a15 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:17:10.067332 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b836c6e345b4404b5fbc1f5d42b5a15. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:10.070065 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:10.070840 21220 tablet_bootstrap.cc:492] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:10.072863 21219 tablet_bootstrap.cc:654] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.085958 21088 tablet_service.cc:1511] Processing CreateTablet for tablet 1b836c6e345b4404b5fbc1f5d42b5a15 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:17:10.085774 21159 tablet_service.cc:1511] Processing CreateTablet for tablet 1b836c6e345b4404b5fbc1f5d42b5a15 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:17:10.087330 21159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b836c6e345b4404b5fbc1f5d42b5a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.094267 20940 tablet_service.cc:1511] Processing CreateTablet for tablet c0c1d07d1b1344a5ad97b6f5a36d343a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:17:10.095944 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c1d07d1b1344a5ad97b6f5a36d343a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.087328 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b836c6e345b4404b5fbc1f5d42b5a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.105623 21222 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:10.106201 21012 tablet_service.cc:1511] Processing CreateTablet for tablet c0c1d07d1b1344a5ad97b6f5a36d343a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:17:10.107661 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c1d07d1b1344a5ad97b6f5a36d343a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.118247 21221 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:10.124660 21220 tablet_bootstrap.cc:654] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.142413 21219 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:10.143106 21219 ts_tablet_manager.cc:1403] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.125s	user 0.010s	sys 0.009s
I20260709 12:17:10.144680 21222 tablet_bootstrap.cc:654] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.151113 21090 tablet_service.cc:1511] Processing CreateTablet for tablet c0c1d07d1b1344a5ad97b6f5a36d343a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:17:10.152559 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c1d07d1b1344a5ad97b6f5a36d343a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.152916 21159 tablet_service.cc:1511] Processing CreateTablet for tablet 4a416937e48a4949a7b5fba8c466eb41 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:17:10.154549 21159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a416937e48a4949a7b5fba8c466eb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.155102 21219 raft_consensus.cc:359] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.160308 21221 tablet_bootstrap.cc:654] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.157840 21219 raft_consensus.cc:385] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.162981 21219 raft_consensus.cc:740] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.166775 21219 consensus_queue.cc:260] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.186558 20941 tablet_service.cc:1511] Processing CreateTablet for tablet 4a416937e48a4949a7b5fba8c466eb41 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:17:10.188755 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a416937e48a4949a7b5fba8c466eb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.198660 21220 tablet_bootstrap.cc:492] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:10.199234 21220 ts_tablet_manager.cc:1403] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.004s
I20260709 12:17:10.199674 21016 tablet_service.cc:1511] Processing CreateTablet for tablet 4a416937e48a4949a7b5fba8c466eb41 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:17:10.201063 21016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a416937e48a4949a7b5fba8c466eb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.202036 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 9b6872f6d7f14caea68ac416bcb17b27 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:17:10.203354 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b6872f6d7f14caea68ac416bcb17b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.204270 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 9b6872f6d7f14caea68ac416bcb17b27 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:17:10.205551 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b6872f6d7f14caea68ac416bcb17b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.206422 21220 raft_consensus.cc:359] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.207269 21220 raft_consensus.cc:385] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.207609 21220 raft_consensus.cc:740] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.211762 21219 ts_tablet_manager.cc:1434] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.020s
I20260709 12:17:10.211506 21086 tablet_service.cc:1511] Processing CreateTablet for tablet 9b6872f6d7f14caea68ac416bcb17b27 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:17:10.222587 21086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b6872f6d7f14caea68ac416bcb17b27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.223023 20976 heartbeater.cc:499] Master 127.9.198.254:35489 was elected leader, sending a full tablet report...
I20260709 12:17:10.223338 20940 tablet_service.cc:1511] Processing CreateTablet for tablet 044d3214b94a448394941bed6d2204a8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:17:10.213935 21011 tablet_service.cc:1511] Processing CreateTablet for tablet f8d45e86f72047658bf7a60b30614c22 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:17:10.224700 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044d3214b94a448394941bed6d2204a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.225268 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8d45e86f72047658bf7a60b30614c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.228195 21219 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:10.215708 21222 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:10.230840 21222 ts_tablet_manager.cc:1403] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.029s
I20260709 12:17:10.215801 21087 tablet_service.cc:1511] Processing CreateTablet for tablet f8d45e86f72047658bf7a60b30614c22 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:17:10.216229 21160 tablet_service.cc:1511] Processing CreateTablet for tablet f8d45e86f72047658bf7a60b30614c22 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:17:10.237150 21012 tablet_service.cc:1511] Processing CreateTablet for tablet 044d3214b94a448394941bed6d2204a8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:17:10.238498 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044d3214b94a448394941bed6d2204a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.239133 21088 tablet_service.cc:1511] Processing CreateTablet for tablet 044d3214b94a448394941bed6d2204a8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:17:10.240506 21162 tablet_service.cc:1511] Processing CreateTablet for tablet 5041cbf53c7e4c588bb5a0b699284c33 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:17:10.235674 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8d45e86f72047658bf7a60b30614c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.242183 21162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5041cbf53c7e4c588bb5a0b699284c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.242277 21222 raft_consensus.cc:359] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.243098 21222 raft_consensus.cc:385] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.243386 21222 raft_consensus.cc:740] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.235672 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8d45e86f72047658bf7a60b30614c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.244225 21222 consensus_queue.cc:260] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.245868 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044d3214b94a448394941bed6d2204a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.253890 20941 tablet_service.cc:1511] Processing CreateTablet for tablet 5041cbf53c7e4c588bb5a0b699284c33 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:17:10.255234 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5041cbf53c7e4c588bb5a0b699284c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.260169 21219 tablet_bootstrap.cc:654] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.270589 21013 tablet_service.cc:1511] Processing CreateTablet for tablet 5041cbf53c7e4c588bb5a0b699284c33 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:17:10.272054 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5041cbf53c7e4c588bb5a0b699284c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.274673 21220 consensus_queue.cc:260] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.276000 21221 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:10.277369 21221 ts_tablet_manager.cc:1403] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.160s	user 0.023s	sys 0.000s
I20260709 12:17:10.280828 21221 raft_consensus.cc:359] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.293674 21221 raft_consensus.cc:385] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.294059 21221 raft_consensus.cc:740] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.294893 21221 consensus_queue.cc:260] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.297964 21199 heartbeater.cc:499] Master 127.9.198.254:35489 was elected leader, sending a full tablet report...
I20260709 12:17:10.298578 21222 ts_tablet_manager.cc:1434] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.028s
I20260709 12:17:10.299695 21222 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:10.299805 21228 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.301802 21088 tablet_service.cc:1511] Processing CreateTablet for tablet ae962db47c4e4fc6a10172d6395e113e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:17:10.307442 21163 tablet_service.cc:1511] Processing CreateTablet for tablet ae962db47c4e4fc6a10172d6395e113e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:17:10.308285 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae962db47c4e4fc6a10172d6395e113e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.309288 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae962db47c4e4fc6a10172d6395e113e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:10.309293 20977 tablet.cc:2406] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:17:10.315966 21015 tablet_service.cc:1511] Processing CreateTablet for tablet d1c050ac450e451d9a27af7cf32b7edc (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:17:10.317315 21015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c050ac450e451d9a27af7cf32b7edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.318971 21090 tablet_service.cc:1511] Processing CreateTablet for tablet d1c050ac450e451d9a27af7cf32b7edc (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:17:10.320336 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c050ac450e451d9a27af7cf32b7edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.304652 21228 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.329376 20941 tablet_service.cc:1511] Processing CreateTablet for tablet ae962db47c4e4fc6a10172d6395e113e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:17:10.329771 21159 tablet_service.cc:1511] Processing CreateTablet for tablet d1c050ac450e451d9a27af7cf32b7edc (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:17:10.330777 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae962db47c4e4fc6a10172d6395e113e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.331118 21159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c050ac450e451d9a27af7cf32b7edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.352456 21228 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:10.383453 21014 tablet_service.cc:1511] Processing CreateTablet for tablet 807ff01cdab14b62885871bdce0d3cd8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:17:10.384068 21219 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:10.384635 21219 ts_tablet_manager.cc:1403] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.029s
I20260709 12:17:10.384914 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 807ff01cdab14b62885871bdce0d3cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.388103 21219 raft_consensus.cc:359] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.388945 21219 raft_consensus.cc:385] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.389849 21219 raft_consensus.cc:740] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.390695 21219 consensus_queue.cc:260] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.396139 21220 ts_tablet_manager.cc:1434] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.197s	user 0.013s	sys 0.011s
I20260709 12:17:10.397157 21125 heartbeater.cc:499] Master 127.9.198.254:35489 was elected leader, sending a full tablet report...
I20260709 12:17:10.397184 20940 tablet_service.cc:1511] Processing CreateTablet for tablet 807ff01cdab14b62885871bdce0d3cd8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:17:10.402138 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 807ff01cdab14b62885871bdce0d3cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.408200 21222 tablet_bootstrap.cc:654] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.410935 21220 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:10.411517 21160 tablet_service.cc:1511] Processing CreateTablet for tablet 775061823aca4c7c887f4f14e7ceb63b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:17:10.413431 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775061823aca4c7c887f4f14e7ceb63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.417830 21231 raft_consensus.cc:493] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.417843 21230 raft_consensus.cc:493] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.422152 21051 heartbeater.cc:499] Master 127.9.198.254:35489 was elected leader, sending a full tablet report...
I20260709 12:17:10.432214 21221 ts_tablet_manager.cc:1434] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.154s	user 0.000s	sys 0.024s
I20260709 12:17:10.430717 21231 raft_consensus.cc:515] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.417196 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 775061823aca4c7c887f4f14e7ceb63b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:17:10.436838 21226 raft_consensus.cc:493] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.437147 21231 leader_election.cc:290] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:10.437465 21226 raft_consensus.cc:515] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.436964 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775061823aca4c7c887f4f14e7ceb63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.433604 21221 tablet_bootstrap.cc:492] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:10.450598 21226 leader_election.cc:290] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:10.463449 21011 tablet_service.cc:1511] Processing CreateTablet for tablet 775061823aca4c7c887f4f14e7ceb63b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:17:10.450527 21230 raft_consensus.cc:515] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.467645 21086 tablet_service.cc:1511] Processing CreateTablet for tablet 807ff01cdab14b62885871bdce0d3cd8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:17:10.469203 21086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 807ff01cdab14b62885871bdce0d3cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.479252 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775061823aca4c7c887f4f14e7ceb63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.495260 21230 leader_election.cc:290] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:10.497231 21087 tablet_service.cc:1511] Processing CreateTablet for tablet 981cbd4c8dab4902a1cd993cb45cd9c3 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:17:10.498816 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981cbd4c8dab4902a1cd993cb45cd9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.520440 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
W20260709 12:17:10.522516 21137 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.539075 21219 ts_tablet_manager.cc:1434] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.154s	user 0.010s	sys 0.001s
I20260709 12:17:10.540432 21219 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:10.552388 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fda386048b4838a8b10fbe2d602568" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
W20260709 12:17:10.554903 20917 leader_election.cc:343] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.563503 20938 tablet_service.cc:1511] Processing CreateTablet for tablet 981cbd4c8dab4902a1cd993cb45cd9c3 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:17:10.565030 20938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981cbd4c8dab4902a1cd993cb45cd9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.569983 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6181c8aa4b7a4d2fa8163b486fec512b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
W20260709 12:17:10.572301 20989 leader_election.cc:343] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.578823 21015 tablet_service.cc:1511] Processing CreateTablet for tablet 72e531cd0b844ef99e5360fe8adabf3e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:17:10.579705 21220 tablet_bootstrap.cc:654] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.580458 21015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e531cd0b844ef99e5360fe8adabf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.598968 21159 tablet_service.cc:1511] Processing CreateTablet for tablet 981cbd4c8dab4902a1cd993cb45cd9c3 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:17:10.600538 21159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981cbd4c8dab4902a1cd993cb45cd9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.600749 21012 tablet_service.cc:1511] Processing CreateTablet for tablet d701ef5192bf4f11a84c7b7fa0660c67 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:17:10.597179 20937 tablet_service.cc:1511] Processing CreateTablet for tablet d701ef5192bf4f11a84c7b7fa0660c67 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:17:10.603235 20937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d701ef5192bf4f11a84c7b7fa0660c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.611002 21219 tablet_bootstrap.cc:654] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.611760 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d701ef5192bf4f11a84c7b7fa0660c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.614739 21088 tablet_service.cc:1511] Processing CreateTablet for tablet 72e531cd0b844ef99e5360fe8adabf3e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:17:10.616461 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e531cd0b844ef99e5360fe8adabf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.616990 21221 tablet_bootstrap.cc:654] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.620639 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6181c8aa4b7a4d2fa8163b486fec512b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:10.621383 20951 raft_consensus.cc:2468] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:10.620692 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:10.623917 20989 leader_election.cc:304] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:10.625238 21231 raft_consensus.cc:2804] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.625609 21231 raft_consensus.cc:493] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.625959 21231 raft_consensus.cc:3060] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.627954 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 72e531cd0b844ef99e5360fe8adabf3e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:17:10.629498 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e531cd0b844ef99e5360fe8adabf3e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:10.634845 21140 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.635393 21140 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:10.635684 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fda386048b4838a8b10fbe2d602568" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:10.636850 21228 raft_consensus.cc:2749] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:17:10.638151 20916 leader_election.cc:343] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.638685 20916 leader_election.cc:304] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:10.639899 21226 raft_consensus.cc:2749] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:10.640002 21162 tablet_service.cc:1511] Processing CreateTablet for tablet f00f76eb99ab413589cdbe3aafd60f9e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:17:10.641484 21162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00f76eb99ab413589cdbe3aafd60f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.643390 21013 tablet_service.cc:1511] Processing CreateTablet for tablet f00f76eb99ab413589cdbe3aafd60f9e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:17:10.644968 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00f76eb99ab413589cdbe3aafd60f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.643604 21090 tablet_service.cc:1511] Processing CreateTablet for tablet d701ef5192bf4f11a84c7b7fa0660c67 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:17:10.653842 21089 tablet_service.cc:1511] Processing CreateTablet for tablet 21144a10cc4945629d70fdd253d940d1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:17:10.656122 20941 tablet_service.cc:1511] Processing CreateTablet for tablet f00f76eb99ab413589cdbe3aafd60f9e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:17:10.647974 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d701ef5192bf4f11a84c7b7fa0660c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.662413 21221 tablet_bootstrap.cc:492] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:10.662985 21221 ts_tablet_manager.cc:1403] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.230s	user 0.007s	sys 0.009s
I20260709 12:17:10.663491 21164 tablet_service.cc:1511] Processing CreateTablet for tablet 21144a10cc4945629d70fdd253d940d1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:17:10.665004 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21144a10cc4945629d70fdd253d940d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.666188 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21144a10cc4945629d70fdd253d940d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.657480 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f00f76eb99ab413589cdbe3aafd60f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.670203 21221 raft_consensus.cc:359] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.676203 21222 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:10.686775 21222 ts_tablet_manager.cc:1403] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.387s	user 0.025s	sys 0.007s
I20260709 12:17:10.687166 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf04654bfa04b2d8d24ec0581483a23" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
W20260709 12:17:10.689474 21063 leader_election.cc:343] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.690045 21222 raft_consensus.cc:359] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.691012 21222 raft_consensus.cc:385] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.691387 21222 raft_consensus.cc:740] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.652185 21231 raft_consensus.cc:515] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.695034 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6181c8aa4b7a4d2fa8163b486fec512b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:10.696708 21231 leader_election.cc:290] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:10.697543 21221 raft_consensus.cc:385] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.700079 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6181c8aa4b7a4d2fa8163b486fec512b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:10.702454 20951 raft_consensus.cc:3060] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.702019 21222 consensus_queue.cc:260] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.703012 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 21144a10cc4945629d70fdd253d940d1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:17:10.701684 21221 raft_consensus.cc:740] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.704552 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21144a10cc4945629d70fdd253d940d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.704680 21222 ts_tablet_manager.cc:1434] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:17:10.705811 21174 raft_consensus.cc:3060] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.707926 21221 consensus_queue.cc:260] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.710367 21221 ts_tablet_manager.cc:1434] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 12:17:10.716035 21015 tablet_service.cc:1511] Processing CreateTablet for tablet d1474ed8c5944a89ade3d329705003ca (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:17:10.717442 21015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1474ed8c5944a89ade3d329705003ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.719746 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf04654bfa04b2d8d24ec0581483a23" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:10.720479 21026 raft_consensus.cc:2468] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:10.721982 21066 leader_election.cc:304] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:10.723299 21230 raft_consensus.cc:2804] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.723675 21230 raft_consensus.cc:493] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.724001 21230 raft_consensus.cc:3060] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.725112 20940 tablet_service.cc:1511] Processing CreateTablet for tablet e5569aee047642449df010d452ed73fa (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:17:10.726701 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5569aee047642449df010d452ed73fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.734444 21011 tablet_service.cc:1511] Processing CreateTablet for tablet e5569aee047642449df010d452ed73fa (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:17:10.735004 21221 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:10.735870 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5569aee047642449df010d452ed73fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.746284 21174 raft_consensus.cc:2468] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:10.732861 21230 raft_consensus.cc:515] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.752699 20989 leader_election.cc:304] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:10.753131 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf04654bfa04b2d8d24ec0581483a23" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:10.753718 21231 raft_consensus.cc:2804] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.754352 21231 raft_consensus.cc:697] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:10.755244 21231 consensus_queue.cc:237] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:10.753785 21026 raft_consensus.cc:3060] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.758242 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf04654bfa04b2d8d24ec0581483a23" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
W20260709 12:17:10.759687 21063 leader_election.cc:343] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.759608 21162 tablet_service.cc:1511] Processing CreateTablet for tablet c32186e2dec443e685ff412c8bda31a6 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:17:10.763208 21162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c32186e2dec443e685ff412c8bda31a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.763929 21086 tablet_service.cc:1511] Processing CreateTablet for tablet e5569aee047642449df010d452ed73fa (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:17:10.769361 20936 tablet_service.cc:1511] Processing CreateTablet for tablet c32186e2dec443e685ff412c8bda31a6 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:17:10.770262 20951 raft_consensus.cc:2468] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:10.770910 20936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c32186e2dec443e685ff412c8bda31a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.765391 21086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5569aee047642449df010d452ed73fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.765273 21230 leader_election.cc:290] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:10.786650 21087 tablet_service.cc:1511] Processing CreateTablet for tablet d1474ed8c5944a89ade3d329705003ca (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:17:10.788942 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1474ed8c5944a89ade3d329705003ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.795590 21014 tablet_service.cc:1511] Processing CreateTablet for tablet c32186e2dec443e685ff412c8bda31a6 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:17:10.776899 21222 tablet_bootstrap.cc:492] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:10.797580 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c32186e2dec443e685ff412c8bda31a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.804030 21220 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:10.804684 21220 ts_tablet_manager.cc:1403] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.394s	user 0.009s	sys 0.016s
I20260709 12:17:10.806530 21221 tablet_bootstrap.cc:654] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.806854 21160 tablet_service.cc:1511] Processing CreateTablet for tablet d1474ed8c5944a89ade3d329705003ca (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:17:10.808508 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1474ed8c5944a89ade3d329705003ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.808526 21220 raft_consensus.cc:359] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:10.814071 21220 raft_consensus.cc:385] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.814430 21220 raft_consensus.cc:740] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.815428 21220 consensus_queue.cc:260] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:10.818775 21090 tablet_service.cc:1511] Processing CreateTablet for tablet 9d53959ed0724fc1905beec27f8a0917 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:17:10.820318 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d53959ed0724fc1905beec27f8a0917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.824389 21220 ts_tablet_manager.cc:1434] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 12:17:10.827932 21026 raft_consensus.cc:2468] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:10.829278 21066 leader_election.cc:304] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:10.830572 21230 raft_consensus.cc:2804] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.831341 21230 raft_consensus.cc:697] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:10.832224 21230 consensus_queue.cc:237] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.839701 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 9d53959ed0724fc1905beec27f8a0917 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:17:10.804031 21219 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:10.841185 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d53959ed0724fc1905beec27f8a0917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.841352 21219 ts_tablet_manager.cc:1403] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.301s	user 0.015s	sys 0.005s
I20260709 12:17:10.851997 21222 tablet_bootstrap.cc:654] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.825623 21220 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:10.840974 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 9d53959ed0724fc1905beec27f8a0917 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:17:10.869149 21089 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9137ae28b54fb58af02339dca65435 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:17:10.865002 21219 raft_consensus.cc:359] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:10.870278 21219 raft_consensus.cc:385] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.870637 21219 raft_consensus.cc:740] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.870801 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9137ae28b54fb58af02339dca65435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.874409 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d53959ed0724fc1905beec27f8a0917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.877133 21219 consensus_queue.cc:260] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:10.855528 20847 catalog_manager.cc:5697] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:10.883633 21220 tablet_bootstrap.cc:654] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.859193 21012 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9137ae28b54fb58af02339dca65435 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:17:10.891803 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9137ae28b54fb58af02339dca65435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.887328 21219 ts_tablet_manager.cc:1434] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:17:10.890368 21222 tablet_bootstrap.cc:492] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:10.906728 21222 ts_tablet_manager.cc:1403] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.201s	user 0.013s	sys 0.001s
I20260709 12:17:10.918445 21219 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:10.918275 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9137ae28b54fb58af02339dca65435 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:17:10.919752 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9137ae28b54fb58af02339dca65435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.921887 20941 tablet_service.cc:1511] Processing CreateTablet for tablet eb0e4b78baa443b7a59d8155afb1dc1a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:17:10.923456 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0e4b78baa443b7a59d8155afb1dc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.945139 21015 tablet_service.cc:1511] Processing CreateTablet for tablet eb0e4b78baa443b7a59d8155afb1dc1a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:17:10.947126 21015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0e4b78baa443b7a59d8155afb1dc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.949352 21086 tablet_service.cc:1511] Processing CreateTablet for tablet eb0e4b78baa443b7a59d8155afb1dc1a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:17:10.950804 21086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb0e4b78baa443b7a59d8155afb1dc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.952936 21011 tablet_service.cc:1511] Processing CreateTablet for tablet b3ddbde7cec2484da93f2b7075232308 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:17:10.954322 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ddbde7cec2484da93f2b7075232308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.956250 21164 tablet_service.cc:1511] Processing CreateTablet for tablet b3ddbde7cec2484da93f2b7075232308 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:17:10.957861 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ddbde7cec2484da93f2b7075232308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.961325 20937 tablet_service.cc:1511] Processing CreateTablet for tablet b3ddbde7cec2484da93f2b7075232308 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:17:10.964905 20937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ddbde7cec2484da93f2b7075232308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.965126 21222 raft_consensus.cc:359] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.965976 21222 raft_consensus.cc:385] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.966281 21222 raft_consensus.cc:740] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.979007 21222 consensus_queue.cc:260] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:10.992641 21222 ts_tablet_manager.cc:1434] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.085s	user 0.012s	sys 0.002s
I20260709 12:17:10.950834 21221 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:11.015353 21221 ts_tablet_manager.cc:1403] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.281s	user 0.016s	sys 0.006s
I20260709 12:17:10.981506 21220 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:11.016095 21220 ts_tablet_manager.cc:1403] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.191s	user 0.012s	sys 0.007s
I20260709 12:17:10.998481 21219 tablet_bootstrap.cc:654] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.013888 21162 tablet_service.cc:1511] Processing CreateTablet for tablet e65504fa010146ef8fef1ff07dfcff69 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:17:11.019076 21162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e65504fa010146ef8fef1ff07dfcff69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.010175 21089 tablet_service.cc:1511] Processing CreateTablet for tablet e65504fa010146ef8fef1ff07dfcff69 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:17:11.031354 21013 tablet_service.cc:1511] Processing CreateTablet for tablet 93ad3b0b41544f61bdc6f80d66c26adf (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:17:11.032863 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ad3b0b41544f61bdc6f80d66c26adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.035308 21221 raft_consensus.cc:359] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.036378 21221 raft_consensus.cc:385] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.036482 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e65504fa010146ef8fef1ff07dfcff69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.036778 21221 raft_consensus.cc:740] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.037824 21221 consensus_queue.cc:260] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.017343 20940 tablet_service.cc:1511] Processing CreateTablet for tablet e65504fa010146ef8fef1ff07dfcff69 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:17:11.045825 21088 tablet_service.cc:1511] Processing CreateTablet for tablet 93ad3b0b41544f61bdc6f80d66c26adf (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:17:11.049418 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ad3b0b41544f61bdc6f80d66c26adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.050052 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e65504fa010146ef8fef1ff07dfcff69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.054788 20846 catalog_manager.cc:5697] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.054710 21222 tablet_bootstrap.cc:492] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
W20260709 12:17:11.073369 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:11.083674 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 93ad3b0b41544f61bdc6f80d66c26adf (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:17:11.085187 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ad3b0b41544f61bdc6f80d66c26adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.086638 20937 tablet_service.cc:1511] Processing CreateTablet for tablet d5353702fdb44369a77c62ee46fa7fcd (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:17:11.086376 21014 tablet_service.cc:1511] Processing CreateTablet for tablet d5353702fdb44369a77c62ee46fa7fcd (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:17:11.088022 20937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5353702fdb44369a77c62ee46fa7fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.088567 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5353702fdb44369a77c62ee46fa7fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.098042 21090 tablet_service.cc:1511] Processing CreateTablet for tablet d5353702fdb44369a77c62ee46fa7fcd (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:17:11.102963 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5353702fdb44369a77c62ee46fa7fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.103303 21230 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.103924 21230 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.104197 21160 tablet_service.cc:1511] Processing CreateTablet for tablet d4fc3437dadf4368a453b79bb7a38482 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:17:11.105710 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4fc3437dadf4368a453b79bb7a38482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.106678 21230 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:11.110196 21221 ts_tablet_manager.cc:1434] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.094s	user 0.009s	sys 0.000s
I20260709 12:17:11.113412 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
W20260709 12:17:11.117985 21063 leader_election.cc:343] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.127782 20941 tablet_service.cc:1511] Processing CreateTablet for tablet d4fc3437dadf4368a453b79bb7a38482 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:17:11.129062 21222 tablet_bootstrap.cc:654] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.129276 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4fc3437dadf4368a453b79bb7a38482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.137759 21220 raft_consensus.cc:359] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:11.138685 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
W20260709 12:17:11.140486 21063 leader_election.cc:343] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.141129 21219 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:11.141052 21063 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:11.141865 21219 ts_tablet_manager.cc:1403] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.224s	user 0.011s	sys 0.022s
I20260709 12:17:11.142516 21230 raft_consensus.cc:2749] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.144965 21219 raft_consensus.cc:359] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.145821 21219 raft_consensus.cc:385] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.149302 21219 raft_consensus.cc:740] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.154274 21220 raft_consensus.cc:385] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.155555 21221 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:11.156615 21220 raft_consensus.cc:740] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.163528 21089 tablet_service.cc:1511] Processing CreateTablet for tablet 91f9400a74934eb4b3269b428057ccef (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:17:11.164561 21219 consensus_queue.cc:260] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.165336 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f9400a74934eb4b3269b428057ccef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.165926 21220 consensus_queue.cc:260] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:11.173667 21160 tablet_service.cc:1511] Processing CreateTablet for tablet 91f9400a74934eb4b3269b428057ccef (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:17:11.175112 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f9400a74934eb4b3269b428057ccef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.176198 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 91f9400a74934eb4b3269b428057ccef (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:17:11.178846 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f9400a74934eb4b3269b428057ccef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.184413 21219 ts_tablet_manager.cc:1434] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 12:17:11.185709 21219 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:11.190253 21013 tablet_service.cc:1511] Processing CreateTablet for tablet d4fc3437dadf4368a453b79bb7a38482 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:17:11.191948 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4fc3437dadf4368a453b79bb7a38482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.199227 21220 ts_tablet_manager.cc:1434] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.178s	user 0.004s	sys 0.004s
I20260709 12:17:11.206197 21231 raft_consensus.cc:493] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.206924 21231 raft_consensus.cc:515] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.210712 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ab161162c04ac8b68e594a1fbabe36" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:11.211480 20951 raft_consensus.cc:2468] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:11.212800 20989 leader_election.cc:304] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:11.214015 21232 raft_consensus.cc:2804] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.214092 21231 leader_election.cc:290] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:11.214529 21232 raft_consensus.cc:493] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.214968 21232 raft_consensus.cc:3060] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.206463 21222 tablet_bootstrap.cc:492] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:11.223213 21220 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:11.226835 21219 tablet_bootstrap.cc:654] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.230830 21222 ts_tablet_manager.cc:1403] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.189s	user 0.017s	sys 0.000s
W20260709 12:17:11.239403 21063 consensus_peers.cc:597] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e -> Peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Couldn't send request to peer eb45c1a4fe5341368530840de4608989. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:11.239526 21222 raft_consensus.cc:359] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:11.240500 21222 raft_consensus.cc:385] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.240857 21222 raft_consensus.cc:740] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.248716 21089 tablet_service.cc:1511] Processing CreateTablet for tablet fe6f9c0020a4484ca89737318ee6cc3c (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:17:11.250320 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe6f9c0020a4484ca89737318ee6cc3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.250523 21014 tablet_service.cc:1511] Processing CreateTablet for tablet fe6f9c0020a4484ca89737318ee6cc3c (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:17:11.250641 21160 tablet_service.cc:1511] Processing CreateTablet for tablet fe6f9c0020a4484ca89737318ee6cc3c (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:17:11.252046 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe6f9c0020a4484ca89737318ee6cc3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.251633 21222 consensus_queue.cc:260] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:11.252897 20941 tablet_service.cc:1511] Processing CreateTablet for tablet 033d781150df48e1ba4cefa59d033f42 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:17:11.254223 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033d781150df48e1ba4cefa59d033f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.254511 21220 tablet_bootstrap.cc:654] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.256278 21232 raft_consensus.cc:515] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.258277 21226 raft_consensus.cc:493] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.260131 21226 raft_consensus.cc:515] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.262315 21221 tablet_bootstrap.cc:654] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.264690 21226 leader_election.cc:290] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:11.275194 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe6f9c0020a4484ca89737318ee6cc3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.275570 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:11.276899 21012 tablet_service.cc:1511] Processing CreateTablet for tablet 033d781150df48e1ba4cefa59d033f42 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 12:17:11.277408 20916 leader_election.cc:343] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.278436 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033d781150df48e1ba4cefa59d033f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.279281 21232 leader_election.cc:290] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:11.279981 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ab161162c04ac8b68e594a1fbabe36" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:11.280646 20951 raft_consensus.cc:3060] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.287799 21219 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:11.288399 21219 ts_tablet_manager.cc:1403] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.103s	user 0.021s	sys 0.000s
I20260709 12:17:11.289352 21222 ts_tablet_manager.cc:1434] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 12:17:11.291822 21219 raft_consensus.cc:359] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.292618 21219 raft_consensus.cc:385] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.292909 21219 raft_consensus.cc:740] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.293748 21219 consensus_queue.cc:260] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.295987 21222 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:11.296119 21219 ts_tablet_manager.cc:1434] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:11.304961 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 98d7127b20aa4482ae857ddb5430b70e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:17:11.306638 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d7127b20aa4482ae857ddb5430b70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.314898 21219 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:11.316589 21088 tablet_service.cc:1511] Processing CreateTablet for tablet 033d781150df48e1ba4cefa59d033f42 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:17:11.318049 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033d781150df48e1ba4cefa59d033f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.318928 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ab161162c04ac8b68e594a1fbabe36" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:11.320823 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ab161162c04ac8b68e594a1fbabe36" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
W20260709 12:17:11.322386 20991 leader_election.cc:343] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.323380 21219 tablet_bootstrap.cc:654] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.302418 21220 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
W20260709 12:17:11.335866 20991 leader_election.cc:343] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.338976 21232 consensus_queue.cc:1048] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:11.341944 21014 tablet_service.cc:1511] Processing CreateTablet for tablet 98d7127b20aa4482ae857ddb5430b70e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:17:11.343525 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d7127b20aa4482ae857ddb5430b70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.350999 21090 tablet_service.cc:1511] Processing CreateTablet for tablet 181a9cec50b94395aac6bd156652a9f2 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:17:11.352543 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181a9cec50b94395aac6bd156652a9f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.355650 21228 raft_consensus.cc:493] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.356408 20951 raft_consensus.cc:2468] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:11.356225 21228 raft_consensus.cc:515] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:11.359200 21228 leader_election.cc:290] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:11.359567 21222 tablet_bootstrap.cc:654] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.361184 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf04654bfa04b2d8d24ec0581483a23" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:11.362160 21026 raft_consensus.cc:2393] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:11.364661 20989 leader_election.cc:304] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:11.346052 21220 ts_tablet_manager.cc:1403] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.117s	user 0.019s	sys 0.000s
I20260709 12:17:11.366752 21231 raft_consensus.cc:2804] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.367290 21231 raft_consensus.cc:697] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:11.367183 21160 tablet_service.cc:1511] Processing CreateTablet for tablet 98d7127b20aa4482ae857ddb5430b70e (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:17:11.368707 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d7127b20aa4482ae857ddb5430b70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.368342 21231 consensus_queue.cc:237] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.371450 21221 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:11.372040 21221 ts_tablet_manager.cc:1403] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.218s	user 0.016s	sys 0.001s
I20260709 12:17:11.374882 21232 consensus_queue.cc:1048] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.392774 21013 tablet_service.cc:1511] Processing CreateTablet for tablet 741c5b82a89142c2840a00c76ede6243 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:17:11.394486 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741c5b82a89142c2840a00c76ede6243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.398433 21221 raft_consensus.cc:359] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:11.399335 21221 raft_consensus.cc:385] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.399684 21221 raft_consensus.cc:740] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.400651 21221 consensus_queue.cc:260] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:11.403926 21220 raft_consensus.cc:359] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.404989 21220 raft_consensus.cc:385] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.406322 21220 raft_consensus.cc:740] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.407804 21220 consensus_queue.cc:260] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.408859 20941 tablet_service.cc:1511] Processing CreateTablet for tablet 181a9cec50b94395aac6bd156652a9f2 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:17:11.421365 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181a9cec50b94395aac6bd156652a9f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.423827 21221 ts_tablet_manager.cc:1434] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 12:17:11.410763 21230 consensus_queue.cc:1048] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.427268 21221 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:11.440796 21220 ts_tablet_manager.cc:1434] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.001s
I20260709 12:17:11.442169 21222 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:11.443797 21222 ts_tablet_manager.cc:1403] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.148s	user 0.014s	sys 0.005s
I20260709 12:17:11.446561 21220 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:11.447764 21222 raft_consensus.cc:359] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.450001 21222 raft_consensus.cc:385] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.450904 21222 raft_consensus.cc:740] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.453922 21219 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:11.454543 21219 ts_tablet_manager.cc:1403] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.140s	user 0.020s	sys 0.027s
I20260709 12:17:11.463723 21228 raft_consensus.cc:493] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.465379 21228 raft_consensus.cc:515] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:11.451678 21222 consensus_queue.cc:260] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.474949 21089 tablet_service.cc:1511] Processing CreateTablet for tablet 741c5b82a89142c2840a00c76ede6243 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:17:11.475469 20846 catalog_manager.cc:5697] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.476604 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741c5b82a89142c2840a00c76ede6243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.478001 21228 leader_election.cc:290] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:11.479207 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b836c6e345b4404b5fbc1f5d42b5a15" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:11.481251 21160 tablet_service.cc:1511] Processing CreateTablet for tablet 181a9cec50b94395aac6bd156652a9f2 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:17:11.482661 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181a9cec50b94395aac6bd156652a9f2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:11.482793 21140 leader_election.cc:343] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.488083 21230 raft_consensus.cc:493] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.488689 21230 raft_consensus.cc:515] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:11.494897 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fda386048b4838a8b10fbe2d602568" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:11.495836 20951 raft_consensus.cc:2468] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:11.498025 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 81cad3626c3646ed896f5fbda5206d01 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:17:11.498106 21230 leader_election.cc:290] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:11.500188 21014 tablet_service.cc:1511] Processing CreateTablet for tablet 81cad3626c3646ed896f5fbda5206d01 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:17:11.502276 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cad3626c3646ed896f5fbda5206d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.502079 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fda386048b4838a8b10fbe2d602568" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:11.502976 21025 raft_consensus.cc:2468] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:11.504467 21066 leader_election.cc:304] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:11.514627 21230 raft_consensus.cc:2804] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.514590 21086 tablet_service.cc:1511] Processing CreateTablet for tablet 81cad3626c3646ed896f5fbda5206d01 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:17:11.515249 21230 raft_consensus.cc:493] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.515654 21230 raft_consensus.cc:3060] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.516261 21086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cad3626c3646ed896f5fbda5206d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.521966 20941 tablet_service.cc:1511] Processing CreateTablet for tablet 38019ba094624915aa5f69bc244bc955 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:17:11.524471 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 741c5b82a89142c2840a00c76ede6243 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:17:11.526168 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741c5b82a89142c2840a00c76ede6243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.524967 21230 raft_consensus.cc:515] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:11.531440 21219 raft_consensus.cc:359] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:11.532086 21012 tablet_service.cc:1511] Processing CreateTablet for tablet 38019ba094624915aa5f69bc244bc955 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:17:11.541990 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38019ba094624915aa5f69bc244bc955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.543511 21164 tablet_service.cc:1511] Processing CreateTablet for tablet 38019ba094624915aa5f69bc244bc955 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:17:11.535725 21220 tablet_bootstrap.cc:654] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.544957 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38019ba094624915aa5f69bc244bc955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.537977 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38019ba094624915aa5f69bc244bc955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.540313 21087 tablet_service.cc:1511] Processing CreateTablet for tablet 698227e80f3c48a28b91fa4df79955cf (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:17:11.546448 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698227e80f3c48a28b91fa4df79955cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.524498 21221 tablet_bootstrap.cc:654] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.556370 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81cad3626c3646ed896f5fbda5206d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.568912 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf04654bfa04b2d8d24ec0581483a23" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:11.573283 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b836c6e345b4404b5fbc1f5d42b5a15" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:11.575313 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fda386048b4838a8b10fbe2d602568" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:11.554013 21219 raft_consensus.cc:385] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.576835 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:11.579589 21219 raft_consensus.cc:740] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.579859 21174 raft_consensus.cc:2468] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:11.580502 21219 consensus_queue.cc:260] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:11.583207 21219 ts_tablet_manager.cc:1434] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.128s	user 0.009s	sys 0.000s
I20260709 12:17:11.584372 21219 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:11.587566 21230 leader_election.cc:290] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:11.587980 20950 raft_consensus.cc:3060] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.590921 20940 tablet_service.cc:1511] Processing CreateTablet for tablet 698227e80f3c48a28b91fa4df79955cf (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:17:11.591713 21219 tablet_bootstrap.cc:654] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.592643 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698227e80f3c48a28b91fa4df79955cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.591007 21222 ts_tablet_manager.cc:1434] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.147s	user 0.019s	sys 0.017s
I20260709 12:17:11.601923 21226 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.608385 21226 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.610926 21226 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:11.615149 21013 tablet_service.cc:1511] Processing CreateTablet for tablet b79db6c634bd4743b1cb15b5310dd4f8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:17:11.617131 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b79db6c634bd4743b1cb15b5310dd4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.575559 21139 leader_election.cc:304] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:11.624639 21220 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:11.625350 21220 ts_tablet_manager.cc:1403] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.179s	user 0.002s	sys 0.014s
I20260709 12:17:11.627254 21226 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.628582 21274 raft_consensus.cc:3060] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:11.628638 21139 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 53379.5us
W20260709 12:17:11.630489 21139 leader_election.cc:343] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:11.617981 21066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 62531.1us
I20260709 12:17:11.630992 21139 leader_election.cc:304] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:11.633307 21228 raft_consensus.cc:2749] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.627995 21226 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:11.636379 20914 leader_election.cc:304] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:11.639662 21222 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:11.640496 21226 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:11.641080 21226 raft_consensus.cc:2804] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.641426 21226 raft_consensus.cc:493] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.641515 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:11.644477 20950 raft_consensus.cc:2468] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
W20260709 12:17:11.645087 20917 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.648813 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fda386048b4838a8b10fbe2d602568" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:11.648970 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:11.649727 21026 raft_consensus.cc:3060] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.649801 21100 raft_consensus.cc:2468] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:11.651427 20916 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:11.652264 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:11.643412 21226 raft_consensus.cc:3060] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.657066 21280 raft_consensus.cc:2804] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.663211 21162 tablet_service.cc:1511] Processing CreateTablet for tablet b79db6c634bd4743b1cb15b5310dd4f8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:17:11.664808 21162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b79db6c634bd4743b1cb15b5310dd4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.667250 21087 tablet_service.cc:1511] Processing CreateTablet for tablet b79db6c634bd4743b1cb15b5310dd4f8 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:17:11.673911 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 395043703c424af7b41072d0e92012a1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:17:11.682324 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395043703c424af7b41072d0e92012a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.682608 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:11.683708 21174 raft_consensus.cc:2468] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
W20260709 12:17:11.672863 20914 leader_election.cc:343] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.687086 20914 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:11.672446 21063 leader_election.cc:304] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:11.675596 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b79db6c634bd4743b1cb15b5310dd4f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.694001 21230 raft_consensus.cc:2804] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.691119 21221 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:11.704743 21286 raft_consensus.cc:493] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.706954 21286 raft_consensus.cc:515] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.671993 21280 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.678647 21164 tablet_service.cc:1511] Processing CreateTablet for tablet 698227e80f3c48a28b91fa4df79955cf (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:17:11.674755 21274 raft_consensus.cc:2749] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.680552 21026 raft_consensus.cc:2468] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:11.680764 21220 raft_consensus.cc:359] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.729664 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ab161162c04ac8b68e594a1fbabe36" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:11.739058 21011 tablet_service.cc:1511] Processing CreateTablet for tablet 395043703c424af7b41072d0e92012a1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:17:11.747233 21222 tablet_bootstrap.cc:654] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.753168 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395043703c424af7b41072d0e92012a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.742872 21220 raft_consensus.cc:385] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.754024 21220 raft_consensus.cc:740] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.755426 21220 consensus_queue.cc:260] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.745796 21088 tablet_service.cc:1511] Processing CreateTablet for tablet 395043703c424af7b41072d0e92012a1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:17:11.712821 21221 ts_tablet_manager.cc:1403] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.287s	user 0.021s	sys 0.011s
I20260709 12:17:11.735895 21280 raft_consensus.cc:3060] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.758508 21220 ts_tablet_manager.cc:1434] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.127s	user 0.007s	sys 0.003s
I20260709 12:17:11.735773 21286 leader_election.cc:290] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:11.766057 21088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395043703c424af7b41072d0e92012a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.723125 21226 raft_consensus.cc:515] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.763731 21230 raft_consensus.cc:697] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:11.783964 21160 tablet_service.cc:1511] Processing CreateTablet for tablet e842b1b7f85a4cd9a991bb35684df6b9 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:17:11.788231 20939 tablet_service.cc:1511] Processing CreateTablet for tablet e842b1b7f85a4cd9a991bb35684df6b9 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:17:11.786655 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698227e80f3c48a28b91fa4df79955cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.790122 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e842b1b7f85a4cd9a991bb35684df6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.791265 21226 leader_election.cc:290] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:11.791953 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e842b1b7f85a4cd9a991bb35684df6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.792430 21283 raft_consensus.cc:2804] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.792595 21014 tablet_service.cc:1511] Processing CreateTablet for tablet e842b1b7f85a4cd9a991bb35684df6b9 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:17:11.794032 21014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e842b1b7f85a4cd9a991bb35684df6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.795285 21219 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:11.795828 21219 ts_tablet_manager.cc:1403] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.212s	user 0.013s	sys 0.017s
I20260709 12:17:11.792933 21283 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.797955 21283 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.799445 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:11.800158 21100 raft_consensus.cc:3060] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.799800 21090 tablet_service.cc:1511] Processing CreateTablet for tablet 352e00dbe52446aca63ec1b6cc70f554 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:17:11.801275 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352e00dbe52446aca63ec1b6cc70f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.802542 21280 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.804765 21280 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:11.805936 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:11.806043 21230 consensus_queue.cc:237] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:11.806690 21099 raft_consensus.cc:3060] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.807739 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 352e00dbe52446aca63ec1b6cc70f554 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:17:11.809201 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352e00dbe52446aca63ec1b6cc70f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.810634 21012 tablet_service.cc:1511] Processing CreateTablet for tablet 505a760357a94c70a8de37a7358e7ab5 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:17:11.812013 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505a760357a94c70a8de37a7358e7ab5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:11.775728 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 88737.6us
I20260709 12:17:11.801505 21221 raft_consensus.cc:359] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.814406 21221 raft_consensus.cc:385] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.817271 21221 raft_consensus.cc:740] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.817534 21220 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:11.816845 21100 raft_consensus.cc:2468] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:11.811095 21219 raft_consensus.cc:359] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.821308 21219 raft_consensus.cc:385] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.821437 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17ab161162c04ac8b68e594a1fbabe36" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:11.821857 21219 raft_consensus.cc:740] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.822500 20950 raft_consensus.cc:2393] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:11.822981 21219 consensus_queue.cc:260] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.823654 21063 leader_election.cc:304] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:11.824586 21286 raft_consensus.cc:3060] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.826242 21172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:11.827525 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:11.828150 21173 raft_consensus.cc:3060] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.831751 20941 tablet_service.cc:1511] Processing CreateTablet for tablet f85cfddd6d98445c951b81e38136c039 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:17:11.832957 21286 raft_consensus.cc:2749] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.833005 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85cfddd6d98445c951b81e38136c039. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:11.831692 20914 leader_election.cc:343] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.818427 21221 consensus_queue.cc:260] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.840827 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 505a760357a94c70a8de37a7358e7ab5 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:17:11.843894 21089 tablet_service.cc:1511] Processing CreateTablet for tablet 505a760357a94c70a8de37a7358e7ab5 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:17:11.845254 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505a760357a94c70a8de37a7358e7ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.846395 20916 leader_election.cc:304] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:11.847517 21280 raft_consensus.cc:2804] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.848016 21280 raft_consensus.cc:697] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:11.848829 21280 consensus_queue.cc:237] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.857555 21219 ts_tablet_manager.cc:1434] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.000s
I20260709 12:17:11.857800 20940 tablet_service.cc:1511] Processing CreateTablet for tablet 352e00dbe52446aca63ec1b6cc70f554 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:17:11.859169 21219 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:11.859411 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352e00dbe52446aca63ec1b6cc70f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.869551 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505a760357a94c70a8de37a7358e7ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.888031 21173 raft_consensus.cc:2468] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:11.890148 21231 raft_consensus.cc:493] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.890722 21231 raft_consensus.cc:515] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:11.890982 21013 tablet_service.cc:1511] Processing CreateTablet for tablet f85cfddd6d98445c951b81e38136c039 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:17:11.892328 21013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85cfddd6d98445c951b81e38136c039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.893026 21231 leader_election.cc:290] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:11.874653 21099 raft_consensus.cc:2468] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:11.928637 21222 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:11.929270 21222 ts_tablet_manager.cc:1403] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.294s	user 0.043s	sys 0.023s
I20260709 12:17:11.932809 21222 raft_consensus.cc:359] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.933562 21222 raft_consensus.cc:385] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.933986 21222 raft_consensus.cc:740] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.934870 21222 consensus_queue.cc:260] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.937803 21222 ts_tablet_manager.cc:1434] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 12:17:11.938640 21221 ts_tablet_manager.cc:1434] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.173s	user 0.006s	sys 0.001s
I20260709 12:17:11.938925 21222 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:11.939487 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:11.910594 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:11.958401 20950 raft_consensus.cc:2468] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:11.960731 20846 catalog_manager.cc:5697] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:11.962293 20991 leader_election.cc:343] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.966922 21162 tablet_service.cc:1511] Processing CreateTablet for tablet fc706075ca374b5eb2cc3867a4f7a456 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:17:11.947172 21219 tablet_bootstrap.cc:654] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.949813 20916 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:11.950534 21221 tablet_bootstrap.cc:492] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:11.951831 21087 tablet_service.cc:1511] Processing CreateTablet for tablet ca20df817314411a9f94ee42a3b5220a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:17:11.952775 21090 tablet_service.cc:1511] Processing CreateTablet for tablet f85cfddd6d98445c951b81e38136c039 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:17:11.944407 21220 tablet_bootstrap.cc:654] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.974475 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85cfddd6d98445c951b81e38136c039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.967106 21011 tablet_service.cc:1511] Processing CreateTablet for tablet fc706075ca374b5eb2cc3867a4f7a456 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:17:11.999215 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc706075ca374b5eb2cc3867a4f7a456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.978808 21280 raft_consensus.cc:2804] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.980202 21222 tablet_bootstrap.cc:654] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.002127 21280 raft_consensus.cc:697] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:12.003177 21280 consensus_queue.cc:237] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:11.979559 20989 leader_election.cc:304] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:11.983073 21163 tablet_service.cc:1511] Processing CreateTablet for tablet ca20df817314411a9f94ee42a3b5220a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:17:11.983174 20940 tablet_service.cc:1511] Processing CreateTablet for tablet ca20df817314411a9f94ee42a3b5220a (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:17:11.968590 21162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc706075ca374b5eb2cc3867a4f7a456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.985878 20941 tablet_service.cc:1511] Processing CreateTablet for tablet fc706075ca374b5eb2cc3867a4f7a456 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:17:11.985607 21012 tablet_service.cc:1511] Processing CreateTablet for tablet 7a820defdda646daaa5ffa1d6aba18f1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:17:11.989527 21232 consensus_queue.cc:1048] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.006681 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca20df817314411a9f94ee42a3b5220a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:11.972647 21283 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.019763 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a820defdda646daaa5ffa1d6aba18f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.021023 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc706075ca374b5eb2cc3867a4f7a456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.036648 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca20df817314411a9f94ee42a3b5220a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.028155 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca20df817314411a9f94ee42a3b5220a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.024411 21231 raft_consensus.cc:2804] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:17:12.039634 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 60189.6us
I20260709 12:17:12.059573 21283 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:12.049861 21231 raft_consensus.cc:493] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.060714 21231 raft_consensus.cc:3060] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.063937 21219 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:12.064656 21219 ts_tablet_manager.cc:1403] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.206s	user 0.019s	sys 0.028s
I20260709 12:17:12.068334 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:12.076983 21090 tablet_service.cc:1511] Processing CreateTablet for tablet 7a820defdda646daaa5ffa1d6aba18f1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:17:12.068485 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:12.080559 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a820defdda646daaa5ffa1d6aba18f1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:12.094662 20917 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.081347 21173 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.099761 21220 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:12.101565 21220 ts_tablet_manager.cc:1403] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.284s	user 0.011s	sys 0.014s
W20260709 12:17:12.100255 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:12.100678 21221 tablet_bootstrap.cc:654] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.105654 21164 tablet_service.cc:1511] Processing CreateTablet for tablet 7a820defdda646daaa5ffa1d6aba18f1 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:17:12.107142 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a820defdda646daaa5ffa1d6aba18f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.106858 20846 catalog_manager.cc:5697] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.101294 21232 consensus_queue.cc:1048] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.114687 21220 raft_consensus.cc:359] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.121857 21220 raft_consensus.cc:385] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.123553 21220 raft_consensus.cc:740] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.123929 21173 raft_consensus.cc:2468] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:12.119089 21219 raft_consensus.cc:359] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.124614 21219 raft_consensus.cc:385] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.125001 21219 raft_consensus.cc:740] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.125264 20914 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:12.126094 21219 consensus_queue.cc:260] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.126932 21283 raft_consensus.cc:2804] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.127585 21283 raft_consensus.cc:697] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:12.129078 21219 ts_tablet_manager.cc:1434] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.003s
I20260709 12:17:12.131191 21222 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:12.122700 21231 raft_consensus.cc:515] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.131835 21222 ts_tablet_manager.cc:1403] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.193s	user 0.021s	sys 0.019s
I20260709 12:17:12.129913 21220 consensus_queue.cc:260] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.128733 21283 consensus_queue.cc:237] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.138090 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:12.146409 20951 raft_consensus.cc:3060] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.149540 20941 tablet_service.cc:1511] Processing CreateTablet for tablet a47e447bb0004896847c1e74485df964 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:17:12.150861 20941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47e447bb0004896847c1e74485df964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.140027 21012 tablet_service.cc:1511] Processing CreateTablet for tablet a29e7a2e40b345b09d50b1699ce33f47 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:17:12.143826 20940 tablet_service.cc:1511] Processing CreateTablet for tablet a29e7a2e40b345b09d50b1699ce33f47 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:17:12.152946 20940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a29e7a2e40b345b09d50b1699ce33f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.152849 20846 catalog_manager.cc:5697] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:12.152854 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a29e7a2e40b345b09d50b1699ce33f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.159435 21219 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:12.145459 21221 tablet_bootstrap.cc:492] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:12.175078 20951 raft_consensus.cc:2468] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:12.176571 20989 leader_election.cc:304] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:12.149164 21222 raft_consensus.cc:359] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.178040 21222 raft_consensus.cc:385] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.178378 21222 raft_consensus.cc:740] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.179368 21222 consensus_queue.cc:260] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.180727 21232 raft_consensus.cc:2804] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.180842 21089 tablet_service.cc:1511] Processing CreateTablet for tablet 03a4ea8a219940a3852084f1d3451536 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:17:12.181370 21232 raft_consensus.cc:697] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:12.182185 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a4ea8a219940a3852084f1d3451536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.182317 21232 consensus_queue.cc:237] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.186632 21163 tablet_service.cc:1511] Processing CreateTablet for tablet 03a4ea8a219940a3852084f1d3451536 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:17:12.187937 21163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a4ea8a219940a3852084f1d3451536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.193480 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 03a4ea8a219940a3852084f1d3451536 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:17:12.194824 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a4ea8a219940a3852084f1d3451536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.177415 21221 ts_tablet_manager.cc:1403] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.227s	user 0.008s	sys 0.013s
I20260709 12:17:12.177345 21087 tablet_service.cc:1511] Processing CreateTablet for tablet a29e7a2e40b345b09d50b1699ce33f47 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:17:12.203675 21090 tablet_service.cc:1511] Processing CreateTablet for tablet 0468c28c130f44858399b670b2b9cb8b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:17:12.207978 21160 tablet_service.cc:1511] Processing CreateTablet for tablet 0468c28c130f44858399b670b2b9cb8b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:17:12.214551 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0468c28c130f44858399b670b2b9cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.203504 21012 tablet_service.cc:1511] Processing CreateTablet for tablet a47e447bb0004896847c1e74485df964 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:17:12.217552 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47e447bb0004896847c1e74485df964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.199517 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a29e7a2e40b345b09d50b1699ce33f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.199513 21011 tablet_service.cc:1511] Processing CreateTablet for tablet 0468c28c130f44858399b670b2b9cb8b (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:17:12.206128 20937 tablet_service.cc:1511] Processing CreateTablet for tablet f01e89f8d2d941d494585d3147a3dd2f (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:17:12.220229 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0468c28c130f44858399b670b2b9cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.220968 20937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01e89f8d2d941d494585d3147a3dd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.222327 21164 tablet_service.cc:1511] Processing CreateTablet for tablet a47e447bb0004896847c1e74485df964 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:17:12.223701 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47e447bb0004896847c1e74485df964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.201349 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:12.226181 21100 raft_consensus.cc:3060] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.198309 21231 leader_election.cc:290] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:12.228708 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0468c28c130f44858399b670b2b9cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.227106 21220 ts_tablet_manager.cc:1434] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.125s	user 0.005s	sys 0.003s
I20260709 12:17:12.234766 21220 tablet_bootstrap.cc:492] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:12.260555 21100 raft_consensus.cc:2468] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:12.264778 21221 raft_consensus.cc:359] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.265758 21221 raft_consensus.cc:385] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.266134 21221 raft_consensus.cc:740] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.267168 21221 consensus_queue.cc:260] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.268308 21219 tablet_bootstrap.cc:654] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.269984 21221 ts_tablet_manager.cc:1434] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.059s	user 0.004s	sys 0.003s
I20260709 12:17:12.210978 21222 ts_tablet_manager.cc:1434] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.003s
I20260709 12:17:12.272181 21222 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:12.277386 21220 tablet_bootstrap.cc:654] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.300292 21231 raft_consensus.cc:493] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.300887 21231 raft_consensus.cc:515] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.303143 21231 leader_election.cc:290] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:12.304705 21221 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:12.306246 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b836c6e345b4404b5fbc1f5d42b5a15" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:12.307287 21173 raft_consensus.cc:2468] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:12.311721 21221 tablet_bootstrap.cc:654] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.312790 21222 tablet_bootstrap.cc:654] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.321537 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b836c6e345b4404b5fbc1f5d42b5a15" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:12.322283 20846 catalog_manager.cc:5697] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.325798 21090 tablet_service.cc:1511] Processing CreateTablet for tablet 16b20f7ffdc94676bd1cbade62c4dcc5 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:17:12.325977 20937 tablet_service.cc:1511] Processing CreateTablet for tablet 3e5b0a17978446c6bee0b8db24be9e49 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:17:12.327111 21090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b20f7ffdc94676bd1cbade62c4dcc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.327019 21160 tablet_service.cc:1511] Processing CreateTablet for tablet 16b20f7ffdc94676bd1cbade62c4dcc5 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:17:12.328274 21160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b20f7ffdc94676bd1cbade62c4dcc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.328169 21011 tablet_service.cc:1511] Processing CreateTablet for tablet 3e5b0a17978446c6bee0b8db24be9e49 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:17:12.329360 21011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5b0a17978446c6bee0b8db24be9e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.331718 21012 tablet_service.cc:1511] Processing CreateTablet for tablet f01e89f8d2d941d494585d3147a3dd2f (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:17:12.333470 21012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01e89f8d2d941d494585d3147a3dd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.343698 21087 tablet_service.cc:1511] Processing CreateTablet for tablet f01e89f8d2d941d494585d3147a3dd2f (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:17:12.344959 21087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01e89f8d2d941d494585d3147a3dd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.356443 20937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5b0a17978446c6bee0b8db24be9e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.336942 20989 leader_election.cc:304] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:12.374336 21164 tablet_service.cc:1511] Processing CreateTablet for tablet 3e5b0a17978446c6bee0b8db24be9e49 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:17:12.374459 20939 tablet_service.cc:1511] Processing CreateTablet for tablet 16b20f7ffdc94676bd1cbade62c4dcc5 (DEFAULT_TABLE table=test-workload [id=08aa035b051c4f0fa6bc2e92730affc0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:17:12.378377 21219 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:12.378715 20939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16b20f7ffdc94676bd1cbade62c4dcc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.378881 21219 ts_tablet_manager.cc:1403] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.247s	user 0.015s	sys 0.006s
I20260709 12:17:12.380079 21164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5b0a17978446c6bee0b8db24be9e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.380918 21221 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:12.381424 21221 ts_tablet_manager.cc:1403] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 12:17:12.375712 20847 catalog_manager.cc:5697] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:12.384488 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 74378.8us
W20260709 12:17:12.379230 20991 leader_election.cc:343] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.385845 21232 raft_consensus.cc:2804] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.386298 21232 raft_consensus.cc:493] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.386689 21232 raft_consensus.cc:3060] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:12.387643 20991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 44770us
I20260709 12:17:12.392817 21220 tablet_bootstrap.cc:492] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:12.393419 21220 ts_tablet_manager.cc:1403] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.159s	user 0.007s	sys 0.025s
I20260709 12:17:12.395452 21219 raft_consensus.cc:359] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.396222 21219 raft_consensus.cc:385] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.396507 21219 raft_consensus.cc:740] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.386899 21221 raft_consensus.cc:359] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.397315 21221 raft_consensus.cc:385] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.397950 21221 raft_consensus.cc:740] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.400381 21222 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:12.396618 21220 raft_consensus.cc:359] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.406352 21220 raft_consensus.cc:385] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.406028 21222 ts_tablet_manager.cc:1403] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.134s	user 0.019s	sys 0.033s
I20260709 12:17:12.401218 21286 consensus_queue.cc:1048] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.397197 21219 consensus_queue.cc:260] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.410058 21219 ts_tablet_manager.cc:1434] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 12:17:12.404111 21221 consensus_queue.cc:260] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.414620 21274 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.422710 21274 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.426209 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:12.427173 21099 raft_consensus.cc:2393] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:12.427969 21283 consensus_queue.cc:1048] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.429317 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:12.431066 21137 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:12.432914 21274 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:12.434041 21274 raft_consensus.cc:3060] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.435150 21232 raft_consensus.cc:515] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.414965 21220 raft_consensus.cc:740] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.425261 21286 consensus_queue.cc:1048] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.443599 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b836c6e345b4404b5fbc1f5d42b5a15" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:12.446429 21220 consensus_queue.cc:260] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.448939 21099 raft_consensus.cc:3060] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.449103 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b836c6e345b4404b5fbc1f5d42b5a15" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:12.449980 21174 raft_consensus.cc:3060] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.454764 21283 consensus_queue.cc:1048] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.463274 21283 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.463954 21283 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.436746 21219 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:12.412596 21222 raft_consensus.cc:359] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.468401 21222 raft_consensus.cc:385] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.468703 21222 raft_consensus.cc:740] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.469374 21222 consensus_queue.cc:260] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.472484 21099 raft_consensus.cc:2468] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:12.475840 21222 ts_tablet_manager.cc:1434] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.001s
I20260709 12:17:12.476826 21222 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:12.481717 21283 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:12.483084 21231 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.483567 21231 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.484077 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:12.484907 21171 raft_consensus.cc:2468] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:12.486282 21174 raft_consensus.cc:2468] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:12.487586 20989 leader_election.cc:304] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:12.487782 21231 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:12.488440 21231 raft_consensus.cc:2804] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.488888 21231 raft_consensus.cc:697] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:12.489379 21222 tablet_bootstrap.cc:654] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.490036 20914 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:12.491225 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:12.492093 21171 raft_consensus.cc:2393] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:12.493359 21231 consensus_queue.cc:237] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.493837 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:12.495769 21280 raft_consensus.cc:2804] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.496505 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:12.510666 21280 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.499769 21221 ts_tablet_manager.cc:1434] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.118s	user 0.008s	sys 0.000s
W20260709 12:17:12.510959 20917 leader_election.cc:343] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.511250 21280 raft_consensus.cc:3060] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.512455 21221 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:12.519560 21312 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.520409 21312 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:12.522725 21137 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 91774.4us
I20260709 12:17:12.522869 20989 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:12.524187 21297 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.449980 21232 leader_election.cc:290] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:12.537190 21297 raft_consensus.cc:2749] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:12.504153 21274 raft_consensus.cc:2749] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:12.480991 21220 ts_tablet_manager.cc:1434] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 12:17:12.506278 21219 tablet_bootstrap.cc:654] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.509053 21283 consensus_queue.cc:1048] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.545698 21220 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:12.560520 21220 tablet_bootstrap.cc:654] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.579627 21226 raft_consensus.cc:493] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.580504 21226 raft_consensus.cc:515] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.584671 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:12.585381 21226 leader_election.cc:290] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:12.585991 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:12.587895 20916 leader_election.cc:343] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.589670 21312 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:12.590945 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:12.591055 21283 consensus_queue.cc:1048] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:17:12.592516 21140 leader_election.cc:343] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.593103 21283 consensus_queue.cc:1048] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.596961 21172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:12.598361 21172 raft_consensus.cc:2468] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:12.603545 21221 tablet_bootstrap.cc:654] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.603843 20914 leader_election.cc:304] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:12.608150 21226 raft_consensus.cc:2804] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.608618 21226 raft_consensus.cc:493] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.609027 21226 raft_consensus.cc:3060] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.609056 21219 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:12.610121 21219 ts_tablet_manager.cc:1403] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.174s	user 0.013s	sys 0.008s
I20260709 12:17:12.613483 21219 raft_consensus.cc:359] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.622653 21219 raft_consensus.cc:385] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.623014 21219 raft_consensus.cc:740] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.627779 21219 consensus_queue.cc:260] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.638521 21219 ts_tablet_manager.cc:1434] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:17:12.655679 21222 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:12.656433 21222 ts_tablet_manager.cc:1403] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.180s	user 0.053s	sys 0.078s
I20260709 12:17:12.657864 21219 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:12.664683 21222 raft_consensus.cc:359] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.665673 21222 raft_consensus.cc:385] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.666035 21222 raft_consensus.cc:740] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.667001 21222 consensus_queue.cc:260] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.670540 20849 catalog_manager.cc:5697] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.673738 21219 tablet_bootstrap.cc:654] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.677963 21222 ts_tablet_manager.cc:1434] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 12:17:12.682775 21222 tablet_bootstrap.cc:492] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:12.682879 21280 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.683882 21312 raft_consensus.cc:493] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.684104 20950 raft_consensus.cc:2393] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:12.684440 21312 raft_consensus.cc:515] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.689746 21222 tablet_bootstrap.cc:654] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.690819 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041cbf53c7e4c588bb5a0b699284c33" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:12.693954 21312 leader_election.cc:290] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:12.696571 21137 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:12.697160 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041cbf53c7e4c588bb5a0b699284c33" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:12.698173 20950 raft_consensus.cc:2468] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:12.698973 21312 raft_consensus.cc:3060] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.706192 21172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
W20260709 12:17:12.707383 21319 log.cc:927] Time spent T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 12:17:12.710122 21140 leader_election.cc:343] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.710424 21280 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:12.693954 21226 raft_consensus.cc:515] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.719954 21297 consensus_queue.cc:1048] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.722018 21137 leader_election.cc:304] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:12.724926 21312 raft_consensus.cc:2749] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:12.725162 21315 consensus_queue.cc:1048] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.727203 21221 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:12.727427 21274 raft_consensus.cc:2804] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:12.727779 21221 ts_tablet_manager.cc:1403] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.216s	user 0.029s	sys 0.041s
I20260709 12:17:12.727836 21274 raft_consensus.cc:493] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.728295 21274 raft_consensus.cc:3060] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.731096 21221 raft_consensus.cc:359] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.732878 21221 raft_consensus.cc:385] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.733189 21221 raft_consensus.cc:740] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.736094 21219 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:12.737568 21219 ts_tablet_manager.cc:1403] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.092s	user 0.032s	sys 0.006s
I20260709 12:17:12.739159 21220 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:12.739782 21220 ts_tablet_manager.cc:1403] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.194s	user 0.045s	sys 0.050s
I20260709 12:17:12.743767 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:12.745064 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
W20260709 12:17:12.747241 20916 leader_election.cc:343] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.755542 21219 raft_consensus.cc:359] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.756175 21226 leader_election.cc:290] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:12.756529 21219 raft_consensus.cc:385] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.756994 21219 raft_consensus.cc:740] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.757499 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:12.758167 21173 raft_consensus.cc:3060] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.757920 21219 consensus_queue.cc:260] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.759171 21274 raft_consensus.cc:515] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.761265 21219 ts_tablet_manager.cc:1434] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:17:12.762669 21219 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:12.764842 21274 leader_election.cc:290] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:12.766567 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041cbf53c7e4c588bb5a0b699284c33" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:12.767155 21172 raft_consensus.cc:2468] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:12.769001 21221 consensus_queue.cc:260] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.769032 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041cbf53c7e4c588bb5a0b699284c33" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:12.769948 21026 raft_consensus.cc:3060] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.770186 20951 raft_consensus.cc:3060] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.769260 20914 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:12.776875 21226 raft_consensus.cc:2804] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.778610 21173 raft_consensus.cc:2468] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
W20260709 12:17:12.780032 21140 leader_election.cc:343] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.781183 21226 raft_consensus.cc:697] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:12.783499 21226 consensus_queue.cc:237] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.785298 20914 leader_election.cc:304] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:12.787585 21026 raft_consensus.cc:2468] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:12.793207 20951 raft_consensus.cc:2468] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:12.795917 21221 ts_tablet_manager.cc:1434] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.004s
I20260709 12:17:12.797073 21221 tablet_bootstrap.cc:492] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:12.798250 21280 raft_consensus.cc:2804] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.798832 21280 raft_consensus.cc:697] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:12.799099 21232 consensus_queue.cc:1048] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.799713 21280 consensus_queue.cc:237] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.803460 21221 tablet_bootstrap.cc:654] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.806335 21222 tablet_bootstrap.cc:492] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:12.806938 21222 ts_tablet_manager.cc:1403] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.124s	user 0.025s	sys 0.032s
I20260709 12:17:12.807149 21219 tablet_bootstrap.cc:654] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.809896 21220 raft_consensus.cc:359] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.810462 21220 raft_consensus.cc:385] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.810694 21220 raft_consensus.cc:740] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.814795 21137 leader_election.cc:304] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:12.814932 21220 consensus_queue.cc:260] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.815961 21222 raft_consensus.cc:359] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.816812 21222 raft_consensus.cc:385] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.817235 21222 raft_consensus.cc:740] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.815707 21274 raft_consensus.cc:2804] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:12.821405 21220 ts_tablet_manager.cc:1434] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.002s
I20260709 12:17:12.822458 21220 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:12.820061 21222 consensus_queue.cc:260] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.827605 21222 ts_tablet_manager.cc:1434] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:17:12.828563 21274 raft_consensus.cc:697] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:12.829499 21274 consensus_queue.cc:237] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.829520 21222 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:12.836418 21220 tablet_bootstrap.cc:654] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.840677 21222 tablet_bootstrap.cc:654] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.854996 20848 catalog_manager.cc:5697] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.862378 20846 catalog_manager.cc:5697] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.866415 21221 tablet_bootstrap.cc:492] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:12.867064 21221 ts_tablet_manager.cc:1403] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.022s
I20260709 12:17:12.869475 21219 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:12.869973 21221 raft_consensus.cc:359] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.870198 21219 ts_tablet_manager.cc:1403] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.003s
I20260709 12:17:12.870579 21221 raft_consensus.cc:385] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.870973 21221 raft_consensus.cc:740] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.871608 21221 consensus_queue.cc:260] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.874122 21219 raft_consensus.cc:359] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.875099 21221 ts_tablet_manager.cc:1434] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:17:12.875094 21219 raft_consensus.cc:385] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.875735 21219 raft_consensus.cc:740] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.876097 21221 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:12.877831 21219 consensus_queue.cc:260] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.883958 21219 ts_tablet_manager.cc:1434] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:17:12.886224 21220 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:12.886741 21220 ts_tablet_manager.cc:1403] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.006s
I20260709 12:17:12.889473 21220 raft_consensus.cc:359] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.890167 21220 raft_consensus.cc:385] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.890430 21220 raft_consensus.cc:740] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.891445 21221 tablet_bootstrap.cc:654] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.892221 21219 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:12.892346 21222 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:12.892851 21222 ts_tablet_manager.cc:1403] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.064s	user 0.037s	sys 0.007s
I20260709 12:17:12.895857 21222 raft_consensus.cc:359] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.896567 21222 raft_consensus.cc:385] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.896827 21222 raft_consensus.cc:740] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.891095 21220 consensus_queue.cc:260] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.900782 21220 ts_tablet_manager.cc:1434] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:17:12.897987 21222 consensus_queue.cc:260] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.901873 21220 tablet_bootstrap.cc:492] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:12.903088 21221 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:12.903595 21221 ts_tablet_manager.cc:1403] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.003s
I20260709 12:17:12.903697 21222 ts_tablet_manager.cc:1434] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:17:12.904805 21222 tablet_bootstrap.cc:492] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:12.906646 21221 raft_consensus.cc:359] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.907327 21221 raft_consensus.cc:385] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.907610 21221 raft_consensus.cc:740] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.908442 21220 tablet_bootstrap.cc:654] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.908304 21221 consensus_queue.cc:260] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.911518 21222 tablet_bootstrap.cc:654] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.911934 21221 ts_tablet_manager.cc:1434] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:17:12.913043 21221 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:12.914732 21219 tablet_bootstrap.cc:654] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.916143 21231 consensus_queue.cc:1048] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.919428 21221 tablet_bootstrap.cc:654] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.927927 21219 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:12.928508 21219 ts_tablet_manager.cc:1403] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.003s
I20260709 12:17:12.932021 21221 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:12.926972 20848 catalog_manager.cc:5697] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:12.936410 21226 raft_consensus.cc:493] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.937093 21226 raft_consensus.cc:515] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.938607 21232 consensus_queue.cc:1048] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.940528 21222 tablet_bootstrap.cc:492] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:12.941092 21222 ts_tablet_manager.cc:1403] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.007s
I20260709 12:17:12.941033 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:12.943086 20916 leader_election.cc:343] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.941202 21226 leader_election.cc:290] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:12.944464 21222 raft_consensus.cc:359] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.946537 21222 raft_consensus.cc:385] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.941339 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:12.946969 21222 raft_consensus.cc:740] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.947945 21220 tablet_bootstrap.cc:492] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:12.947835 21222 consensus_queue.cc:260] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
W20260709 12:17:12.948422 20917 leader_election.cc:343] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:12.949160 20917 leader_election.cc:304] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:12.950213 21226 raft_consensus.cc:2749] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:12.950317 21222 ts_tablet_manager.cc:1434] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:17:12.948511 21220 ts_tablet_manager.cc:1403] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.009s
I20260709 12:17:12.951467 21222 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:12.952769 21221 ts_tablet_manager.cc:1403] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.005s
I20260709 12:17:12.954656 21232 raft_consensus.cc:493] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.960520 21232 raft_consensus.cc:515] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:12.960510 21220 raft_consensus.cc:359] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.962822 21232 leader_election.cc:290] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:12.962877 21220 raft_consensus.cc:385] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.963537 21220 raft_consensus.cc:740] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.964504 21220 consensus_queue.cc:260] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:12.955083 21219 raft_consensus.cc:359] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.966871 21220 ts_tablet_manager.cc:1434] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:17:12.967156 21219 raft_consensus.cc:385] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.967504 21219 raft_consensus.cc:740] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.967628 21221 raft_consensus.cc:359] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.968295 21221 raft_consensus.cc:385] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:12.968376 21220 tablet_bootstrap.cc:492] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:12.968883 21221 raft_consensus.cc:740] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.969313 21222 tablet_bootstrap.cc:654] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.969820 21221 consensus_queue.cc:260] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:12.970387 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041cbf53c7e4c588bb5a0b699284c33" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:12.968245 21219 consensus_queue.cc:260] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:12.972323 21221 ts_tablet_manager.cc:1434] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:17:12.973368 21221 tablet_bootstrap.cc:492] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:12.974040 21219 ts_tablet_manager.cc:1434] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:17:12.975272 21219 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:12.979267 21221 tablet_bootstrap.cc:654] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:12.965525 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5041cbf53c7e4c588bb5a0b699284c33" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:12.982231 20951 raft_consensus.cc:2393] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:12.987269 20989 leader_election.cc:304] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:12.989390 21219 tablet_bootstrap.cc:654] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.002709 21232 raft_consensus.cc:3060] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:12.992297 21221 tablet_bootstrap.cc:492] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.005959 21221 ts_tablet_manager.cc:1403] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.004s
I20260709 12:17:13.009366 21297 raft_consensus.cc:493] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.009966 21297 raft_consensus.cc:515] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.010205 21221 raft_consensus.cc:359] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.011251 21221 raft_consensus.cc:385] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.011575 21232 raft_consensus.cc:2749] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.011623 21221 raft_consensus.cc:740] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:12.999519 21220 tablet_bootstrap.cc:654] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.013491 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.013343 21221 consensus_queue.cc:260] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.014253 21099 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.015820 21221 ts_tablet_manager.cc:1434] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:17:13.015888 20991 leader_election.cc:304] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:13.017079 21221 tablet_bootstrap.cc:492] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:13.017258 21231 raft_consensus.cc:2804] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.017733 21231 raft_consensus.cc:493] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.018111 21231 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.018466 21232 raft_consensus.cc:493] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.019004 21232 raft_consensus.cc:515] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.021371 21232 leader_election.cc:290] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.023416 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044d3214b94a448394941bed6d2204a8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.024250 21099 raft_consensus.cc:2468] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.025883 20991 leader_election.cc:304] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:13.027410 21297 leader_election.cc:290] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.028035 21232 raft_consensus.cc:2804] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.029080 21232 raft_consensus.cc:493] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.029670 21232 raft_consensus.cc:3060] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.030242 21219 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:13.030844 21219 ts_tablet_manager.cc:1403] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.056s	user 0.011s	sys 0.008s
I20260709 12:17:13.028223 21221 tablet_bootstrap.cc:654] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.035534 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044d3214b94a448394941bed6d2204a8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.036410 20951 raft_consensus.cc:2468] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.026062 21220 tablet_bootstrap.cc:492] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:13.038144 21220 ts_tablet_manager.cc:1403] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.006s
I20260709 12:17:13.038203 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.041582 21173 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.034262 21219 raft_consensus.cc:359] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.039413 21231 raft_consensus.cc:515] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.049785 21231 leader_election.cc:290] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.051623 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.052356 21099 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.051842 21220 raft_consensus.cc:359] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.055267 21220 raft_consensus.cc:385] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.056682 21220 raft_consensus.cc:740] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.058110 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:13.058815 21173 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.062268 21231 raft_consensus.cc:493] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.062950 21231 raft_consensus.cc:515] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.066083 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.059010 21220 consensus_queue.cc:260] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.069362 21098 raft_consensus.cc:2468] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.069958 21231 leader_election.cc:290] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.071497 21274 raft_consensus.cc:493] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.071843 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.072099 21274 raft_consensus.cc:515] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.074561 21274 leader_election.cc:290] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.075695 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:13.076624 21026 raft_consensus.cc:2468] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:13.078354 21140 leader_election.cc:304] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:13.078966 21099 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:13.082096 21219 raft_consensus.cc:385] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.082453 21219 raft_consensus.cc:740] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.082619 20991 leader_election.cc:304] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:13.083659 21174 raft_consensus.cc:2468] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.084568 21231 raft_consensus.cc:2804] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.084450 21219 consensus_queue.cc:260] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.085021 21231 raft_consensus.cc:493] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.086241 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.086941 21219 ts_tablet_manager.cc:1434] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.001s
I20260709 12:17:13.085421 21231 raft_consensus.cc:3060] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.087348 21098 raft_consensus.cc:2468] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:13.089025 21221 tablet_bootstrap.cc:492] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.089591 21221 ts_tablet_manager.cc:1403] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.036s
I20260709 12:17:13.083717 20991 leader_election.cc:304] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:13.090776 21297 raft_consensus.cc:2804] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.093151 21274 raft_consensus.cc:2804] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.093927 21274 raft_consensus.cc:493] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.095031 21173 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:13.097110 21274 raft_consensus.cc:3060] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.092867 21221 raft_consensus.cc:359] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.099308 21221 raft_consensus.cc:385] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.100631 21221 raft_consensus.cc:740] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.101603 21221 consensus_queue.cc:260] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.112666 21226 raft_consensus.cc:493] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.113552 21226 raft_consensus.cc:515] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.114475 21222 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:13.068778 21232 raft_consensus.cc:515] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.115165 21222 ts_tablet_manager.cc:1403] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.164s	user 0.044s	sys 0.089s
I20260709 12:17:13.121795 21219 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:13.124249 21222 raft_consensus.cc:359] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.124976 21222 raft_consensus.cc:385] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.116652 21220 ts_tablet_manager.cc:1434] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 12:17:13.125264 21222 raft_consensus.cc:740] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.126528 21297 raft_consensus.cc:697] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:13.128261 21297 consensus_queue.cc:237] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.126802 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044d3214b94a448394941bed6d2204a8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:13.130092 20951 raft_consensus.cc:3060] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.130144 21222 consensus_queue.cc:260] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.131501 21312 raft_consensus.cc:493] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.130995 21232 leader_election.cc:290] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.132402 21222 ts_tablet_manager.cc:1434] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.001s
I20260709 12:17:13.132750 21220 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:13.134281 21222 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:13.134944 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044d3214b94a448394941bed6d2204a8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.135617 21099 raft_consensus.cc:3060] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.132026 21312 raft_consensus.cc:515] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.140432 21312 leader_election.cc:290] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.096227 21231 raft_consensus.cc:515] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.141573 21219 tablet_bootstrap.cc:654] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.144464 21222 tablet_bootstrap.cc:654] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.157050 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:13.146154 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.158777 20950 raft_consensus.cc:2468] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:13.160244 21220 tablet_bootstrap.cc:654] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.160163 21137 leader_election.cc:304] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:13.161244 21312 raft_consensus.cc:2804] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.161689 21312 raft_consensus.cc:493] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.162055 21312 raft_consensus.cc:3060] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.163225 21221 ts_tablet_manager.cc:1434] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.073s	user 0.022s	sys 0.028s
I20260709 12:17:13.164269 21221 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:13.168943 21226 leader_election.cc:290] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.170583 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.170842 21221 tablet_bootstrap.cc:654] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:13.172125 20916 leader_election.cc:343] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.174019 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.153100 21231 leader_election.cc:290] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.152976 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.179075 21097 raft_consensus.cc:3060] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.142588 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:13.180850 21139 leader_election.cc:343] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.152439 21274 raft_consensus.cc:515] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.187551 21173 raft_consensus.cc:2393] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:13.190260 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:13.191625 21026 raft_consensus.cc:2393] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:13.196344 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c050ac450e451d9a27af7cf32b7edc" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.198578 21139 leader_election.cc:304] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:13.199973 21311 raft_consensus.cc:2749] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.201769 21274 leader_election.cc:290] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.206580 21312 raft_consensus.cc:515] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.207929 21174 raft_consensus.cc:2393] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:13.208353 20951 raft_consensus.cc:2468] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:13.210147 20914 leader_election.cc:304] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.210078 20989 leader_election.cc:304] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
W20260709 12:17:13.204873 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:13.209872 20845 catalog_manager.cc:5697] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.211724 21354 raft_consensus.cc:2804] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.212258 21354 raft_consensus.cc:697] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:13.212815 21325 raft_consensus.cc:3060] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.213100 21354 consensus_queue.cc:237] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.213749 21221 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.216029 21221 ts_tablet_manager.cc:1403] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.052s	user 0.032s	sys 0.016s
I20260709 12:17:13.222735 21325 raft_consensus.cc:2749] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.228542 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:13.228999 21312 leader_election.cc:290] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.232190 21219 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:13.232684 21219 ts_tablet_manager.cc:1403] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.128s	user 0.013s	sys 0.010s
I20260709 12:17:13.219679 21221 raft_consensus.cc:359] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.204246 21099 raft_consensus.cc:2468] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:13.225685 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.226263 21097 raft_consensus.cc:2468] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:13.236326 21220 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:13.236692 21221 raft_consensus.cc:385] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.243914 21221 raft_consensus.cc:740] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
W20260709 12:17:13.237979 21139 leader_election.cc:343] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.244822 21221 consensus_queue.cc:260] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.245556 21222 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:13.245616 20991 leader_election.cc:304] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.246150 21222 ts_tablet_manager.cc:1403] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.010s
I20260709 12:17:13.246981 21354 raft_consensus.cc:2804] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.247299 21221 ts_tablet_manager.cc:1434] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.008s
I20260709 12:17:13.247749 21354 raft_consensus.cc:697] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:13.248292 21221 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:13.248653 21354 consensus_queue.cc:237] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.249079 21220 ts_tablet_manager.cc:1403] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.122s	user 0.020s	sys 0.006s
I20260709 12:17:13.249094 21222 raft_consensus.cc:359] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.250011 21222 raft_consensus.cc:385] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.250169 20951 raft_consensus.cc:2468] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:13.250475 21222 raft_consensus.cc:740] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.251325 21222 consensus_queue.cc:260] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.251538 21137 leader_election.cc:304] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:13.238095 20845 catalog_manager.cc:5697] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.252728 21312 raft_consensus.cc:2804] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.252545 21220 raft_consensus.cc:359] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.253669 21312 raft_consensus.cc:697] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:13.253403 21220 raft_consensus.cc:385] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.254347 21220 raft_consensus.cc:740] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.255260 21220 consensus_queue.cc:260] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.255064 21312 consensus_queue.cc:237] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.257507 21221 tablet_bootstrap.cc:654] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.258054 21222 ts_tablet_manager.cc:1434] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:17:13.259438 21222 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:13.263511 21220 ts_tablet_manager.cc:1434] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:17:13.265409 21220 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:13.269953 21222 tablet_bootstrap.cc:654] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.271425 21221 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.271974 21221 ts_tablet_manager.cc:1403] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.009s
I20260709 12:17:13.274888 21220 tablet_bootstrap.cc:654] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.275857 21221 raft_consensus.cc:359] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.284037 21221 raft_consensus.cc:385] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.238363 21219 raft_consensus.cc:359] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.284438 21221 raft_consensus.cc:740] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.284691 21219 raft_consensus.cc:385] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.285049 21219 raft_consensus.cc:740] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.285755 21219 consensus_queue.cc:260] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.372618 21222 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:13.373369 21222 ts_tablet_manager.cc:1403] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.114s	user 0.017s	sys 0.000s
I20260709 12:17:13.311501 21226 raft_consensus.cc:493] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.373661 21221 consensus_queue.cc:260] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.374377 21226 raft_consensus.cc:515] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.375967 21221 ts_tablet_manager.cc:1434] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.104s	user 0.016s	sys 0.026s
I20260709 12:17:13.376502 21226 leader_election.cc:290] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.376966 21221 tablet_bootstrap.cc:492] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:13.330468 21274 raft_consensus.cc:493] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.377555 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.377923 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
W20260709 12:17:13.379884 20916 leader_election.cc:343] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.381891 20845 catalog_manager.cc:5697] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:13.384004 20914 leader_election.cc:343] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.287859 21219 ts_tablet_manager.cc:1434] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 12:17:13.385885 21219 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:13.329943 21097 raft_consensus.cc:3060] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.389886 21274 raft_consensus.cc:515] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.384549 20914 leader_election.cc:304] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.392082 21274 leader_election.cc:290] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.392647 21274 consensus_queue.cc:1048] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.393348 21280 raft_consensus.cc:2749] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.394129 21280 consensus_queue.cc:1048] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.394362 21226 raft_consensus.cc:493] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.394922 21226 raft_consensus.cc:515] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.395381 21221 tablet_bootstrap.cc:654] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.395749 21311 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.396248 21311 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.397081 21226 leader_election.cc:290] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.398399 21311 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.398563 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:13.399288 21025 raft_consensus.cc:2468] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:13.399755 21354 raft_consensus.cc:493] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.399987 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:13.400409 21219 tablet_bootstrap.cc:654] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.400285 21354 raft_consensus.cc:515] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.400695 21024 raft_consensus.cc:2468] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:13.402611 21354 leader_election.cc:290] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.404179 21277 raft_consensus.cc:493] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.404910 21277 raft_consensus.cc:515] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.407421 20917 leader_election.cc:304] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:13.408459 21226 raft_consensus.cc:2804] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.408838 21226 raft_consensus.cc:493] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.408980 21220 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:13.408929 21277 leader_election.cc:290] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.409825 21220 ts_tablet_manager.cc:1403] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.031s
I20260709 12:17:13.435669 21222 raft_consensus.cc:359] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.436543 21222 raft_consensus.cc:385] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.436880 21222 raft_consensus.cc:740] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.437922 21222 consensus_queue.cc:260] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.439435 21220 raft_consensus.cc:359] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.440410 21220 raft_consensus.cc:385] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.440742 21220 raft_consensus.cc:740] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.441006 21222 ts_tablet_manager.cc:1434] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 12:17:13.410461 21226 raft_consensus.cc:3060] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.442456 21222 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:13.411185 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.414068 21140 leader_election.cc:304] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:13.416126 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:13.417689 21280 consensus_queue.cc:1048] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.420378 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e531cd0b844ef99e5360fe8adabf3e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.420380 21219 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:13.448660 21219 ts_tablet_manager.cc:1403] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.004s
W20260709 12:17:13.450882 21140 leader_election.cc:343] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.451784 21219 raft_consensus.cc:359] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.452535 21219 raft_consensus.cc:385] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.421495 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
W20260709 12:17:13.455838 20991 leader_election.cc:343] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.425037 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.425937 20846 catalog_manager.cc:5697] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.427506 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.428248 21221 tablet_bootstrap.cc:492] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.458145 21221 ts_tablet_manager.cc:1403] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.006s
I20260709 12:17:13.461061 21221 raft_consensus.cc:359] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.462797 21221 raft_consensus.cc:385] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.463090 21221 raft_consensus.cc:740] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.463776 21221 consensus_queue.cc:260] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.429004 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e531cd0b844ef99e5360fe8adabf3e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.430644 21368 consensus_queue.cc:1048] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.468356 21371 consensus_queue.cc:1048] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.431421 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.447188 21274 raft_consensus.cc:2804] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.452798 21219 raft_consensus.cc:740] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
W20260709 12:17:13.480249 20989 leader_election.cc:343] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.480876 20989 leader_election.cc:304] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.481262 21219 consensus_queue.cc:260] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
W20260709 12:17:13.482784 21137 leader_election.cc:343] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.483111 21221 ts_tablet_manager.cc:1434] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 12:17:13.483345 21137 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:13.483901 21219 ts_tablet_manager.cc:1434] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:17:13.484225 21221 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:13.484880 21219 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:13.485811 21354 raft_consensus.cc:2749] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.485953 21274 raft_consensus.cc:493] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.486611 21274 raft_consensus.cc:3060] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.488742 21226 raft_consensus.cc:515] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.489327 21381 consensus_queue.cc:1048] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.455983 21220 consensus_queue.cc:260] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.490432 21373 raft_consensus.cc:493] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.490899 21373 raft_consensus.cc:515] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.491667 21220 ts_tablet_manager.cc:1434] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.001s
I20260709 12:17:13.492687 21220 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:13.501292 21221 tablet_bootstrap.cc:654] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.503306 21312 raft_consensus.cc:2749] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.503042 21274 raft_consensus.cc:515] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.509775 21222 tablet_bootstrap.cc:654] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.513159 21171 raft_consensus.cc:2393] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:13.515532 20951 raft_consensus.cc:2393] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
W20260709 12:17:13.521368 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 88517.6us
I20260709 12:17:13.521766 21274 leader_election.cc:290] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.522560 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:13.523702 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:13.524576 21025 raft_consensus.cc:3060] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.556885 21232 raft_consensus.cc:493] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.557732 21232 raft_consensus.cc:515] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.559806 21226 leader_election.cc:290] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.560235 21219 tablet_bootstrap.cc:654] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.560624 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:13.562199 21232 leader_election.cc:290] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:13.563833 21025 raft_consensus.cc:2468] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:13.563966 21373 leader_election.cc:290] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.572319 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.574612 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:13.574595 21063 leader_election.cc:304] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.577618 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:13.581143 20989 leader_election.cc:343] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.582811 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:13.584035 21173 raft_consensus.cc:2393] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:13.566439 21232 raft_consensus.cc:493] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.590749 20914 leader_election.cc:304] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.592657 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.593595 20950 raft_consensus.cc:2468] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:13.594894 21280 raft_consensus.cc:2749] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.597962 21220 tablet_bootstrap.cc:654] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:13.599020 20991 leader_election.cc:343] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.599609 20991 leader_election.cc:304] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.599915 21063 leader_election.cc:304] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:13.599124 21140 leader_election.cc:304] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:13.603058 21375 raft_consensus.cc:2804] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.603469 21375 raft_consensus.cc:493] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.603811 21375 raft_consensus.cc:3060] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.604699 21231 raft_consensus.cc:2749] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.597362 21373 raft_consensus.cc:2749] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.597469 21232 raft_consensus.cc:515] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.607941 21311 raft_consensus.cc:2804] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.620965 21222 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:13.621456 21222 ts_tablet_manager.cc:1403] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.179s	user 0.013s	sys 0.005s
I20260709 12:17:13.630448 21222 raft_consensus.cc:359] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.631644 21222 raft_consensus.cc:385] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.626144 21311 raft_consensus.cc:697] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:13.633914 21232 leader_election.cc:290] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.633744 21222 raft_consensus.cc:740] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.634974 21311 consensus_queue.cc:237] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.636695 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.637777 21222 consensus_queue.cc:260] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.633001 21026 raft_consensus.cc:2468] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:13.640815 20951 raft_consensus.cc:2468] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.650874 20989 leader_election.cc:304] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:13.651875 21354 raft_consensus.cc:2804] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.652347 21354 raft_consensus.cc:493] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.652737 21354 raft_consensus.cc:3060] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.653112 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.654767 21232 consensus_queue.cc:1048] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:17:13.661492 21221 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.666558 21231 consensus_queue.cc:1048] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.667205 21297 consensus_queue.cc:1048] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.662755 21221 ts_tablet_manager.cc:1403] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.179s	user 0.016s	sys 0.009s
I20260709 12:17:13.672197 21219 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:13.672776 21219 ts_tablet_manager.cc:1403] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.188s	user 0.021s	sys 0.002s
I20260709 12:17:13.674584 21222 ts_tablet_manager.cc:1434] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 12:17:13.675734 21222 tablet_bootstrap.cc:492] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:13.704449 21219 raft_consensus.cc:359] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.705662 21219 raft_consensus.cc:385] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.706087 21219 raft_consensus.cc:740] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.709412 21354 raft_consensus.cc:515] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.697992 21221 raft_consensus.cc:359] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.716192 21373 raft_consensus.cc:493] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.730129 21373 raft_consensus.cc:515] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.732564 21373 leader_election.cc:290] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.723191 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:13.734318 20949 raft_consensus.cc:3060] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.735221 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.738713 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae962db47c4e4fc6a10172d6395e113e" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.739615 20951 raft_consensus.cc:2393] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:13.740901 21063 leader_election.cc:304] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:13.742331 21373 raft_consensus.cc:3060] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.716048 21221 raft_consensus.cc:385] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.746240 21221 raft_consensus.cc:740] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.747592 21221 consensus_queue.cc:260] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.750947 21221 ts_tablet_manager.cc:1434] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.002s
I20260709 12:17:13.751650 21373 raft_consensus.cc:2749] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.752341 21221 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:13.765192 21220 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:13.765990 21220 ts_tablet_manager.cc:1403] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.274s	user 0.009s	sys 0.014s
I20260709 12:17:13.724749 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.723251 21354 leader_election.cc:290] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.713896 21219 consensus_queue.cc:260] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.725353 21375 raft_consensus.cc:515] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.770493 21098 raft_consensus.cc:2393] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:13.770730 20949 raft_consensus.cc:2468] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:13.772342 20989 leader_election.cc:304] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:13.776247 21219 ts_tablet_manager.cc:1434] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.103s	user 0.011s	sys 0.000s
I20260709 12:17:13.776780 21232 raft_consensus.cc:2804] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.777413 21232 raft_consensus.cc:697] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:13.778573 21232 consensus_queue.cc:237] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.787058 21221 tablet_bootstrap.cc:654] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.790804 21220 raft_consensus.cc:359] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.791756 21220 raft_consensus.cc:385] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.792125 21220 raft_consensus.cc:740] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.792846 21220 consensus_queue.cc:260] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.795079 21220 ts_tablet_manager.cc:1434] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 12:17:13.796168 21220 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:13.777329 21219 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:13.799170 21375 leader_election.cc:290] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.800720 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:13.803548 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "807ff01cdab14b62885871bdce0d3cd8" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:13.803990 21232 consensus_queue.cc:1048] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.804982 20951 raft_consensus.cc:2393] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:13.805588 21219 tablet_bootstrap.cc:654] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.806371 21063 leader_election.cc:304] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:13.810381 21222 tablet_bootstrap.cc:654] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.810137 20846 catalog_manager.cc:5697] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.814328 21220 tablet_bootstrap.cc:654] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.815133 21375 raft_consensus.cc:2749] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.818547 21354 consensus_queue.cc:1048] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.818364 21311 consensus_queue.cc:1048] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.825274 21219 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:13.825340 21221 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.826045 21219 ts_tablet_manager.cc:1403] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.005s
I20260709 12:17:13.826119 21221 ts_tablet_manager.cc:1403] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.074s	user 0.017s	sys 0.000s
I20260709 12:17:13.829066 21219 raft_consensus.cc:359] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.829872 21219 raft_consensus.cc:385] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.830358 21219 raft_consensus.cc:740] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.829764 21221 raft_consensus.cc:359] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.830780 21221 raft_consensus.cc:385] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.831069 21221 raft_consensus.cc:740] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.831175 21219 consensus_queue.cc:260] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.831769 21221 consensus_queue.cc:260] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.833666 21221 ts_tablet_manager.cc:1434] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:13.836163 21219 ts_tablet_manager.cc:1434] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:17:13.837205 21219 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:13.847398 21219 tablet_bootstrap.cc:654] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.849898 21231 consensus_queue.cc:1048] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:13.852720 21372 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.853291 21372 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.855696 21372 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.858510 21311 raft_consensus.cc:493] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.859193 21311 raft_consensus.cc:515] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.862010 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:13.862810 21173 raft_consensus.cc:2468] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:13.864885 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:13.868062 21381 raft_consensus.cc:493] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.868718 21381 raft_consensus.cc:515] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:13.871073 21381 leader_election.cc:290] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.872498 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.873188 20914 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:13.875344 21221 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
W20260709 12:17:13.875465 20917 leader_election.cc:343] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:13.877074 21139 leader_election.cc:343] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.878786 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.879585 21098 raft_consensus.cc:2468] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:13.861589 21311 leader_election.cc:290] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.880425 21274 consensus_queue.cc:1048] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:13.881819 21221 tablet_bootstrap.cc:654] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.882679 21372 raft_consensus.cc:2804] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.888849 21372 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.889328 21372 raft_consensus.cc:3060] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.889369 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.883996 21139 leader_election.cc:304] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:13.887158 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.894053 20951 raft_consensus.cc:2468] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:13.894469 21381 raft_consensus.cc:2804] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.894948 21381 raft_consensus.cc:493] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.895344 21381 raft_consensus.cc:3060] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.894646 20846 catalog_manager.cc:5697] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.895623 21137 leader_election.cc:304] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:13.896641 21311 raft_consensus.cc:2804] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.897076 21311 raft_consensus.cc:493] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.900146 21297 raft_consensus.cc:493] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.900735 21297 raft_consensus.cc:515] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.901389 21220 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:13.888849 21232 raft_consensus.cc:493] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.901973 21220 ts_tablet_manager.cc:1403] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.106s	user 0.032s	sys 0.010s
I20260709 12:17:13.902211 21232 raft_consensus.cc:515] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.903093 21297 leader_election.cc:290] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.904501 21232 leader_election.cc:290] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:13.905105 21220 raft_consensus.cc:359] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.905836 21220 raft_consensus.cc:385] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.906100 21220 raft_consensus.cc:740] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.906831 21220 consensus_queue.cc:260] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.907804 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.908710 21097 raft_consensus.cc:2468] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.890337 20949 raft_consensus.cc:2468] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:13.915143 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.915868 20951 raft_consensus.cc:2468] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:13.916620 21222 tablet_bootstrap.cc:492] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:13.917222 21222 ts_tablet_manager.cc:1403] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.242s	user 0.006s	sys 0.013s
I20260709 12:17:13.917855 20989 leader_election.cc:304] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:13.897457 21311 raft_consensus.cc:3060] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.919370 21372 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.922541 21297 raft_consensus.cc:2804] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:13.922704 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:13.923635 21222 raft_consensus.cc:359] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.924021 21173 raft_consensus.cc:3060] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.924500 21297 raft_consensus.cc:493] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.925658 21297 raft_consensus.cc:3060] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.926347 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:13.927840 21311 raft_consensus.cc:515] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:13.928300 20917 leader_election.cc:343] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.926345 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:13.930449 21221 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:13.936327 21173 raft_consensus.cc:2468] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:13.936586 21221 ts_tablet_manager.cc:1403] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.000s
I20260709 12:17:13.937564 20914 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:13.938858 21374 raft_consensus.cc:2804] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.939543 21374 raft_consensus.cc:697] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:13.940429 21374 consensus_queue.cc:237] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.940831 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:13.941494 20951 raft_consensus.cc:3060] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.926342 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:13.935802 21220 ts_tablet_manager.cc:1434] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:17:13.948406 21220 tablet_bootstrap.cc:492] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:13.948834 20951 raft_consensus.cc:2468] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:13.950062 21137 leader_election.cc:304] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:13.950366 21311 leader_election.cc:290] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.951004 21274 raft_consensus.cc:2804] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:13.951511 21274 raft_consensus.cc:697] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:13.952358 21274 consensus_queue.cc:237] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:13.953143 20991 leader_election.cc:343] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.916065 21381 raft_consensus.cc:515] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:13.957569 20989 leader_election.cc:343] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.958773 20989 leader_election.cc:304] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:13.958992 21372 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:13.960400 21381 leader_election.cc:290] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:13.968219 21220 tablet_bootstrap.cc:654] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:13.968559 21222 raft_consensus.cc:385] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.969740 21222 raft_consensus.cc:740] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.953140 21221 raft_consensus.cc:359] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.973529 21221 raft_consensus.cc:385] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:13.973946 21221 raft_consensus.cc:740] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:13.976197 21222 consensus_queue.cc:260] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:13.980536 21221 consensus_queue.cc:260] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:13.969305 21232 raft_consensus.cc:2749] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:13.962332 21297 raft_consensus.cc:515] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:13.988889 21222 ts_tablet_manager.cc:1434] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.000s
I20260709 12:17:13.989704 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:13.990027 21222 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:13.990422 20951 raft_consensus.cc:3060] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:13.992559 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.995966 21219 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
W20260709 12:17:13.996111 21139 leader_election.cc:343] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:13.996661 21219 ts_tablet_manager.cc:1403] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.160s	user 0.018s	sys 0.005s
I20260709 12:17:13.999922 21219 raft_consensus.cc:359] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.002243 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:14.002681 21219 raft_consensus.cc:385] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.002985 20949 raft_consensus.cc:3060] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.003093 21219 raft_consensus.cc:740] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.004832 21297 leader_election.cc:290] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.002331 21221 ts_tablet_manager.cc:1434] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 12:17:14.004102 21219 consensus_queue.cc:260] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.006091 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:13.979511 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:14.007856 21221 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:14.004369 20847 catalog_manager.cc:5697] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.008574 21099 raft_consensus.cc:3060] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.013664 21219 ts_tablet_manager.cc:1434] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 12:17:14.014750 21219 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:14.019137 21099 raft_consensus.cc:2468] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:14.020586 20991 leader_election.cc:304] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:14.021528 21297 raft_consensus.cc:2804] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.021790 21219 tablet_bootstrap.cc:654] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.022157 21297 raft_consensus.cc:697] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:14.023121 21297 consensus_queue.cc:237] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.011050 21098 raft_consensus.cc:3060] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.028494 21221 tablet_bootstrap.cc:654] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.038411 20949 raft_consensus.cc:2468] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:14.043578 20847 catalog_manager.cc:5697] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.050964 20951 raft_consensus.cc:2468] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:14.054041 21137 leader_election.cc:304] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:14.056811 21274 raft_consensus.cc:2804] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.057572 21274 raft_consensus.cc:697] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:14.058828 21274 consensus_queue.cc:237] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.061254 21098 raft_consensus.cc:2468] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:14.066906 21222 tablet_bootstrap.cc:654] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.067166 21220 tablet_bootstrap.cc:492] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:14.067729 21220 ts_tablet_manager.cc:1403] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.120s	user 0.009s	sys 0.003s
I20260709 12:17:14.068836 21372 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.070029 20846 catalog_manager.cc:5697] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.071372 21219 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:14.071964 21219 ts_tablet_manager.cc:1403] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.057s	user 0.007s	sys 0.012s
I20260709 12:17:14.073350 21221 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:14.073915 21221 ts_tablet_manager.cc:1403] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.066s	user 0.037s	sys 0.004s
I20260709 12:17:14.076007 21221 raft_consensus.cc:359] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.076491 21221 raft_consensus.cc:385] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.069404 21372 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.076928 21221 raft_consensus.cc:740] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.077927 21220 raft_consensus.cc:359] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.080231 21220 raft_consensus.cc:385] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.080611 21220 raft_consensus.cc:740] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.079425 21221 consensus_queue.cc:260] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.081570 21220 consensus_queue.cc:260] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.083575 21219 raft_consensus.cc:359] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.084650 21219 raft_consensus.cc:385] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.085003 21219 raft_consensus.cc:740] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.085809 21219 consensus_queue.cc:260] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.087801 21219 ts_tablet_manager.cc:1434] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:17:14.090950 21372 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:14.093493 21219 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:14.093694 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
W20260709 12:17:14.096095 20914 leader_election.cc:343] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.098313 21375 raft_consensus.cc:493] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.099001 21375 raft_consensus.cc:515] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.101270 21219 tablet_bootstrap.cc:654] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.102415 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e531cd0b844ef99e5360fe8adabf3e" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:14.103214 21025 raft_consensus.cc:2468] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:14.104583 21375 leader_election.cc:290] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:14.106376 21066 leader_election.cc:304] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:14.108206 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:14.109037 21026 raft_consensus.cc:2468] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:14.110044 21221 ts_tablet_manager.cc:1434] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:17:14.110786 20917 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:14.111217 21221 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:14.112305 21372 raft_consensus.cc:2804] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.112723 21372 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.112884 21219 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:14.113147 21372 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.113498 21222 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:14.114121 21222 ts_tablet_manager.cc:1403] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.124s	user 0.011s	sys 0.001s
I20260709 12:17:14.114632 21220 ts_tablet_manager.cc:1434] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 12:17:14.115695 21220 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:14.117200 21222 raft_consensus.cc:359] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.117998 21222 raft_consensus.cc:385] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.118292 21222 raft_consensus.cc:740] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.118197 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e531cd0b844ef99e5360fe8adabf3e" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:14.119029 21221 tablet_bootstrap.cc:654] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.119220 21174 raft_consensus.cc:2468] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:14.119526 21222 consensus_queue.cc:260] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.113498 21219 ts_tablet_manager.cc:1403] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260709 12:17:14.122277 21222 ts_tablet_manager.cc:1434] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:14.123349 21222 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:14.124840 21219 raft_consensus.cc:359] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.125600 21219 raft_consensus.cc:385] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.125912 21219 raft_consensus.cc:740] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.126583 20848 catalog_manager.cc:5697] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.129805 21274 consensus_queue.cc:1048] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.133752 21222 tablet_bootstrap.cc:654] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.130416 21219 consensus_queue.cc:260] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.139631 21219 ts_tablet_manager.cc:1434] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.004s
I20260709 12:17:14.140844 21219 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:14.137296 21375 raft_consensus.cc:2804] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.143661 21220 tablet_bootstrap.cc:654] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.143932 21375 raft_consensus.cc:493] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.148283 21375 raft_consensus.cc:3060] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.147053 21222 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:14.152074 21222 ts_tablet_manager.cc:1403] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.009s
I20260709 12:17:14.155539 21222 raft_consensus.cc:359] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.156381 21222 raft_consensus.cc:385] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.156689 21222 raft_consensus.cc:740] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.157390 21222 consensus_queue.cc:260] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.159166 21312 consensus_queue.cc:1048] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.160302 21222 ts_tablet_manager.cc:1434] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:14.161383 21222 tablet_bootstrap.cc:492] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:14.168658 21222 tablet_bootstrap.cc:654] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.168836 21219 tablet_bootstrap.cc:654] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.174614 21221 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:14.175361 21221 ts_tablet_manager.cc:1403] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.064s	user 0.034s	sys 0.003s
I20260709 12:17:14.180928 21222 tablet_bootstrap.cc:492] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:14.181530 21222 ts_tablet_manager.cc:1403] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260709 12:17:14.184522 21222 raft_consensus.cc:359] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.185272 21222 raft_consensus.cc:385] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.185652 21222 raft_consensus.cc:740] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.185405 21372 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.186707 21222 consensus_queue.cc:260] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.188956 21222 ts_tablet_manager.cc:1434] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:14.190001 21222 tablet_bootstrap.cc:492] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:14.191754 21220 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:14.193871 21220 ts_tablet_manager.cc:1403] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.000s
I20260709 12:17:14.196514 21222 tablet_bootstrap.cc:654] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.201156 21375 raft_consensus.cc:515] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.205291 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e531cd0b844ef99e5360fe8adabf3e" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:14.206079 21026 raft_consensus.cc:3060] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.201048 21220 raft_consensus.cc:359] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.206856 21220 raft_consensus.cc:385] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.207195 21220 raft_consensus.cc:740] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.208742 21220 consensus_queue.cc:260] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.211208 21220 ts_tablet_manager.cc:1434] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:17:14.212235 21220 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:14.221117 21222 tablet_bootstrap.cc:492] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:14.221757 21222 ts_tablet_manager.cc:1403] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.007s
I20260709 12:17:14.221801 21312 raft_consensus.cc:493] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.222465 21312 raft_consensus.cc:515] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.224937 21222 raft_consensus.cc:359] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.225708 21222 raft_consensus.cc:385] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.225981 21222 raft_consensus.cc:740] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.226711 21222 consensus_queue.cc:260] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.228330 21220 tablet_bootstrap.cc:654] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.228952 21222 ts_tablet_manager.cc:1434] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:17:14.230149 21222 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:14.234544 21232 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.235313 21232 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.236584 21222 tablet_bootstrap.cc:654] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.229203 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:14.242517 21139 leader_election.cc:343] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.244949 21026 raft_consensus.cc:2468] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:14.247658 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:14.248636 21025 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.251278 21222 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:14.251817 21222 ts_tablet_manager.cc:1403] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.005s
I20260709 12:17:14.252729 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:14.253777 21024 raft_consensus.cc:2468] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:14.255057 21222 raft_consensus.cc:359] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.255805 21222 raft_consensus.cc:385] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.256078 21222 raft_consensus.cc:740] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.256826 21222 consensus_queue.cc:260] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.257121 21025 raft_consensus.cc:2468] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:14.258867 20917 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:14.259356 21222 ts_tablet_manager.cc:1434] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:14.261304 21226 raft_consensus.cc:2804] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.262180 21226 raft_consensus.cc:697] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:14.263463 21226 consensus_queue.cc:237] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.269169 21311 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.269788 21372 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:14.270035 21311 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.265060 21140 leader_election.cc:304] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:14.275111 21381 raft_consensus.cc:2804] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.275754 21381 raft_consensus.cc:493] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.276237 21381 raft_consensus.cc:3060] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.278406 21221 raft_consensus.cc:359] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.279523 21221 raft_consensus.cc:385] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.279906 21221 raft_consensus.cc:740] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.281865 21297 consensus_queue.cc:1048] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.286494 21066 leader_election.cc:304] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:14.287910 21373 raft_consensus.cc:2804] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.289371 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:14.290130 21174 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.295727 21232 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:14.296408 21312 leader_election.cc:290] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.302130 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:14.303557 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:14.300385 21311 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:14.313845 21231 consensus_queue.cc:1048] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:14.304476 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:14.323346 21026 raft_consensus.cc:2393] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:14.328375 21140 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:14.330703 21226 raft_consensus.cc:493] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.331372 21226 raft_consensus.cc:515] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.335536 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:14.336638 21226 leader_election.cc:290] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
W20260709 12:17:14.337774 20916 leader_election.cc:343] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:14.339540 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:14.315158 21375 leader_election.cc:290] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:14.353261 21312 consensus_queue.cc:1048] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.358039 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:14.358626 21222 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:14.358821 21172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:14.358968 21170 raft_consensus.cc:2468] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:14.359737 21172 raft_consensus.cc:2393] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:14.360585 20914 leader_election.cc:304] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:14.314535 21372 raft_consensus.cc:493] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.358183 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e531cd0b844ef99e5360fe8adabf3e" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:14.362581 21374 raft_consensus.cc:2804] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.367504 21171 raft_consensus.cc:3060] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.367858 21374 raft_consensus.cc:493] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.368323 21374 raft_consensus.cc:3060] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.364256 21219 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:14.370086 21219 ts_tablet_manager.cc:1403] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.230s	user 0.013s	sys 0.000s
I20260709 12:17:14.371148 21372 raft_consensus.cc:515] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.380491 21372 leader_election.cc:290] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.361092 20989 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:14.379597 21219 raft_consensus.cc:359] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.280872 21221 consensus_queue.cc:260] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.383311 21381 raft_consensus.cc:515] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.416666 21221 ts_tablet_manager.cc:1434] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.241s	user 0.011s	sys 0.000s
I20260709 12:17:14.387569 21390 raft_consensus.cc:3060] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.384155 21171 raft_consensus.cc:2468] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:14.418794 21381 leader_election.cc:290] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.420331 21222 tablet_bootstrap.cc:654] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.392773 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:14.423597 21026 raft_consensus.cc:2468] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:14.424896 20917 leader_election.cc:304] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:14.386945 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:14.394446 21219 raft_consensus.cc:385] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.426795 21219 raft_consensus.cc:740] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.427645 21219 consensus_queue.cc:260] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.428699 21432 raft_consensus.cc:2804] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.429131 21432 raft_consensus.cc:493] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.429478 21432 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.431818 21221 tablet_bootstrap.cc:492] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
W20260709 12:17:14.431969 20916 leader_election.cc:343] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.413126 21373 raft_consensus.cc:697] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:14.420831 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:14.421974 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:14.437335 21025 raft_consensus.cc:3060] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:14.438467 21139 leader_election.cc:343] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.437247 21373 consensus_queue.cc:237] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.445294 21221 tablet_bootstrap.cc:654] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.415899 21220 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:14.446462 21220 ts_tablet_manager.cc:1403] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.234s	user 0.083s	sys 0.058s
I20260709 12:17:14.450631 21220 raft_consensus.cc:359] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.451673 21220 raft_consensus.cc:385] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.452086 21220 raft_consensus.cc:740] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.454985 21220 consensus_queue.cc:260] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.460078 21220 ts_tablet_manager.cc:1434] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 12:17:14.461798 21220 tablet_bootstrap.cc:492] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:14.465193 20846 catalog_manager.cc:5697] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.471004 21297 consensus_queue.cc:1048] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.435717 21390 raft_consensus.cc:2749] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.413030 21100 raft_consensus.cc:3060] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.486554 21025 raft_consensus.cc:2468] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:14.479429 21226 consensus_queue.cc:1048] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:17:14.390895 21174 raft_consensus.cc:2468] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:14.489782 21311 raft_consensus.cc:2749] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.470816 21220 tablet_bootstrap.cc:654] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.435153 21219 ts_tablet_manager.cc:1434] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 12:17:14.479552 21381 raft_consensus.cc:493] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.527415 21381 raft_consensus.cc:515] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.490911 21432 raft_consensus.cc:515] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.528590 21390 consensus_queue.cc:1048] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:14.484530 21274 consensus_queue.cc:1048] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 12:17:14.491865 21311 consensus_queue.cc:1048] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 12:17:14.532325 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:14.533944 21220 tablet_bootstrap.cc:492] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:14.534474 21220 ts_tablet_manager.cc:1403] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.000s
I20260709 12:17:14.534839 21311 raft_consensus.cc:493] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.535053 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:14.535790 21024 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.535450 21311 raft_consensus.cc:515] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.537358 21220 raft_consensus.cc:359] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.538158 21381 leader_election.cc:290] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:14.540583 21311 leader_election.cc:290] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.527863 21219 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:14.541186 21137 leader_election.cc:343] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.542725 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:14.500586 21372 consensus_queue.cc:1048] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 12:17:14.544322 21311 consensus_queue.cc:1048] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:14.397119 21374 raft_consensus.cc:515] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.538086 21220 raft_consensus.cc:385] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.547323 21220 raft_consensus.cc:740] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.548852 21374 leader_election.cc:290] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:14.516330 21140 leader_election.cc:304] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:14.552363 21220 consensus_queue.cc:260] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.553759 21099 raft_consensus.cc:2468] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:14.557188 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
W20260709 12:17:14.559284 21140 leader_election.cc:343] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.560698 21139 leader_election.cc:304] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:14.539939 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:14.562032 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:14.564476 21433 raft_consensus.cc:2804] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:17:14.564597 20916 leader_election.cc:343] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.565310 21433 raft_consensus.cc:697] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:14.566715 21024 raft_consensus.cc:2468] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:14.566370 21433 consensus_queue.cc:237] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.571237 21220 ts_tablet_manager.cc:1434] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:17:14.572629 21220 tablet_bootstrap.cc:492] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:14.568418 21312 raft_consensus.cc:2804] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.574255 21312 raft_consensus.cc:493] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.574682 21312 raft_consensus.cc:3060] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.574545 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:14.575373 21171 raft_consensus.cc:3060] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.577343 21219 tablet_bootstrap.cc:654] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:14.582116 21139 leader_election.cc:343] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.582695 21139 leader_election.cc:304] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:14.584133 21381 raft_consensus.cc:2749] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.588090 21435 raft_consensus.cc:493] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.588752 21435 raft_consensus.cc:515] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.593416 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:14.594220 20951 raft_consensus.cc:2468] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:14.597162 20989 leader_election.cc:304] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:14.598302 21231 raft_consensus.cc:2804] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:14.598729 21231 raft_consensus.cc:493] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.599082 21231 raft_consensus.cc:3060] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.600536 21219 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:14.601284 21219 ts_tablet_manager.cc:1403] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.001s
I20260709 12:17:14.603616 21171 raft_consensus.cc:2468] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:14.604370 21220 tablet_bootstrap.cc:654] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.499555 20847 catalog_manager.cc:5697] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.585021 21312 raft_consensus.cc:515] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.631878 20914 leader_election.cc:304] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:14.632480 20917 leader_election.cc:304] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:14.633599 21221 tablet_bootstrap.cc:492] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:14.644781 21221 ts_tablet_manager.cc:1403] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.213s	user 0.031s	sys 0.031s
I20260709 12:17:14.647351 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:14.648416 21098 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.651865 21374 raft_consensus.cc:2804] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.652493 21374 raft_consensus.cc:697] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:14.653445 21374 consensus_queue.cc:237] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.654676 21372 raft_consensus.cc:2804] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.655248 21372 raft_consensus.cc:697] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:14.656209 21372 consensus_queue.cc:237] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.657837 21219 raft_consensus.cc:359] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.659137 21219 raft_consensus.cc:385] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.613672 21231 raft_consensus.cc:515] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.659523 21219 raft_consensus.cc:740] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.666901 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:14.667646 20951 raft_consensus.cc:3060] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:14.670063 21219 consensus_queue.cc:260] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.673590 21219 ts_tablet_manager.cc:1434] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.072s	user 0.012s	sys 0.000s
I20260709 12:17:14.669345 21221 raft_consensus.cc:359] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.678882 21221 raft_consensus.cc:385] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.680393 21221 raft_consensus.cc:740] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.625164 21432 leader_election.cc:290] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.677520 21312 leader_election.cc:290] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.683200 21219 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:14.682813 21221 consensus_queue.cc:260] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.688768 21221 ts_tablet_manager.cc:1434] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.001s
I20260709 12:17:14.680397 21231 leader_election.cc:290] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.684330 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:14.686837 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:14.690483 21435 leader_election.cc:290] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:14.698825 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:14.699620 21097 raft_consensus.cc:3060] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:14.706277 21140 leader_election.cc:343] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.708360 21219 tablet_bootstrap.cc:654] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:14.709499 20991 leader_election.cc:343] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.711236 21221 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:14.711812 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:14.718585 20991 leader_election.cc:343] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.724345 21221 tablet_bootstrap.cc:654] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.726429 21222 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:14.727020 21222 ts_tablet_manager.cc:1403] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.369s	user 0.051s	sys 0.089s
I20260709 12:17:14.727058 21390 raft_consensus.cc:493] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.727661 21390 raft_consensus.cc:515] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.735983 21222 raft_consensus.cc:359] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.736120 21097 raft_consensus.cc:2468] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:14.736939 21222 raft_consensus.cc:385] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.737282 21222 raft_consensus.cc:740] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.737985 21222 consensus_queue.cc:260] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.740331 21390 leader_election.cc:290] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:14.743253 21226 consensus_queue.cc:1048] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:14.743971 21222 ts_tablet_manager.cc:1434] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 12:17:14.745062 21222 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:14.747910 21220 tablet_bootstrap.cc:492] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:14.748519 21220 ts_tablet_manager.cc:1403] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.176s	user 0.013s	sys 0.016s
I20260709 12:17:14.736717 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ad3b0b41544f61bdc6f80d66c26adf" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:14.738080 21139 leader_election.cc:304] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:14.753065 21220 raft_consensus.cc:359] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.753959 21381 raft_consensus.cc:2804] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.754788 21381 raft_consensus.cc:697] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:14.755713 21381 consensus_queue.cc:237] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.756317 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ad3b0b41544f61bdc6f80d66c26adf" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:14.753847 21220 raft_consensus.cc:385] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.758738 21220 raft_consensus.cc:740] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.764868 21220 consensus_queue.cc:260] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:14.766181 21219 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:14.766781 21219 ts_tablet_manager.cc:1403] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.006s
W20260709 12:17:14.754168 20991 leader_election.cc:343] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.774175 21220 ts_tablet_manager.cc:1434] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 12:17:14.775316 21220 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
W20260709 12:17:14.777771 20989 leader_election.cc:343] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:14.781473 21220 tablet_bootstrap.cc:654] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.783841 21098 raft_consensus.cc:2468] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:14.776827 20845 catalog_manager.cc:5697] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:14.793318 20951 raft_consensus.cc:2468] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:14.795264 21219 raft_consensus.cc:359] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.803613 21219 raft_consensus.cc:385] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.804184 21219 raft_consensus.cc:740] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.807909 21219 consensus_queue.cc:260] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.825373 21219 ts_tablet_manager.cc:1434] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.002s
I20260709 12:17:14.802399 20989 leader_election.cc:304] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:14.829257 21219 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:14.827426 21374 consensus_queue.cc:1048] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:17:14.839641 21390 raft_consensus.cc:2749] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:14.848917 21222 tablet_bootstrap.cc:654] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.853224 21221 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:14.854012 21221 ts_tablet_manager.cc:1403] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.143s	user 0.027s	sys 0.039s
I20260709 12:17:14.855403 21219 tablet_bootstrap.cc:654] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:14.842672 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 83222.4us
I20260709 12:17:14.873879 21220 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:14.874627 21220 ts_tablet_manager.cc:1403] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.100s	user 0.020s	sys 0.041s
I20260709 12:17:14.882194 21220 raft_consensus.cc:359] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.883090 21220 raft_consensus.cc:385] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.883496 21220 raft_consensus.cc:740] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.884732 21221 raft_consensus.cc:359] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.884505 21220 consensus_queue.cc:260] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.885488 21221 raft_consensus.cc:385] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.886121 21221 raft_consensus.cc:740] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.886672 21221 consensus_queue.cc:260] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.890045 21221 ts_tablet_manager.cc:1434] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 12:17:14.892764 21220 ts_tablet_manager.cc:1434] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:17:14.894572 21220 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:14.899097 20989 leader_election.cc:304] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:14.900107 21390 raft_consensus.cc:2804] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:14.900665 21390 raft_consensus.cc:697] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:14.901901 21390 consensus_queue.cc:237] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:14.905243 21220 tablet_bootstrap.cc:654] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.919375 21221 tablet_bootstrap.cc:492] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:14.922564 21222 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:14.923311 21222 ts_tablet_manager.cc:1403] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.178s	user 0.021s	sys 0.008s
I20260709 12:17:14.940376 20846 catalog_manager.cc:5697] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:14.918679 21373 consensus_queue.cc:1048] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:17:14.940832 21222 raft_consensus.cc:359] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.949831 21222 raft_consensus.cc:385] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:14.950263 21222 raft_consensus.cc:740] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:14.951313 21222 consensus_queue.cc:260] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:14.954759 21222 ts_tablet_manager.cc:1434] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:17:14.955749 21221 tablet_bootstrap.cc:654] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.942423 20846 catalog_manager.cc:5697] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.960701 21222 tablet_bootstrap.cc:492] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:14.969501 21222 tablet_bootstrap.cc:654] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:14.995159 21221 tablet_bootstrap.cc:492] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:14.995419 21220 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:14.995740 21221 ts_tablet_manager.cc:1403] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.004s
I20260709 12:17:14.995958 21220 ts_tablet_manager.cc:1403] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.102s	user 0.037s	sys 0.037s
I20260709 12:17:14.997491 20845 catalog_manager.cc:5697] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.998986 21220 raft_consensus.cc:359] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:14.999728 21220 raft_consensus.cc:385] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.000028 21220 raft_consensus.cc:740] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.000770 21220 consensus_queue.cc:260] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.002730 21220 ts_tablet_manager.cc:1434] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:17:15.003708 21220 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:15.003413 21221 raft_consensus.cc:359] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.004105 21221 raft_consensus.cc:385] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.006925 21221 raft_consensus.cc:740] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.008558 21221 consensus_queue.cc:260] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.010749 21221 ts_tablet_manager.cc:1434] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:17:15.011740 21221 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:15.012840 21220 tablet_bootstrap.cc:654] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.013978 21372 raft_consensus.cc:493] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.014528 21372 raft_consensus.cc:515] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.016659 21372 leader_election.cc:290] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.017359 21381 raft_consensus.cc:493] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.018041 21381 raft_consensus.cc:515] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.019187 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.019910 21171 raft_consensus.cc:2468] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:15.021520 20914 leader_election.cc:304] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:15.022488 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.023345 21025 raft_consensus.cc:2468] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:15.024873 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.025715 20949 raft_consensus.cc:2468] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:15.031215 21137 leader_election.cc:304] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:15.032302 21312 raft_consensus.cc:2804] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.032765 21312 raft_consensus.cc:493] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.033164 21312 raft_consensus.cc:3060] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.036067 21375 consensus_queue.cc:1048] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.037997 21219 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:15.038503 21219 ts_tablet_manager.cc:1403] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.210s	user 0.015s	sys 0.010s
I20260709 12:17:15.040400 21372 raft_consensus.cc:2804] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.040767 21372 raft_consensus.cc:493] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.041091 21372 raft_consensus.cc:3060] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.042945 20847 catalog_manager.cc:5697] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:15.056713 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.041306 21219 raft_consensus.cc:359] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.057930 21024 raft_consensus.cc:2468] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:15.059501 21375 raft_consensus.cc:493] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.060565 21435 raft_consensus.cc:493] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.060221 21375 raft_consensus.cc:515] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.061097 21435 raft_consensus.cc:515] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.064567 21222 tablet_bootstrap.cc:492] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:15.064658 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:15.064908 21435 leader_election.cc:290] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.065332 21222 ts_tablet_manager.cc:1403] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.105s	user 0.026s	sys 0.009s
I20260709 12:17:15.065696 21100 raft_consensus.cc:2393] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:15.058290 21381 leader_election.cc:290] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.072129 21222 raft_consensus.cc:359] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.068810 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.072880 21222 raft_consensus.cc:385] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.074193 21222 raft_consensus.cc:740] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.074556 21381 raft_consensus.cc:493] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.075151 21222 consensus_queue.cc:260] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.076265 20989 leader_election.cc:304] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:15.075150 21381 raft_consensus.cc:515] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.077734 21390 raft_consensus.cc:3060] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.079536 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.080236 21381 leader_election.cc:290] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.080905 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
W20260709 12:17:15.081203 21137 leader_election.cc:343] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.082376 21222 ts_tablet_manager.cc:1434] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
W20260709 12:17:15.082537 21140 leader_election.cc:343] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.083073 21140 leader_election.cc:304] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:15.083465 21222 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:15.084932 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.089536 21312 raft_consensus.cc:515] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.090447 21222 tablet_bootstrap.cc:654] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.090140 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.091337 21024 raft_consensus.cc:2393] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:15.092576 21066 leader_election.cc:304] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989
I20260709 12:17:15.092967 21450 raft_consensus.cc:493] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.093549 21450 raft_consensus.cc:515] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.094188 21427 raft_consensus.cc:3060] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.095726 21220 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:15.096246 21220 ts_tablet_manager.cc:1403] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.093s	user 0.009s	sys 0.025s
I20260709 12:17:15.096446 21450 leader_election.cc:290] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:15.097680 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:15.098408 21375 leader_election.cc:290] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.099236 21220 raft_consensus.cc:359] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.099929 21220 raft_consensus.cc:385] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.099952 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.100320 21220 raft_consensus.cc:740] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.101084 21220 consensus_queue.cc:260] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:15.101677 20917 leader_election.cc:343] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.101950 21274 raft_consensus.cc:2749] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.101527 21221 tablet_bootstrap.cc:654] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.102662 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:15.104039 20916 leader_election.cc:343] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.104483 20916 leader_election.cc:304] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:15.106381 21450 raft_consensus.cc:2749] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.107064 21220 ts_tablet_manager.cc:1434] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:17:15.108089 21220 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:15.108687 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:15.109337 21026 raft_consensus.cc:3060] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.116106 21390 raft_consensus.cc:2749] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.118563 21220 tablet_bootstrap.cc:654] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.120568 21312 leader_election.cc:290] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.122709 21372 raft_consensus.cc:515] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.057965 21219 raft_consensus.cc:385] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.127709 21219 raft_consensus.cc:740] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.129369 20949 raft_consensus.cc:2393] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:15.134657 21219 consensus_queue.cc:260] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.138545 21026 raft_consensus.cc:2468] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:15.140089 21140 leader_election.cc:304] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:15.141178 21312 raft_consensus.cc:2804] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.141907 21312 raft_consensus.cc:697] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:15.143217 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:15.142946 21312 consensus_queue.cc:237] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.137050 21427 raft_consensus.cc:2749] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.156437 21100 raft_consensus.cc:3060] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.156477 21219 ts_tablet_manager.cc:1434] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.118s	user 0.007s	sys 0.000s
I20260709 12:17:15.158787 21450 consensus_queue.cc:1048] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.162665 21222 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:15.163302 21222 ts_tablet_manager.cc:1403] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.010s
I20260709 12:17:15.165485 21219 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:15.166785 21372 leader_election.cc:290] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.168581 21221 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:15.169119 21221 ts_tablet_manager.cc:1403] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.158s	user 0.010s	sys 0.004s
I20260709 12:17:15.163017 21274 consensus_queue.cc:1048] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:17:15.179872 21221 raft_consensus.cc:359] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.180668 21221 raft_consensus.cc:385] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.180370 21222 raft_consensus.cc:359] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.181097 21222 raft_consensus.cc:385] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.181731 21375 raft_consensus.cc:493] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.182353 21375 raft_consensus.cc:515] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.184849 21375 leader_election.cc:290] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:15.181372 21222 raft_consensus.cc:740] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.194073 21220 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:15.194793 21220 ts_tablet_manager.cc:1403] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.087s	user 0.015s	sys 0.018s
I20260709 12:17:15.194636 21222 consensus_queue.cc:260] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.185506 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ddbde7cec2484da93f2b7075232308" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:15.188448 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.198789 21220 raft_consensus.cc:359] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.188483 21450 consensus_queue.cc:1048] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:15.180965 21221 raft_consensus.cc:740] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
W20260709 12:17:15.200369 21063 leader_election.cc:343] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.208439 21221 consensus_queue.cc:260] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.202385 21433 consensus_queue.cc:1048] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:17:15.211230 21374 consensus_queue.cc:1048] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.204003 21222 ts_tablet_manager.cc:1434] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 12:17:15.214344 21222 tablet_bootstrap.cc:492] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:15.199640 21220 raft_consensus.cc:385] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.214839 21220 raft_consensus.cc:740] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.190009 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.217903 21171 raft_consensus.cc:2468] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:15.219219 21063 leader_election.cc:304] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:15.220247 21375 raft_consensus.cc:2804] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.220625 21375 raft_consensus.cc:493] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.221021 21222 tablet_bootstrap.cc:654] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.221029 21375 raft_consensus.cc:3060] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.200191 21026 raft_consensus.cc:2393] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:15.227279 20917 leader_election.cc:304] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989
I20260709 12:17:15.229097 21372 raft_consensus.cc:2749] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.217515 21220 consensus_queue.cc:260] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.230928 21373 raft_consensus.cc:493] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.231722 21220 ts_tablet_manager.cc:1434] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 12:17:15.231477 21373 raft_consensus.cc:515] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.235576 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.235668 21373 leader_election.cc:290] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.236833 21219 tablet_bootstrap.cc:654] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.236932 20949 raft_consensus.cc:2393] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:15.242726 21220 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:15.229899 20848 catalog_manager.cc:5697] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.214249 21221 ts_tablet_manager.cc:1434] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 12:17:15.249424 21221 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:15.235834 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.238183 21375 raft_consensus.cc:515] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.251693 21066 leader_election.cc:304] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:15.253377 21375 leader_election.cc:290] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:15.253410 21469 raft_consensus.cc:3060] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.262022 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:15.265841 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:15.271062 21372 consensus_queue.cc:1048] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.271782 21171 raft_consensus.cc:3060] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:15.271353 21063 leader_election.cc:343] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.287042 21220 tablet_bootstrap.cc:654] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.288439 21381 consensus_queue.cc:1048] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:15.290999 21219 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:15.291556 21219 ts_tablet_manager.cc:1403] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.128s	user 0.009s	sys 0.004s
I20260709 12:17:15.291738 21469 raft_consensus.cc:2749] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.299250 21219 raft_consensus.cc:359] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.300259 21219 raft_consensus.cc:385] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.300591 21219 raft_consensus.cc:740] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.304766 21222 tablet_bootstrap.cc:492] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:15.305518 21222 ts_tablet_manager.cc:1403] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.091s	user 0.029s	sys 0.014s
I20260709 12:17:15.307346 21221 tablet_bootstrap.cc:654] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.301476 21219 consensus_queue.cc:260] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.315320 21435 consensus_queue.cc:1048] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.323354 21222 raft_consensus.cc:359] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.324379 21222 raft_consensus.cc:385] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.324733 21222 raft_consensus.cc:740] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.325590 21219 ts_tablet_manager.cc:1434] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 12:17:15.327060 21219 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:15.335317 21222 consensus_queue.cc:260] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.339185 21221 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:15.344301 21171 raft_consensus.cc:2468] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:15.356141 21221 ts_tablet_manager.cc:1403] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.005s
I20260709 12:17:15.361526 21482 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.363582 21482 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.366923 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:15.367093 21482 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.367623 21100 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:15.368225 21063 leader_election.cc:304] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:15.369130 21220 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:15.369766 21220 ts_tablet_manager.cc:1403] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.127s	user 0.044s	sys 0.030s
I20260709 12:17:15.372680 21220 raft_consensus.cc:359] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.373369 21220 raft_consensus.cc:385] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.373703 21220 raft_consensus.cc:740] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.373867 21433 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.374399 21433 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.374666 21220 consensus_queue.cc:260] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.374851 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.375607 21174 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:15.376595 21433 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:15.377758 21311 raft_consensus.cc:493] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.378196 21274 consensus_queue.cc:1048] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:15.378252 21311 raft_consensus.cc:515] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.380383 21220 ts_tablet_manager.cc:1434] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:17:15.380726 21311 leader_election.cc:290] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:15.381418 21220 tablet_bootstrap.cc:492] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:15.381796 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:15.382473 21096 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:15.382997 21025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.384649 20914 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
W20260709 12:17:15.385250 21140 leader_election.cc:343] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.385577 21374 raft_consensus.cc:2804] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.386061 21374 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.386407 21374 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.388998 21222 ts_tablet_manager.cc:1434] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.001s
I20260709 12:17:15.389442 21220 tablet_bootstrap.cc:654] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.389654 21221 raft_consensus.cc:359] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.390409 21221 raft_consensus.cc:385] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.391769 21221 raft_consensus.cc:740] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.391450 21222 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:15.397990 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.401371 21220 tablet_bootstrap.cc:492] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:15.401949 21220 ts_tablet_manager.cc:1403] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.006s
I20260709 12:17:15.403142 21219 tablet_bootstrap.cc:654] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.407147 21139 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:15.375473 21469 raft_consensus.cc:2804] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.394624 21221 consensus_queue.cc:260] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.408049 21469 raft_consensus.cc:697] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:15.408851 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:15.408169 21433 raft_consensus.cc:2804] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.409970 21433 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.410367 21433 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.409385 21469 consensus_queue.cc:237] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.415159 21390 consensus_queue.cc:1048] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.413595 21220 raft_consensus.cc:359] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.417447 21220 raft_consensus.cc:385] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.417783 21220 raft_consensus.cc:740] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.418443 21221 ts_tablet_manager.cc:1434] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:17:15.419772 21221 tablet_bootstrap.cc:492] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:15.418499 21220 consensus_queue.cc:260] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.430258 21373 raft_consensus.cc:493] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.430966 21373 raft_consensus.cc:515] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.432025 21374 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.436316 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ad3b0b41544f61bdc6f80d66c26adf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.437044 20951 raft_consensus.cc:2393] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
W20260709 12:17:15.425365 21139 leader_election.cc:343] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.438462 21219 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:15.439088 21219 ts_tablet_manager.cc:1403] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.000s
I20260709 12:17:15.443470 21374 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.444871 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:15.435262 21220 ts_tablet_manager.cc:1434] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:17:15.437089 21026 raft_consensus.cc:2468] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:15.447438 21066 leader_election.cc:304] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:15.448385 21469 raft_consensus.cc:2804] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.445815 21096 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.448817 21220 tablet_bootstrap.cc:492] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:15.435894 21373 leader_election.cc:290] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.448832 21469 raft_consensus.cc:493] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.451215 21469 raft_consensus.cc:3060] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.455044 21222 tablet_bootstrap.cc:654] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.438052 21139 leader_election.cc:304] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:15.459457 21312 raft_consensus.cc:2749] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.460300 21221 tablet_bootstrap.cc:654] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.465894 21469 raft_consensus.cc:515] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.469019 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ad3b0b41544f61bdc6f80d66c26adf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:15.469692 21026 raft_consensus.cc:3060] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.470433 21469 leader_election.cc:290] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.471186 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ad3b0b41544f61bdc6f80d66c26adf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.471933 21171 raft_consensus.cc:2468] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:15.472790 21219 raft_consensus.cc:359] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.473299 21219 raft_consensus.cc:385] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.473559 21219 raft_consensus.cc:740] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.474334 21219 consensus_queue.cc:260] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.476660 21219 ts_tablet_manager.cc:1434] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.000s
I20260709 12:17:15.477584 21219 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:15.478662 21220 tablet_bootstrap.cc:654] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.476679 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:15.482264 21096 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:15.483537 20916 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:15.484573 21450 raft_consensus.cc:2804] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.483454 21433 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.481283 21026 raft_consensus.cc:2468] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:15.490919 21171 raft_consensus.cc:2393] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:15.491277 21066 leader_election.cc:304] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:15.492250 21469 raft_consensus.cc:2804] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.492798 21220 tablet_bootstrap.cc:492] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:15.492760 21469 raft_consensus.cc:697] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:15.493445 21220 ts_tablet_manager.cc:1403] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.001s
I20260709 12:17:15.503273 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ad3b0b41544f61bdc6f80d66c26adf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:15.488459 21450 raft_consensus.cc:697] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:15.503993 21173 raft_consensus.cc:3060] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.504716 21450 consensus_queue.cc:237] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.510751 21219 tablet_bootstrap.cc:654] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.516372 21220 raft_consensus.cc:359] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.518319 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:15.519603 21433 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:15.521623 21220 raft_consensus.cc:385] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.525727 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:15.532850 21096 raft_consensus.cc:2393] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:15.493978 21469 consensus_queue.cc:237] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.535107 21139 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:15.538776 21433 raft_consensus.cc:2749] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.532737 21220 raft_consensus.cc:740] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.541352 21220 consensus_queue.cc:260] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.558437 21220 ts_tablet_manager.cc:1434] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.010s
I20260709 12:17:15.559523 21220 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:15.565402 21173 raft_consensus.cc:2468] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:15.566010 21222 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:15.566588 21222 ts_tablet_manager.cc:1403] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.175s	user 0.010s	sys 0.004s
I20260709 12:17:15.569787 21222 raft_consensus.cc:359] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.570498 21222 raft_consensus.cc:385] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.570768 21222 raft_consensus.cc:740] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.571622 21222 consensus_queue.cc:260] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:15.578430 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:15.594488 21222 ts_tablet_manager.cc:1434] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:17:15.597185 21222 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:15.603719 20845 catalog_manager.cc:5697] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:15.612166 21222 tablet_bootstrap.cc:654] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.624532 21381 consensus_queue.cc:1048] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:15.625455 21220 tablet_bootstrap.cc:654] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.633116 21221 tablet_bootstrap.cc:492] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:15.634157 21221 ts_tablet_manager.cc:1403] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.215s	user 0.014s	sys 0.014s
I20260709 12:17:15.638753 20849 catalog_manager.cc:5697] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.651269 21232 raft_consensus.cc:493] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.652042 21232 raft_consensus.cc:515] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.650702 21221 raft_consensus.cc:359] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.654350 21221 raft_consensus.cc:385] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.654733 21221 raft_consensus.cc:740] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.655591 21232 leader_election.cc:290] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:15.655532 21221 consensus_queue.cc:260] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.659658 21450 raft_consensus.cc:493] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.659031 21221 ts_tablet_manager.cc:1434] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 12:17:15.660379 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.661212 20949 raft_consensus.cc:2468] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:15.661808 21221 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:15.660418 21450 raft_consensus.cc:515] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.665691 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.666374 21026 raft_consensus.cc:2468] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:15.667169 21450 leader_election.cc:290] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.668491 20917 leader_election.cc:304] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:15.669531 21450 raft_consensus.cc:2804] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.670047 21450 raft_consensus.cc:493] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.670172 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:15.670506 21450 raft_consensus.cc:3060] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.670905 21100 raft_consensus.cc:2468] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:15.671229 21221 tablet_bootstrap.cc:654] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.673079 20989 leader_election.cc:304] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:15.675709 21232 raft_consensus.cc:2804] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:15.676198 21232 raft_consensus.cc:493] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.676621 21232 raft_consensus.cc:3060] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.692126 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.692911 21173 raft_consensus.cc:2468] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:15.696728 21450 raft_consensus.cc:515] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.701092 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:15.701305 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:15.701812 21173 raft_consensus.cc:3060] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.702047 21026 raft_consensus.cc:3060] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.711072 21381 raft_consensus.cc:493] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.711722 21381 raft_consensus.cc:515] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.721066 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.724434 21219 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
W20260709 12:17:15.724622 21137 leader_election.cc:343] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.725023 21219 ts_tablet_manager.cc:1403] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.248s	user 0.017s	sys 0.006s
I20260709 12:17:15.726226 21232 raft_consensus.cc:515] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.727969 21219 raft_consensus.cc:359] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.728581 21232 leader_election.cc:290] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:15.728744 21219 raft_consensus.cc:385] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.729053 21482 raft_consensus.cc:493] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.729233 21219 raft_consensus.cc:740] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.731741 21220 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:15.733407 21220 ts_tablet_manager.cc:1403] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.174s	user 0.013s	sys 0.003s
I20260709 12:17:15.733908 21026 raft_consensus.cc:2468] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:15.729687 21482 raft_consensus.cc:515] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:15.737865 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.738247 20917 leader_election.cc:304] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:15.738724 21220 raft_consensus.cc:359] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.739463 21220 raft_consensus.cc:385] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.740106 21220 raft_consensus.cc:740] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.735970 20849 catalog_manager.cc:5697] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.720299 21450 leader_election.cc:290] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.730778 21173 raft_consensus.cc:2468] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:15.742295 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
W20260709 12:17:15.739557 21140 leader_election.cc:343] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.742982 21100 raft_consensus.cc:3060] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.743302 21140 leader_election.cc:304] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:15.731165 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:15.744385 20949 raft_consensus.cc:3060] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.744488 21312 raft_consensus.cc:2749] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:15.739404 21374 raft_consensus.cc:2804] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.747298 21374 raft_consensus.cc:697] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:15.748347 21374 consensus_queue.cc:237] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.749434 21312 consensus_queue.cc:1048] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:15.741747 21381 leader_election.cc:290] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.762941 21219 consensus_queue.cc:260] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.763926 20949 raft_consensus.cc:2468] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:15.741562 21220 consensus_queue.cc:260] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.769366 21222 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:15.769975 21222 ts_tablet_manager.cc:1403] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.174s	user 0.010s	sys 0.034s
I20260709 12:17:15.762997 21100 raft_consensus.cc:2468] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:15.772704 21482 leader_election.cc:290] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.771528 20989 leader_election.cc:304] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:15.777837 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.778760 21173 raft_consensus.cc:2393] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:15.781157 21221 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:15.781725 21221 ts_tablet_manager.cc:1403] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.120s	user 0.027s	sys 0.016s
I20260709 12:17:15.784570 21221 raft_consensus.cc:359] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.785247 21221 raft_consensus.cc:385] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.785954 21232 raft_consensus.cc:2804] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:15.786578 21232 raft_consensus.cc:697] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:15.787801 21220 ts_tablet_manager.cc:1434] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 12:17:15.787533 21232 consensus_queue.cc:237] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.788923 21220 tablet_bootstrap.cc:492] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:15.785526 21221 raft_consensus.cc:740] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.791056 21221 consensus_queue.cc:260] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.801134 21469 raft_consensus.cc:493] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.801766 21469 raft_consensus.cc:515] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.804083 21469 leader_election.cc:290] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:15.800274 21222 raft_consensus.cc:359] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.808168 21222 raft_consensus.cc:385] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:15.808616 21222 raft_consensus.cc:740] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:15.809520 21222 consensus_queue.cc:260] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:15.828888 21221 ts_tablet_manager.cc:1434] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 12:17:15.833125 21221 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:15.800192 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e65504fa010146ef8fef1ff07dfcff69" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:15.835393 21219 ts_tablet_manager.cc:1434] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.015s
I20260709 12:17:15.836532 21219 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:15.841150 21222 ts_tablet_manager.cc:1434] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.002s
I20260709 12:17:15.842809 21222 tablet_bootstrap.cc:492] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:15.846789 21219 tablet_bootstrap.cc:654] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.846597 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:15.851682 21222 tablet_bootstrap.cc:654] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.852137 21220 tablet_bootstrap.cc:654] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:15.857796 21469 raft_consensus.cc:493] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.858394 21469 raft_consensus.cc:515] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.860558 21469 leader_election.cc:290] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:15.909498 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:15.912377 20916 leader_election.cc:304] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
W20260709 12:17:15.947644 21063 leader_election.cc:343] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:15.947758 20951 raft_consensus.cc:3060] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:15.930478 20846 catalog_manager.cc:5697] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.977138 21232 raft_consensus.cc:493] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.978405 21232 raft_consensus.cc:515] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:15.979465 21219 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:15.980561 21219 ts_tablet_manager.cc:1403] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.144s	user 0.024s	sys 0.006s
I20260709 12:17:15.991405 21374 raft_consensus.cc:493] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.992502 21374 raft_consensus.cc:515] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:15.992697 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:15.996228 21469 consensus_queue.cc:1048] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.000929 21220 tablet_bootstrap.cc:492] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:16.002394 21220 ts_tablet_manager.cc:1403] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.214s	user 0.017s	sys 0.000s
W20260709 12:17:16.013455 20916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 46996.4us
I20260709 12:17:15.994149 21173 raft_consensus.cc:2468] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:16.024403 21374 leader_election.cc:290] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:16.027124 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:16.037570 21221 tablet_bootstrap.cc:654] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.034186 21232 leader_election.cc:290] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:16.044389 21222 tablet_bootstrap.cc:492] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:16.055227 21063 leader_election.cc:304] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:16.056917 21469 raft_consensus.cc:2804] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.045537 21222 ts_tablet_manager.cc:1403] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.203s	user 0.014s	sys 0.001s
I20260709 12:17:16.057490 21469 raft_consensus.cc:493] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.048339 20849 catalog_manager.cc:5697] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.075368 21469 raft_consensus.cc:3060] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.076972 21219 raft_consensus.cc:359] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.078080 21219 raft_consensus.cc:385] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.078459 21219 raft_consensus.cc:740] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.081018 21219 consensus_queue.cc:260] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.083238 21219 ts_tablet_manager.cc:1434] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.102s	user 0.003s	sys 0.008s
W20260709 12:17:16.083293 20916 leader_election.cc:343] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.085291 21219 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:16.086591 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 52764.3us
I20260709 12:17:16.111289 21373 consensus_queue.cc:1048] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.093376 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:16.113059 21173 raft_consensus.cc:2468] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
W20260709 12:17:16.121213 21066 leader_election.cc:343] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.121855 21066 leader_election.cc:304] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:16.124382 21373 raft_consensus.cc:2749] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.126984 20989 leader_election.cc:304] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:16.130831 21482 consensus_queue.cc:1048] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.135787 21232 raft_consensus.cc:2804] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.136606 21232 raft_consensus.cc:493] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.137166 21232 raft_consensus.cc:3060] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.139640 21222 raft_consensus.cc:359] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.140561 21222 raft_consensus.cc:385] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.140882 21222 raft_consensus.cc:740] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.057868 21220 raft_consensus.cc:359] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.148101 21220 raft_consensus.cc:385] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.148589 21220 raft_consensus.cc:740] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.143816 21222 consensus_queue.cc:260] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.173187 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:16.178829 21373 consensus_queue.cc:1048] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.181814 21220 consensus_queue.cc:260] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.202256 21311 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.203068 21311 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.207486 21222 ts_tablet_manager.cc:1434] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.149s	user 0.009s	sys 0.001s
I20260709 12:17:16.220265 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:16.220487 21311 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:16.223222 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
W20260709 12:17:16.227437 21140 leader_election.cc:343] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.266520 21506 raft_consensus.cc:2749] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:17:16.267623 21137 leader_election.cc:343] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.269764 21221 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:16.270557 21221 ts_tablet_manager.cc:1403] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.440s	user 0.024s	sys 0.020s
I20260709 12:17:16.257476 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:16.278112 21026 raft_consensus.cc:2468] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:16.280547 21374 raft_consensus.cc:493] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.281193 21374 raft_consensus.cc:515] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.280366 21232 raft_consensus.cc:515] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.283244 21022 raft_consensus.cc:2393] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:16.283761 21374 leader_election.cc:290] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:16.284926 21219 tablet_bootstrap.cc:654] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.289204 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:16.285724 21374 consensus_queue.cc:1048] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:17:16.291038 21170 raft_consensus.cc:2468] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:16.289534 21222 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:16.293663 20917 leader_election.cc:304] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:16.294811 21518 raft_consensus.cc:2804] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.295266 21518 raft_consensus.cc:493] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.295674 21518 raft_consensus.cc:3060] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.293870 21523 raft_consensus.cc:493] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.296638 21220 ts_tablet_manager.cc:1434] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.294s	user 0.023s	sys 0.011s
I20260709 12:17:16.307301 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:16.311342 21221 raft_consensus.cc:359] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.327471 21221 raft_consensus.cc:385] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.327759 21221 raft_consensus.cc:740] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.297549 21523 raft_consensus.cc:515] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
W20260709 12:17:16.302067 20991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 352297us
I20260709 12:17:16.305066 21137 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:16.339010 21221 consensus_queue.cc:260] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.341161 21221 ts_tablet_manager.cc:1434] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 12:17:16.321436 21469 raft_consensus.cc:515] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.326980 21518 raft_consensus.cc:515] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:16.344543 20914 leader_election.cc:304] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:16.345762 21311 raft_consensus.cc:2749] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.353027 21515 raft_consensus.cc:2804] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.353493 21515 raft_consensus.cc:493] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.353894 21515 raft_consensus.cc:3060] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.357273 21523 leader_election.cc:290] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:16.359987 21232 leader_election.cc:290] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:16.361127 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:16.361999 21170 raft_consensus.cc:3060] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:16.349900 21137 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 103905us
I20260709 12:17:16.345140 21022 raft_consensus.cc:2468] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:16.373057 21518 leader_election.cc:290] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:16.373850 21220 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:16.375838 21523 consensus_queue.cc:1048] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.378350 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741c5b82a89142c2840a00c76ede6243" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:16.379693 21169 raft_consensus.cc:2468] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:16.380167 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:16.381129 21097 raft_consensus.cc:2393] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:16.393304 21518 consensus_queue.cc:1048] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.394953 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:16.395709 21026 raft_consensus.cc:3060] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:16.402539 21527 log.cc:927] Time spent T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:17:16.404865 21529 log.cc:927] Time spent T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989: Append to log took a long time: real 0.086s	user 0.003s	sys 0.001s
I20260709 12:17:16.407081 21232 consensus_queue.cc:1048] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:16.406687 21066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 54715.3us
I20260709 12:17:16.435808 21219 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:16.436688 21219 ts_tablet_manager.cc:1403] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.352s	user 0.025s	sys 0.015s
I20260709 12:17:16.439739 21221 tablet_bootstrap.cc:492] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:16.444662 21222 tablet_bootstrap.cc:654] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:16.454634 21526 log.cc:927] Time spent T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:17:16.500593 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:16.500233 21063 leader_election.cc:304] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:16.506382 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741c5b82a89142c2840a00c76ede6243" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:16.506860 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:16.507829 21096 raft_consensus.cc:2393] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:16.509765 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:16.510787 21023 raft_consensus.cc:2393] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:16.482219 21469 leader_election.cc:290] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:16.529716 21528 raft_consensus.cc:2804] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.530323 21528 raft_consensus.cc:493] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.530786 21528 raft_consensus.cc:3060] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.506412 21515 raft_consensus.cc:515] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.554090 21026 raft_consensus.cc:2468] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:16.573230 21220 tablet_bootstrap.cc:654] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.574020 20917 leader_election.cc:304] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:16.575158 21450 raft_consensus.cc:2804] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.575680 21450 raft_consensus.cc:697] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:16.577307 21450 consensus_queue.cc:237] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:16.580539 21221 tablet_bootstrap.cc:654] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:16.566123 21066 leader_election.cc:343] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.602865 21515 leader_election.cc:290] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:16.604563 21219 raft_consensus.cc:359] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:16.610370 21219 raft_consensus.cc:385] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.610667 21219 raft_consensus.cc:740] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.611421 21219 consensus_queue.cc:260] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:16.613030 21219 ts_tablet_manager.cc:1434] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.176s	user 0.002s	sys 0.007s
I20260709 12:17:16.621316 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:16.622090 21022 raft_consensus.cc:3060] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.630424 21514 consensus_queue.cc:1048] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.625222 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:16.632174 21169 raft_consensus.cc:3060] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.633860 21528 raft_consensus.cc:515] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.638550 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6f9c0020a4484ca89737318ee6cc3c" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:16.637212 21528 leader_election.cc:290] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:16.645447 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741c5b82a89142c2840a00c76ede6243" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
W20260709 12:17:16.650048 21066 leader_election.cc:343] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.652117 21219 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:16.655249 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741c5b82a89142c2840a00c76ede6243" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:16.656000 21171 raft_consensus.cc:3060] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.658763 21170 raft_consensus.cc:2468] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:16.660466 20989 leader_election.cc:304] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:16.661482 21435 raft_consensus.cc:2804] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.662046 21435 raft_consensus.cc:697] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:16.662433 21063 leader_election.cc:304] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989
I20260709 12:17:16.663393 21528 raft_consensus.cc:2749] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.663184 21435 consensus_queue.cc:237] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.669107 21518 consensus_queue.cc:1048] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:16.744091 21222 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:16.744709 21222 ts_tablet_manager.cc:1403] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.469s	user 0.022s	sys 0.017s
I20260709 12:17:16.747644 21022 raft_consensus.cc:2468] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:16.751140 21504 raft_consensus.cc:493] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.753993 21504 raft_consensus.cc:515] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:16.754814 21219 tablet_bootstrap.cc:654] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:16.730329 20916 leader_election.cc:343] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.763515 20917 leader_election.cc:304] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:16.766716 21504 leader_election.cc:290] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:16.767455 21504 raft_consensus.cc:2804] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.767931 21504 raft_consensus.cc:697] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:16.773501 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:16.781028 21222 raft_consensus.cc:359] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:16.781798 21222 raft_consensus.cc:385] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.782570 21222 raft_consensus.cc:740] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.768849 21504 consensus_queue.cc:237] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:16.785320 21222 consensus_queue.cc:260] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:16.786006 20914 leader_election.cc:343] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.810555 21222 ts_tablet_manager.cc:1434] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 12:17:16.811982 21222 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:16.836990 21220 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:16.837682 21220 ts_tablet_manager.cc:1403] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.485s	user 0.011s	sys 0.022s
W20260709 12:17:16.838187 21546 log.cc:927] Time spent T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:17:16.843946 21169 raft_consensus.cc:2468] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:16.850206 21220 raft_consensus.cc:359] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:16.851213 21220 raft_consensus.cc:385] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.851577 21220 raft_consensus.cc:740] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.854473 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:16.857967 21221 tablet_bootstrap.cc:492] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:16.877315 21221 ts_tablet_manager.cc:1403] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.460s	user 0.013s	sys 0.006s
I20260709 12:17:16.879930 21221 raft_consensus.cc:359] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
W20260709 12:17:16.864282 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:16.880645 21221 raft_consensus.cc:385] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.881016 21221 raft_consensus.cc:740] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.839807 20847 catalog_manager.cc:5697] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.881882 21221 consensus_queue.cc:260] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.884449 21221 ts_tablet_manager.cc:1434] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:17:16.859553 21171 raft_consensus.cc:2468] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:16.885684 21221 tablet_bootstrap.cc:492] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:16.872140 21219 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:16.892262 21219 ts_tablet_manager.cc:1403] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.241s	user 0.012s	sys 0.003s
I20260709 12:17:16.896387 21222 tablet_bootstrap.cc:654] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.893368 21063 leader_election.cc:304] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:16.899895 21219 raft_consensus.cc:359] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:16.900700 21219 raft_consensus.cc:385] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.901019 21219 raft_consensus.cc:740] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.901922 21525 raft_consensus.cc:2804] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.903394 21219 consensus_queue.cc:260] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:16.902511 21525 raft_consensus.cc:697] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:16.906564 21219 ts_tablet_manager.cc:1434] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 12:17:16.905723 21525 consensus_queue.cc:237] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:16.907927 21219 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:16.852432 21220 consensus_queue.cc:260] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
W20260709 12:17:16.911096 20916 leader_election.cc:343] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.911827 20916 leader_election.cc:304] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:16.914204 21504 raft_consensus.cc:2749] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.915071 21220 ts_tablet_manager.cc:1434] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.077s	user 0.005s	sys 0.003s
I20260709 12:17:16.916548 21220 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:16.922729 21221 tablet_bootstrap.cc:654] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.926566 21311 raft_consensus.cc:493] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.927152 21311 raft_consensus.cc:515] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:16.929287 21219 tablet_bootstrap.cc:654] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.929564 21311 leader_election.cc:290] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:16.930491 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:16.939229 21139 leader_election.cc:343] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.945693 21222 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:16.946288 21222 ts_tablet_manager.cc:1403] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.135s	user 0.009s	sys 0.013s
I20260709 12:17:16.949774 21222 raft_consensus.cc:359] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:16.954133 21222 raft_consensus.cc:385] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.954530 21222 raft_consensus.cc:740] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.955416 21222 consensus_queue.cc:260] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:16.957983 21222 ts_tablet_manager.cc:1434] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:17:16.959221 21222 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:16.959304 21220 tablet_bootstrap.cc:654] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.999541 21221 tablet_bootstrap.cc:492] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:17.000191 21221 ts_tablet_manager.cc:1403] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.006s
I20260709 12:17:17.009729 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.015291 20950 raft_consensus.cc:2468] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:17.017273 21137 leader_election.cc:304] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:17.018913 21312 raft_consensus.cc:2804] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.019376 21312 raft_consensus.cc:493] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.019771 21312 raft_consensus.cc:3060] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.009824 21221 raft_consensus.cc:359] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.026588 21221 raft_consensus.cc:385] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.026912 21221 raft_consensus.cc:740] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.027870 21221 consensus_queue.cc:260] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.030951 21221 ts_tablet_manager.cc:1434] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 12:17:17.031158 21312 raft_consensus.cc:515] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.032297 21221 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:17.040479 21312 leader_election.cc:290] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:17.042872 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
W20260709 12:17:17.044821 21139 leader_election.cc:343] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.032974 21220 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:17.035650 20848 catalog_manager.cc:5697] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.046599 21220 ts_tablet_manager.cc:1403] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.130s	user 0.023s	sys 0.009s
I20260709 12:17:17.049559 21221 tablet_bootstrap.cc:654] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.031247 21222 tablet_bootstrap.cc:654] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.054591 21220 raft_consensus.cc:359] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.055440 21220 raft_consensus.cc:385] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.058167 21220 raft_consensus.cc:740] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.058969 21220 consensus_queue.cc:260] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.061581 21220 ts_tablet_manager.cc:1434] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:17:17.067487 21220 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:17.067754 20849 catalog_manager.cc:5697] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.077250 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:17.077981 20949 raft_consensus.cc:3060] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.083827 21221 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:17.085307 21221 ts_tablet_manager.cc:1403] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.006s
I20260709 12:17:17.091503 21100 raft_consensus.cc:3060] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.082846 20848 catalog_manager.cc:5697] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.097929 20949 raft_consensus.cc:2468] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:17.097666 21221 raft_consensus.cc:359] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.098601 21221 raft_consensus.cc:385] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.098935 21221 raft_consensus.cc:740] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.099920 21516 raft_consensus.cc:493] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.100485 21516 raft_consensus.cc:515] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.106290 21137 leader_election.cc:304] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:17.106892 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741c5b82a89142c2840a00c76ede6243" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.108006 21174 raft_consensus.cc:2393] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:17.108487 21433 raft_consensus.cc:2804] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:17.109458 21433 raft_consensus.cc:697] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:17.110718 21433 consensus_queue.cc:237] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.111413 21516 leader_election.cc:290] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:17.113047 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741c5b82a89142c2840a00c76ede6243" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:17.099790 21221 consensus_queue.cc:260] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.114750 20991 leader_election.cc:304] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:17.116389 21516 raft_consensus.cc:3060] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.100485 21220 tablet_bootstrap.cc:654] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.121922 21221 ts_tablet_manager.cc:1434] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:17:17.123263 21221 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:17.123459 21219 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:17.123999 21219 ts_tablet_manager.cc:1403] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.216s	user 0.008s	sys 0.014s
I20260709 12:17:17.132287 21504 consensus_queue.cc:1048] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:17.136950 21222 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:17.137573 21222 ts_tablet_manager.cc:1403] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.179s	user 0.012s	sys 0.002s
I20260709 12:17:17.142303 21221 tablet_bootstrap.cc:654] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.140978 21222 raft_consensus.cc:359] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.144147 21222 raft_consensus.cc:385] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.144500 21222 raft_consensus.cc:740] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.146318 21222 consensus_queue.cc:260] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.148479 21219 raft_consensus.cc:359] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.149375 21219 raft_consensus.cc:385] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.149565 21222 ts_tablet_manager.cc:1434] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:17:17.149787 21219 raft_consensus.cc:740] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.150143 21518 consensus_queue.cc:1048] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.151074 21219 consensus_queue.cc:260] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.154601 21222 tablet_bootstrap.cc:492] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:17.158061 21312 raft_consensus.cc:493] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.158641 21312 raft_consensus.cc:515] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.161062 21312 leader_election.cc:290] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:17.163069 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e842b1b7f85a4cd9a991bb35684df6b9" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:17.165354 21222 tablet_bootstrap.cc:654] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:17.166765 21140 leader_election.cc:343] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.177572 21220 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:17.178272 21220 ts_tablet_manager.cc:1403] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.006s
I20260709 12:17:17.182096 21525 raft_consensus.cc:493] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.182754 21525 raft_consensus.cc:515] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.181990 21220 raft_consensus.cc:359] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.184378 21220 raft_consensus.cc:385] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.184765 21220 raft_consensus.cc:740] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.153981 21516 raft_consensus.cc:2749] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:17.189507 21219 ts_tablet_manager.cc:1434] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.005s
I20260709 12:17:17.193411 20949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.194502 21525 leader_election.cc:290] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:17.196346 21219 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:17.198035 21220 consensus_queue.cc:260] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.199893 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:17.201246 21220 ts_tablet_manager.cc:1434] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:17:17.202095 21066 leader_election.cc:304] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:17.203785 21542 raft_consensus.cc:2749] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.205826 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e842b1b7f85a4cd9a991bb35684df6b9" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.210860 21433 raft_consensus.cc:493] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.211495 21433 raft_consensus.cc:515] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.215457 21220 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:17.216125 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:17.218138 21139 leader_election.cc:343] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:17.220129 21137 leader_election.cc:343] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.220767 21137 leader_election.cc:304] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:17.221740 21312 raft_consensus.cc:2749] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:17.193250 21373 raft_consensus.cc:493] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.223502 21373 raft_consensus.cc:515] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.231137 21433 leader_election.cc:290] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:17.238965 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.239799 21219 tablet_bootstrap.cc:654] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.239171 21373 leader_election.cc:290] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:17.247941 21221 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:17.248539 21221 ts_tablet_manager.cc:1403] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.126s	user 0.040s	sys 0.013s
I20260709 12:17:17.251507 21221 raft_consensus.cc:359] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.252269 21221 raft_consensus.cc:385] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.252549 21221 raft_consensus.cc:740] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.253955 21221 consensus_queue.cc:260] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.254524 21519 consensus_queue.cc:1048] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.254709 20849 catalog_manager.cc:5697] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.257486 21221 ts_tablet_manager.cc:1434] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:17:17.258777 21221 tablet_bootstrap.cc:492] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:17.259896 21222 tablet_bootstrap.cc:492] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:17.260452 21222 ts_tablet_manager.cc:1403] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.109s	user 0.009s	sys 0.016s
I20260709 12:17:17.263918 21220 tablet_bootstrap.cc:654] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.264890 21221 tablet_bootstrap.cc:654] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.268078 21222 raft_consensus.cc:359] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.268920 21222 raft_consensus.cc:385] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.269191 21222 raft_consensus.cc:740] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.269984 21222 consensus_queue.cc:260] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.273237 21222 ts_tablet_manager.cc:1434] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:17:17.273562 21219 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:17.275982 21219 ts_tablet_manager.cc:1403] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.000s
I20260709 12:17:17.279083 21219 raft_consensus.cc:359] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.279851 21219 raft_consensus.cc:385] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.280122 21219 raft_consensus.cc:740] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.280884 21219 consensus_queue.cc:260] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.274432 21222 tablet_bootstrap.cc:492] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:17.286610 21219 ts_tablet_manager.cc:1434] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:17:17.288166 21219 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:17.293251 21221 tablet_bootstrap.cc:492] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:17.294015 21221 ts_tablet_manager.cc:1403] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.002s
I20260709 12:17:17.297281 21221 raft_consensus.cc:359] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.298111 21221 raft_consensus.cc:385] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.298390 21221 raft_consensus.cc:740] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.299106 21221 consensus_queue.cc:260] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.303515 21220 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:17.304056 21220 ts_tablet_manager.cc:1403] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.008s
I20260709 12:17:17.304848 21219 tablet_bootstrap.cc:654] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.306972 21220 raft_consensus.cc:359] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.307719 21220 raft_consensus.cc:385] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.307978 21220 raft_consensus.cc:740] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.308599 21221 ts_tablet_manager.cc:1434] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:17:17.293396 21222 tablet_bootstrap.cc:654] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.308800 21220 consensus_queue.cc:260] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.311780 21519 consensus_queue.cc:1048] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.312085 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181a9cec50b94395aac6bd156652a9f2" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.312980 20951 raft_consensus.cc:2393] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:17.314487 21220 ts_tablet_manager.cc:1434] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:17:17.322404 21221 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:17.327418 21063 leader_election.cc:304] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:17.328783 21450 raft_consensus.cc:493] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.332684 21220 tablet_bootstrap.cc:492] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:17.333035 21450 raft_consensus.cc:515] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.335327 21450 leader_election.cc:290] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:17.336869 21373 raft_consensus.cc:3060] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.345075 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:17.346019 21022 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:17.348004 20917 leader_election.cc:304] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:17.350523 21450 raft_consensus.cc:2804] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.360769 21220 tablet_bootstrap.cc:654] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.366007 21450 raft_consensus.cc:493] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.365878 21525 consensus_queue.cc:1048] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.366914 21222 tablet_bootstrap.cc:492] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:17.367725 21222 ts_tablet_manager.cc:1403] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.094s	user 0.013s	sys 0.009s
I20260709 12:17:17.374594 21450 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.377483 21516 raft_consensus.cc:493] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.392879 21516 raft_consensus.cc:515] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.394980 21516 leader_election.cc:290] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:17.400173 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.400558 21219 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:17.401098 21219 ts_tablet_manager.cc:1403] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.113s	user 0.009s	sys 0.010s
I20260709 12:17:17.405889 21221 tablet_bootstrap.cc:654] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.407704 21450 raft_consensus.cc:515] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.410596 21373 raft_consensus.cc:2749] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:17.423872 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:17.423025 21450 leader_election.cc:290] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:17.425590 21220 tablet_bootstrap.cc:492] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:17.426066 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:17.426393 21220 ts_tablet_manager.cc:1403] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.094s	user 0.017s	sys 0.014s
I20260709 12:17:17.426833 21022 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.429447 21220 raft_consensus.cc:359] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.430265 21220 raft_consensus.cc:385] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.430600 21220 raft_consensus.cc:740] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.425779 21219 raft_consensus.cc:359] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.431520 21219 raft_consensus.cc:385] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.429139 21222 raft_consensus.cc:359] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.432709 21222 raft_consensus.cc:385] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.432230 21220 consensus_queue.cc:260] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.433002 21222 raft_consensus.cc:740] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.431895 21219 raft_consensus.cc:740] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.438026 21222 consensus_queue.cc:260] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.458086 21170 raft_consensus.cc:2468] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:17.462001 21373 consensus_queue.cc:1048] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.463450 20989 leader_election.cc:304] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:17.469368 21435 raft_consensus.cc:2804] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.469918 21435 raft_consensus.cc:493] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.470328 21435 raft_consensus.cc:3060] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.471010 21220 ts_tablet_manager.cc:1434] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:17:17.473263 21219 consensus_queue.cc:260] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.484529 21222 ts_tablet_manager.cc:1434] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.002s
I20260709 12:17:17.519649 21220 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
W20260709 12:17:17.488073 20991 leader_election.cc:343] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.522809 21219 ts_tablet_manager.cc:1434] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.121s	user 0.011s	sys 0.000s
I20260709 12:17:17.524456 21219 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:17.529986 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.532454 21221 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:17.533026 21221 ts_tablet_manager.cc:1403] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.211s	user 0.008s	sys 0.014s
I20260709 12:17:17.536985 21221 raft_consensus.cc:359] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.537706 21221 raft_consensus.cc:385] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.510233 21516 raft_consensus.cc:493] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.537992 21221 raft_consensus.cc:740] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.538445 21516 raft_consensus.cc:515] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.538894 21221 consensus_queue.cc:260] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.537715 21222 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
W20260709 12:17:17.554858 20991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 67548.7us
I20260709 12:17:17.563383 21435 raft_consensus.cc:515] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.563405 21433 raft_consensus.cc:493] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.572023 21433 raft_consensus.cc:515] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.575874 21022 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:17.581372 20917 leader_election.cc:304] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:17.577474 21516 leader_election.cc:290] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:17.578487 21312 consensus_queue.cc:1048] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.581344 21433 leader_election.cc:290] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:17.588719 21221 ts_tablet_manager.cc:1434] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.003s
I20260709 12:17:17.559034 21576 raft_consensus.cc:493] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.594172 21222 tablet_bootstrap.cc:654] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.584295 21504 raft_consensus.cc:2804] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:17.596100 21504 raft_consensus.cc:697] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:17.597276 21504 consensus_queue.cc:237] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.600768 21518 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.601480 21518 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.604025 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:17.605844 21139 leader_election.cc:343] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.605577 21576 raft_consensus.cc:515] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:17.608180 21576 leader_election.cc:290] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:17.608768 21518 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:17.591367 21435 leader_election.cc:290] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:17.611395 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:17.612071 21026 raft_consensus.cc:2468] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:17.615530 21140 leader_election.cc:304] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:17.617120 21220 tablet_bootstrap.cc:654] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.617147 21311 raft_consensus.cc:2804] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.617803 21311 raft_consensus.cc:493] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.618161 21311 raft_consensus.cc:3060] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.621068 21518 raft_consensus.cc:493] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.621855 21518 raft_consensus.cc:515] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.624241 21516 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.624766 21516 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.625253 21525 raft_consensus.cc:493] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.625867 21525 raft_consensus.cc:515] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.626906 21516 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:17.628072 21525 leader_election.cc:290] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:17.630082 21219 tablet_bootstrap.cc:654] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.630401 21221 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:17.631139 21096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:17.631357 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:17.631889 21096 raft_consensus.cc:3060] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:17.594008 21066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 74688us
I20260709 12:17:17.674188 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:17.675088 21098 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
W20260709 12:17:17.679123 20991 leader_election.cc:343] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.682305 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:17.683151 21095 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.689620 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.645378 21518 leader_election.cc:290] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:17.748201 20947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.749161 20947 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:17.719367 21311 raft_consensus.cc:515] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:17.750980 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.752099 21169 raft_consensus.cc:2393] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:17.752795 21311 leader_election.cc:290] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:17.753839 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:17.754653 21169 raft_consensus.cc:3060] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.778687 21221 tablet_bootstrap.cc:654] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:17.778098 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 130751us
W20260709 12:17:17.780395 20916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 91674.2us
I20260709 12:17:17.754884 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395043703c424af7b41072d0e92012a1" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:17.752960 21096 raft_consensus.cc:2468] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:17.801971 20989 leader_election.cc:304] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:17.795312 21169 raft_consensus.cc:2468] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:17.808310 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:17.808614 21589 raft_consensus.cc:3060] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.810591 20989 leader_election.cc:304] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:17.809218 21022 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:17.795246 21023 raft_consensus.cc:2468] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:17.808310 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395043703c424af7b41072d0e92012a1" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.834802 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:17.821532 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:17.836683 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.819690 21519 raft_consensus.cc:2804] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:17.847225 21219 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:17.848117 21219 ts_tablet_manager.cc:1403] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.324s	user 0.026s	sys 0.011s
W20260709 12:17:17.856377 20916 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 70640.1us
W20260709 12:17:17.859206 21063 leader_election.cc:343] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.828258 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.865100 21222 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:17.865787 21222 ts_tablet_manager.cc:1403] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.362s	user 0.015s	sys 0.007s
I20260709 12:17:17.852028 20950 raft_consensus.cc:2468] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:17.853904 21519 raft_consensus.cc:697] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:17.846009 20946 raft_consensus.cc:2468] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
W20260709 12:17:17.872903 21139 leader_election.cc:343] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.867760 21169 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:17.889714 20845 catalog_manager.cc:5697] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.896001 21063 leader_election.cc:304] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: eb45c1a4fe5341368530840de4608989
W20260709 12:17:17.896289 20914 leader_election.cc:343] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:17.901289 21063 leader_election.cc:304] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:17.903056 21587 raft_consensus.cc:2804] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.904729 21587 raft_consensus.cc:493] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.905087 21587 raft_consensus.cc:3060] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.906725 21525 raft_consensus.cc:2804] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.925283 20914 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:17.924036 21220 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:17.926260 21220 ts_tablet_manager.cc:1403] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.433s	user 0.019s	sys 0.016s
I20260709 12:17:17.935180 21221 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:17.935783 21221 ts_tablet_manager.cc:1403] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.306s	user 0.023s	sys 0.003s
I20260709 12:17:17.938215 21589 raft_consensus.cc:2749] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:17:17.938656 21599 log.cc:927] Time spent T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:17:17.939400 21221 raft_consensus.cc:359] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.940245 21221 raft_consensus.cc:385] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.940516 21221 raft_consensus.cc:740] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.946762 21221 consensus_queue.cc:260] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:17.946976 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:17.947876 21174 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:17.948904 21221 ts_tablet_manager.cc:1434] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:17:17.924750 21525 raft_consensus.cc:493] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.949649 21525 raft_consensus.cc:3060] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.951123 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:17.957908 21593 consensus_queue.cc:1048] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:17:17.960494 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:17.972689 21222 raft_consensus.cc:359] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.001118 21222 raft_consensus.cc:385] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.909108 21519 consensus_queue.cc:237] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:18.003069 21587 raft_consensus.cc:515] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.006124 21587 leader_election.cc:290] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:17.982847 21219 raft_consensus.cc:359] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.010999 21219 raft_consensus.cc:385] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.011456 21219 raft_consensus.cc:740] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.987377 21220 raft_consensus.cc:359] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:17.979300 21588 raft_consensus.cc:2804] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:18.012183 21588 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.012621 21588 raft_consensus.cc:3060] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.012859 21219 consensus_queue.cc:260] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.016315 21219 ts_tablet_manager.cc:1434] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.168s	user 0.008s	sys 0.000s
I20260709 12:17:18.017668 21220 raft_consensus.cc:385] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.018216 21220 raft_consensus.cc:740] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:18.019474 21221 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:18.017745 21222 raft_consensus.cc:740] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.947692 21095 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:18.034611 21220 consensus_queue.cc:260] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.037590 21222 consensus_queue.cc:260] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.039852 21220 ts_tablet_manager.cc:1434] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.000s
I20260709 12:17:18.042300 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:18.042492 20989 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:18.043845 21596 consensus_queue.cc:1048] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:18.051287 21222 ts_tablet_manager.cc:1434] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.185s	user 0.005s	sys 0.003s
I20260709 12:17:18.046900 20946 raft_consensus.cc:3060] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.058048 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:18.058818 21174 raft_consensus.cc:3060] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.050119 21516 raft_consensus.cc:2804] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:18.066365 21516 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.066747 21516 raft_consensus.cc:3060] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.070519 21220 tablet_bootstrap.cc:492] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:18.074393 21219 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:18.078164 21137 leader_election.cc:343] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:18.078730 21137 leader_election.cc:304] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:18.085196 21221 tablet_bootstrap.cc:654] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:18.090451 21022 raft_consensus.cc:2468] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:18.092120 21140 leader_election.cc:304] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:18.086404 21312 raft_consensus.cc:2749] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:18.045499 21588 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.113374 21433 raft_consensus.cc:2804] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:18.097086 20946 raft_consensus.cc:2468] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:18.115945 21063 leader_election.cc:304] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:18.117123 21542 raft_consensus.cc:2804] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:17:18.130330 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 49996.4us
I20260709 12:17:18.121546 21433 raft_consensus.cc:697] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:18.138974 21174 raft_consensus.cc:2468] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:18.139998 21433 consensus_queue.cc:237] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:18.174678 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:18.171020 21168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:18.127357 21525 raft_consensus.cc:515] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
W20260709 12:17:18.166424 21137 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 85824.8us
I20260709 12:17:18.184861 21222 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:18.215099 21219 tablet_bootstrap.cc:654] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:18.198405 21168 raft_consensus.cc:3060] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.198904 21542 raft_consensus.cc:697] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
W20260709 12:17:18.227510 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 87142.4us
I20260709 12:17:18.226327 21588 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:18.277441 21542 consensus_queue.cc:237] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.282349 21525 leader_election.cc:290] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:18.303915 21220 tablet_bootstrap.cc:654] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:18.293866 20991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 122691us
W20260709 12:17:18.351444 21619 log.cc:927] Time spent T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:17:18.351809 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395043703c424af7b41072d0e92012a1" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:18.352474 21022 raft_consensus.cc:3060] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.351239 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:18.304629 21609 consensus_queue.cc:1048] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
I20260709 12:17:18.335398 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395043703c424af7b41072d0e92012a1" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:18.357725 20946 raft_consensus.cc:3060] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.371829 21516 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.348860 21578 raft_consensus.cc:493] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.376739 21578 raft_consensus.cc:515] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.431052 21221 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:18.430207 21024 raft_consensus.cc:2393] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:18.447760 21221 ts_tablet_manager.cc:1403] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.429s	user 0.033s	sys 0.051s
I20260709 12:17:18.433290 21610 raft_consensus.cc:493] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.436218 21022 raft_consensus.cc:2468] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
W20260709 12:17:18.439889 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 59579.2us
I20260709 12:17:18.451977 21610 raft_consensus.cc:515] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.441577 21219 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:18.453665 21219 ts_tablet_manager.cc:1403] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.436s	user 0.036s	sys 0.025s
I20260709 12:17:18.454958 21578 leader_election.cc:290] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:18.462575 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e842b1b7f85a4cd9a991bb35684df6b9" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:18.463573 21174 raft_consensus.cc:2468] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:18.465065 20989 leader_election.cc:304] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:18.465989 21585 raft_consensus.cc:2804] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:18.466379 21585 raft_consensus.cc:493] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.466747 21585 raft_consensus.cc:3060] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.467562 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e842b1b7f85a4cd9a991bb35684df6b9" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:18.468487 20951 raft_consensus.cc:2468] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:18.475221 21585 raft_consensus.cc:515] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.480839 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e842b1b7f85a4cd9a991bb35684df6b9" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:18.481467 21174 raft_consensus.cc:3060] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.484668 21221 raft_consensus.cc:359] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:18.485733 21221 raft_consensus.cc:385] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.486095 21221 raft_consensus.cc:740] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:18.491840 21219 raft_consensus.cc:359] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:18.492762 21219 raft_consensus.cc:385] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.493036 21219 raft_consensus.cc:740] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:18.493716 21219 consensus_queue.cc:260] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:18.507331 21222 tablet_bootstrap.cc:654] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:18.506567 21221 consensus_queue.cc:260] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:18.513125 21518 consensus_queue.cc:1048] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:18.515998 21219 ts_tablet_manager.cc:1434] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 12:17:18.514175 21221 ts_tablet_manager.cc:1434] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 12:17:18.539579 21221 tablet_bootstrap.cc:492] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:18.542829 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:18.538705 21586 consensus_queue.cc:1048] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:17:18.544081 21171 raft_consensus.cc:2468] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 0.
I20260709 12:17:18.546880 21063 leader_election.cc:304] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:18.552506 21576 raft_consensus.cc:2804] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:18.559384 20951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:18.560240 20951 raft_consensus.cc:2393] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:18.562031 20947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e842b1b7f85a4cd9a991bb35684df6b9" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:18.569712 21219 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:18.554661 21585 leader_election.cc:290] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:18.556399 21576 raft_consensus.cc:493] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:18.576922 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 71536us
I20260709 12:17:18.576282 20916 leader_election.cc:304] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:18.560009 21516 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:18.566368 20947 raft_consensus.cc:3060] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.587877 21220 tablet_bootstrap.cc:492] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:18.588349 21220 ts_tablet_manager.cc:1403] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.518s	user 0.010s	sys 0.026s
I20260709 12:17:18.577961 21576 raft_consensus.cc:3060] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.591192 21628 raft_consensus.cc:2749] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:18.588948 21168 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:18.609431 21219 tablet_bootstrap.cc:654] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:18.596747 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:18.613117 21171 raft_consensus.cc:2393] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:18.598158 20946 raft_consensus.cc:2468] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:18.615068 20989 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:18.589995 21066 leader_election.cc:304] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
W20260709 12:17:18.604065 21624 log.cc:927] Time spent T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: Append to log took a long time: real 0.165s	user 0.000s	sys 0.002s
I20260709 12:17:18.616919 20914 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:18.627012 21222 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:18.627797 21222 ts_tablet_manager.cc:1403] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.443s	user 0.013s	sys 0.026s
I20260709 12:17:18.618304 21435 raft_consensus.cc:2749] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:18.627666 21601 raft_consensus.cc:2804] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:18.633116 21601 raft_consensus.cc:697] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:18.634073 21601 consensus_queue.cc:237] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.635788 21612 raft_consensus.cc:493] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.636325 21612 raft_consensus.cc:515] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:18.624745 21542 raft_consensus.cc:2804] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:18.638667 21612 leader_election.cc:290] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:18.641343 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85cfddd6d98445c951b81e38136c039" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:18.642122 20950 raft_consensus.cc:2468] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:18.644833 21542 raft_consensus.cc:697] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:18.645385 21222 raft_consensus.cc:359] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.646253 21222 raft_consensus.cc:385] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.645887 21542 consensus_queue.cc:237] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:18.646648 21222 raft_consensus.cc:740] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:18.647584 21222 consensus_queue.cc:260] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.651996 20989 leader_election.cc:304] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:18.618484 21174 raft_consensus.cc:2468] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:18.658581 21578 consensus_queue.cc:1048] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:18.658335 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85cfddd6d98445c951b81e38136c039" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:18.662413 21222 ts_tablet_manager.cc:1434] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.003s
I20260709 12:17:18.663401 20847 catalog_manager.cc:5697] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.673592 21220 raft_consensus.cc:359] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:18.678189 21612 raft_consensus.cc:2804] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:18.678848 21610 leader_election.cc:290] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:18.684008 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:18.686545 20947 raft_consensus.cc:2468] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:18.686791 21221 tablet_bootstrap.cc:654] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:18.714138 21220 raft_consensus.cc:385] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.715669 21220 raft_consensus.cc:740] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:18.721141 21220 consensus_queue.cc:260] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:18.720608 20989 leader_election.cc:304] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:18.694672 21576 raft_consensus.cc:515] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.714656 21612 raft_consensus.cc:493] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:18.734586 21063 leader_election.cc:343] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:18.739594 21222 tablet_bootstrap.cc:492] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:18.725201 21519 raft_consensus.cc:2804] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:18.741261 21519 raft_consensus.cc:697] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:18.742197 21519 consensus_queue.cc:237] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:18.743536 21576 leader_election.cc:290] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:18.735733 21612 raft_consensus.cc:3060] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.746534 21613 raft_consensus.cc:493] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.747179 21613 raft_consensus.cc:515] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:18.714900 21585 consensus_queue.cc:1048] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
W20260709 12:17:18.753782 20991 leader_election.cc:343] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:18.756420 21220 ts_tablet_manager.cc:1434] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.168s	user 0.008s	sys 0.000s
I20260709 12:17:18.763635 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:18.764377 21174 raft_consensus.cc:3060] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.765398 21614 raft_consensus.cc:493] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.774893 21220 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
W20260709 12:17:18.722244 20917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 80018.5us
I20260709 12:17:18.777199 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:18.789529 21613 leader_election.cc:290] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
W20260709 12:17:18.793150 21063 leader_election.cc:343] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:18.804489 21613 consensus_queue.cc:1048] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:18.808655 21168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:18.782277 21614 raft_consensus.cc:515] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.813657 21614 leader_election.cc:290] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:18.814719 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:18.816784 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
W20260709 12:17:18.820178 21139 leader_election.cc:343] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:18.825238 21222 tablet_bootstrap.cc:654] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:18.842648 21137 leader_election.cc:343] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:18.843267 21137 leader_election.cc:304] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:18.844846 21598 raft_consensus.cc:2749] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:18.846808 20845 catalog_manager.cc:5697] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.901362 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:18.906927 21022 raft_consensus.cc:2393] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:18.912953 21220 tablet_bootstrap.cc:654] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:18.915445 21221 tablet_bootstrap.cc:492] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:18.915980 21221 ts_tablet_manager.cc:1403] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.377s	user 0.022s	sys 0.032s
I20260709 12:17:18.916332 21612 raft_consensus.cc:515] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:18.918524 21091 raft_consensus.cc:3060] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.918838 21221 raft_consensus.cc:359] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:18.919631 21221 raft_consensus.cc:385] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.924445 21312 consensus_queue.cc:1048] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:18.919914 21221 raft_consensus.cc:740] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:18.926806 21221 consensus_queue.cc:260] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:18.929847 21221 ts_tablet_manager.cc:1434] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 12:17:18.933276 21066 leader_election.cc:304] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989
I20260709 12:17:18.934768 21612 leader_election.cc:290] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:18.944187 21174 raft_consensus.cc:2468] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:18.947551 21542 consensus_queue.cc:1048] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:18.951009 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85cfddd6d98445c951b81e38136c039" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:18.951841 20946 raft_consensus.cc:3060] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:18.954699 21063 leader_election.cc:304] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:18.961453 21221 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:18.969219 21610 raft_consensus.cc:2749] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:18.974772 21610 raft_consensus.cc:2804] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:18.975612 21610 raft_consensus.cc:697] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:18.976887 21610 consensus_queue.cc:237] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.984225 20946 raft_consensus.cc:2468] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:18.986965 21219 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:18.990674 21219 ts_tablet_manager.cc:1403] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.438s	user 0.022s	sys 0.001s
I20260709 12:17:18.993783 21219 raft_consensus.cc:359] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:18.995400 21219 raft_consensus.cc:385] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:18.993322 20848 catalog_manager.cc:5697] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:18.995689 20989 leader_election.cc:304] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:18.997309 21608 raft_consensus.cc:2804] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:18.998025 21608 raft_consensus.cc:697] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:18.999212 21608 consensus_queue.cc:237] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.000146 21219 raft_consensus.cc:740] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.007746 20849 catalog_manager.cc:5697] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:19.014129 21063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 59568.8us
I20260709 12:17:19.016638 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85cfddd6d98445c951b81e38136c039" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:19.004864 21219 consensus_queue.cc:260] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.029760 21219 ts_tablet_manager.cc:1434] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:17:19.043138 21433 consensus_queue.cc:1048] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:19.055668 21219 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:19.072219 20991 leader_election.cc:343] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:19.075371 21219 tablet_bootstrap.cc:654] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.078137 21222 tablet_bootstrap.cc:492] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:19.078941 21222 ts_tablet_manager.cc:1403] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.348s	user 0.012s	sys 0.016s
I20260709 12:17:19.082051 21222 raft_consensus.cc:359] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.085821 21222 raft_consensus.cc:385] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.086292 21222 raft_consensus.cc:740] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.087312 21222 consensus_queue.cc:260] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.089090 21222 ts_tablet_manager.cc:1434] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:17:19.091154 21222 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:19.093732 20847 catalog_manager.cc:5697] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:19.084210 21221 tablet_bootstrap.cc:654] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.096099 20847 catalog_manager.cc:5697] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:19.103184 21221 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:19.105607 21221 ts_tablet_manager.cc:1403] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.145s	user 0.015s	sys 0.000s
I20260709 12:17:19.114063 21221 raft_consensus.cc:359] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.110692 21219 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.115309 21219 ts_tablet_manager.cc:1403] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.000s
I20260709 12:17:19.119982 21219 raft_consensus.cc:359] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.120888 21219 raft_consensus.cc:385] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.121248 21219 raft_consensus.cc:740] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.122174 21219 consensus_queue.cc:260] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.108739 21222 tablet_bootstrap.cc:654] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.130731 21221 raft_consensus.cc:385] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.131225 21221 raft_consensus.cc:740] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.127365 21219 ts_tablet_manager.cc:1434] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:17:19.133298 21219 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:19.133930 21221 consensus_queue.cc:260] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.324292 21610 consensus_queue.cc:1048] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:19.328614 21578 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.329282 21578 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.331597 21578 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:19.334344 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:19.335762 20991 leader_election.cc:343] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:19.336015 21222 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:19.336668 21222 ts_tablet_manager.cc:1403] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.246s	user 0.011s	sys 0.002s
I20260709 12:17:19.338994 21220 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:19.340173 21220 ts_tablet_manager.cc:1403] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.566s	user 0.038s	sys 0.028s
I20260709 12:17:19.340952 21221 ts_tablet_manager.cc:1434] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.230s	user 0.011s	sys 0.033s
W20260709 12:17:19.341763 20917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 80964.5us
I20260709 12:17:19.342865 21633 raft_consensus.cc:493] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.343415 21633 raft_consensus.cc:515] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.345449 20845 catalog_manager.cc:5697] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 1 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:19.346367 21633 leader_election.cc:290] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:19.349169 21221 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:19.344002 21219 tablet_bootstrap.cc:654] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.349946 21602 consensus_queue.cc:1048] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:19.350889 21220 raft_consensus.cc:359] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.353536 21220 raft_consensus.cc:385] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.354040 21220 raft_consensus.cc:740] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.354140 21596 raft_consensus.cc:493] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.355026 21596 raft_consensus.cc:515] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.356591 20947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:19.357368 21596 leader_election.cc:290] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:19.360323 21220 consensus_queue.cc:260] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
W20260709 12:17:19.353837 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 45414.9us
I20260709 12:17:19.362094 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:19.367762 21578 consensus_queue.cc:1048] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:19.369583 20989 leader_election.cc:343] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:19.370057 20989 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:19.371312 21578 raft_consensus.cc:2749] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:19.373224 21587 consensus_queue.cc:1048] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:19.374099 21614 raft_consensus.cc:493] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.374678 21614 raft_consensus.cc:515] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.375952 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc706075ca374b5eb2cc3867a4f7a456" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:19.377163 21614 leader_election.cc:290] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:19.378224 20917 leader_election.cc:343] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:19.381985 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "352e00dbe52446aca63ec1b6cc70f554" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:19.382941 21173 raft_consensus.cc:2393] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:19.386077 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:19.388530 20914 leader_election.cc:304] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:19.389421 21633 raft_consensus.cc:3060] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.391247 21518 consensus_queue.cc:1048] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:19.393983 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc706075ca374b5eb2cc3867a4f7a456" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:19.396946 21585 raft_consensus.cc:493] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.397588 21585 raft_consensus.cc:515] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.399781 21222 raft_consensus.cc:359] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.400389 21585 leader_election.cc:290] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:19.400786 21222 raft_consensus.cc:385] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.401271 21222 raft_consensus.cc:740] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.401850 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:19.404045 20991 leader_election.cc:343] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:19.407250 21139 leader_election.cc:343] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:19.402302 21222 consensus_queue.cc:260] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.423167 21222 ts_tablet_manager.cc:1434] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.002s
I20260709 12:17:19.422585 21219 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.423892 21219 ts_tablet_manager.cc:1403] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.291s	user 0.024s	sys 0.021s
I20260709 12:17:19.424388 21222 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:19.428200 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:19.429014 21020 raft_consensus.cc:2468] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:19.433523 21220 ts_tablet_manager.cc:1434] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.000s
I20260709 12:17:19.434743 21220 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:19.436805 21608 consensus_queue.cc:1048] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:19.441321 21221 tablet_bootstrap.cc:654] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:19.451426 20914 leader_election.cc:343] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:19.452033 20914 leader_election.cc:304] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989
I20260709 12:17:19.452702 21633 raft_consensus.cc:2749] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:19.456094 21219 raft_consensus.cc:359] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.456974 21219 raft_consensus.cc:385] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.457311 21219 raft_consensus.cc:740] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.462570 21140 leader_election.cc:304] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:19.463611 21518 raft_consensus.cc:2749] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:19.464272 21598 raft_consensus.cc:2804] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:19.464788 21598 raft_consensus.cc:493] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:19.469972 21598 raft_consensus.cc:3060] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.476293 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:19.480499 21219 consensus_queue.cc:260] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.492173 21219 ts_tablet_manager.cc:1434] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 12:17:19.495860 21219 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:19.509397 21219 tablet_bootstrap.cc:654] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.520222 20849 catalog_manager.cc:5697] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:19.544835 21610 raft_consensus.cc:493] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.545444 21610 raft_consensus.cc:515] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.547812 21610 leader_election.cc:290] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:19.547222 21219 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.549042 21219 ts_tablet_manager.cc:1403] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.006s
W20260709 12:17:19.555131 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:19.562948 21221 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:19.563565 21221 ts_tablet_manager.cc:1403] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.215s	user 0.015s	sys 0.000s
I20260709 12:17:19.563202 21610 consensus_queue.cc:1048] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:17:19.568600 21516 consensus_queue.cc:1048] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:19.569733 21222 tablet_bootstrap.cc:654] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.571487 21221 raft_consensus.cc:359] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.572314 21221 raft_consensus.cc:385] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.572667 21221 raft_consensus.cc:740] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.573431 21221 consensus_queue.cc:260] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.551812 21219 raft_consensus.cc:359] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.584394 21219 raft_consensus.cc:385] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.584878 21219 raft_consensus.cc:740] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.587920 21219 consensus_queue.cc:260] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.592039 21220 tablet_bootstrap.cc:654] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.592844 21221 ts_tablet_manager.cc:1434] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:17:19.594183 21221 tablet_bootstrap.cc:492] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:19.595494 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85cfddd6d98445c951b81e38136c039" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:19.596278 21098 raft_consensus.cc:3060] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.588936 20950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f85cfddd6d98445c951b81e38136c039" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:19.621420 21219 ts_tablet_manager.cc:1434] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.000s
I20260709 12:17:19.625734 21219 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:19.622265 21063 leader_election.cc:304] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:19.628724 21221 tablet_bootstrap.cc:654] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.633059 21219 tablet_bootstrap.cc:654] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.624147 21587 consensus_queue.cc:1048] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:17:19.653955 21598 raft_consensus.cc:515] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.659752 21222 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:19.675069 21222 ts_tablet_manager.cc:1403] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.251s	user 0.016s	sys 0.004s
I20260709 12:17:19.678429 21222 raft_consensus.cc:359] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.679245 21222 raft_consensus.cc:385] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.679559 21222 raft_consensus.cc:740] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.666254 21576 raft_consensus.cc:2749] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:19.673527 21585 consensus_queue.cc:1048] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:19.682113 21222 consensus_queue.cc:260] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.690610 21222 ts_tablet_manager.cc:1434] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:17:19.692051 21222 tablet_bootstrap.cc:492] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:19.692880 21174 raft_consensus.cc:2393] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:19.701493 21614 raft_consensus.cc:493] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.701678 20989 leader_election.cc:304] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:19.702157 21614 raft_consensus.cc:515] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.703400 21620 raft_consensus.cc:3060] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.704675 21614 leader_election.cc:290] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:19.708645 20947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc706075ca374b5eb2cc3867a4f7a456" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:19.709448 20947 raft_consensus.cc:2468] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:19.709957 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc706075ca374b5eb2cc3867a4f7a456" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:19.710726 21024 raft_consensus.cc:2468] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:19.710850 21137 leader_election.cc:304] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:19.711783 21614 raft_consensus.cc:2804] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:19.712188 21614 raft_consensus.cc:493] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:19.712564 21614 raft_consensus.cc:3060] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.697906 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:19.722296 21598 leader_election.cc:290] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:19.725782 21220 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:19.723853 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:19.726429 21220 ts_tablet_manager.cc:1403] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.292s	user 0.019s	sys 0.000s
I20260709 12:17:19.726826 21620 raft_consensus.cc:2749] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:19.731243 21222 tablet_bootstrap.cc:654] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.736234 21022 raft_consensus.cc:2468] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
W20260709 12:17:19.731356 21139 leader_election.cc:343] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:19.741473 21219 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.742095 21219 ts_tablet_manager.cc:1403] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.117s	user 0.041s	sys 0.028s
I20260709 12:17:19.745551 21614 raft_consensus.cc:515] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.762287 21614 leader_election.cc:290] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:19.762555 21219 raft_consensus.cc:359] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.763270 21221 tablet_bootstrap.cc:492] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:19.763447 21219 raft_consensus.cc:385] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.763810 21219 raft_consensus.cc:740] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.763906 21221 ts_tablet_manager.cc:1403] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.170s	user 0.011s	sys 0.004s
I20260709 12:17:19.764636 21219 consensus_queue.cc:260] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.750787 21602 raft_consensus.cc:493] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.767019 21221 raft_consensus.cc:359] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.767599 21219 ts_tablet_manager.cc:1434] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:17:19.767863 21221 raft_consensus.cc:385] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.768172 21221 raft_consensus.cc:740] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.768982 21219 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:19.766768 21602 raft_consensus.cc:515] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.768918 21221 consensus_queue.cc:260] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.773602 21221 ts_tablet_manager.cc:1434] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:17:19.775245 21221 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:19.776646 21219 tablet_bootstrap.cc:654] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.755836 21220 raft_consensus.cc:359] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.784065 21220 raft_consensus.cc:385] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.784430 21220 raft_consensus.cc:740] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.785502 21220 consensus_queue.cc:260] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.773226 21602 leader_election.cc:290] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:19.788113 21220 ts_tablet_manager.cc:1434] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 12:17:19.788173 21219 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.788862 21219 ts_tablet_manager.cc:1403] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.005s
I20260709 12:17:19.789345 21220 tablet_bootstrap.cc:492] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:19.791795 21219 raft_consensus.cc:359] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.792567 21219 raft_consensus.cc:385] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.792886 21219 raft_consensus.cc:740] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.793584 21219 consensus_queue.cc:260] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.782866 21221 tablet_bootstrap.cc:654] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.795825 21219 ts_tablet_manager.cc:1434] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:19.797010 21219 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:19.801422 21220 tablet_bootstrap.cc:654] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.801759 21140 leader_election.cc:304] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:19.803452 21219 tablet_bootstrap.cc:654] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.803643 21614 raft_consensus.cc:2804] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:19.804437 21614 raft_consensus.cc:697] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:19.805421 21614 consensus_queue.cc:237] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.809130 21221 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:19.809859 21221 ts_tablet_manager.cc:1403] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.005s
I20260709 12:17:19.813210 21221 raft_consensus.cc:359] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.813910 21221 raft_consensus.cc:385] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.814193 21221 raft_consensus.cc:740] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.814597 21219 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.814781 21221 consensus_queue.cc:260] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:19.815034 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 63928.3us
I20260709 12:17:19.817047 21221 ts_tablet_manager.cc:1434] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:17:19.817385 21220 tablet_bootstrap.cc:492] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:19.817972 21220 ts_tablet_manager.cc:1403] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.001s
I20260709 12:17:19.818102 21221 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:19.820706 21220 raft_consensus.cc:359] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.815090 21219 ts_tablet_manager.cc:1403] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:17:19.821425 21220 raft_consensus.cc:385] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.821887 21220 raft_consensus.cc:740] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.822650 21220 consensus_queue.cc:260] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:19.824710 21220 ts_tablet_manager.cc:1434] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:19.824527 21219 raft_consensus.cc:359] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.825248 21219 raft_consensus.cc:385] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.825560 21219 raft_consensus.cc:740] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.825850 21220 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:19.826295 21219 consensus_queue.cc:260] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.828441 21219 ts_tablet_manager.cc:1434] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:17:19.829547 21219 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:19.832470 21220 tablet_bootstrap.cc:654] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.840216 21602 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.841053 21602 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.845055 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:19.846197 21024 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:19.848308 20917 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:19.849617 21596 raft_consensus.cc:2804] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:19.850225 21596 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:19.850672 21596 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.865655 21221 tablet_bootstrap.cc:654] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.873210 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:19.874274 21098 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:19.887414 21598 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.872155 21219 tablet_bootstrap.cc:654] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.888119 21598 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.890780 21598 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:19.895022 21221 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:19.895780 21221 ts_tablet_manager.cc:1403] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.002s
I20260709 12:17:19.899108 21221 raft_consensus.cc:359] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.899838 21221 raft_consensus.cc:385] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.900243 21221 raft_consensus.cc:740] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.900975 20947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc706075ca374b5eb2cc3867a4f7a456" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:19.902007 20947 raft_consensus.cc:3060] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.903573 21219 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.904234 21219 ts_tablet_manager.cc:1403] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.002s
I20260709 12:17:19.901410 21221 consensus_queue.cc:260] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.909837 21221 ts_tablet_manager.cc:1434] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 12:17:19.913070 21219 raft_consensus.cc:359] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.914628 21219 raft_consensus.cc:385] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.914915 21219 raft_consensus.cc:740] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.915591 21219 consensus_queue.cc:260] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:19.918318 21219 ts_tablet_manager.cc:1434] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:17:19.919768 21219 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:19.925951 21221 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36: Bootstrap starting.
I20260709 12:17:19.928754 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc706075ca374b5eb2cc3867a4f7a456" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:19.929920 21024 raft_consensus.cc:3060] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:19.939764 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:19.940932 21170 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:19.954632 21221 tablet_bootstrap.cc:654] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:19.957232 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:19.960868 21219 tablet_bootstrap.cc:654] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:19.965387 20917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 68409.3us
I20260709 12:17:19.972761 21221 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36: No bootstrap required, opened a new log
I20260709 12:17:19.973337 21221 ts_tablet_manager.cc:1403] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.005s
I20260709 12:17:19.977161 21221 raft_consensus.cc:359] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.978191 21518 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.972745 21602 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:19.981313 21219 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: No bootstrap required, opened a new log
I20260709 12:17:19.982143 21219 ts_tablet_manager.cc:1403] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.006s
I20260709 12:17:19.985934 21219 raft_consensus.cc:359] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.986893 21219 raft_consensus.cc:385] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.987254 21219 raft_consensus.cc:740] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.988078 21219 consensus_queue.cc:260] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:19.989714 21221 raft_consensus.cc:385] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:19.990254 21221 raft_consensus.cc:740] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Initialized, Role: FOLLOWER
I20260709 12:17:19.991165 21221 consensus_queue.cc:260] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:19.994745 21221 ts_tablet_manager.cc:1434] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:17:19.996658 21219 ts_tablet_manager.cc:1434] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:17:20.002995 21602 raft_consensus.cc:493] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.003643 21602 raft_consensus.cc:515] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:20.007915 20916 leader_election.cc:304] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:19.979358 21518 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.016682 21683 raft_consensus.cc:2804] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.017263 21683 raft_consensus.cc:493] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.017696 21683 raft_consensus.cc:3060] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.027653 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:20.028440 21023 raft_consensus.cc:2468] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:20.031504 21222 tablet_bootstrap.cc:492] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:20.032112 21222 ts_tablet_manager.cc:1403] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.340s	user 0.092s	sys 0.115s
I20260709 12:17:20.032227 21168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:20.032987 21168 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:20.034343 21602 leader_election.cc:290] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:20.035455 21222 raft_consensus.cc:359] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.036250 21222 raft_consensus.cc:385] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:20.036561 21222 raft_consensus.cc:740] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
W20260709 12:17:20.037313 21139 leader_election.cc:343] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.035859 20848 catalog_manager.cc:5697] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:20.038056 21222 consensus_queue.cc:260] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.015971 21518 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:20.040707 21222 ts_tablet_manager.cc:1434] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:17:20.041896 21222 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:20.045948 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:20.046707 20946 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:20.035888 21602 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.047799 21602 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.044582 20917 leader_election.cc:304] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:20.049867 21608 raft_consensus.cc:493] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.050606 21518 raft_consensus.cc:2804] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.051384 21518 raft_consensus.cc:493] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.050674 21608 raft_consensus.cc:515] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.065685 21596 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:20.066377 21608 leader_election.cc:290] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:20.068461 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:20.068810 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:20.069440 21170 raft_consensus.cc:2468] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
W20260709 12:17:20.070200 20991 leader_election.cc:343] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.070621 21602 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:20.071228 20989 leader_election.cc:304] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:20.072197 21518 raft_consensus.cc:3060] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.073127 21608 raft_consensus.cc:2804] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.073650 21608 raft_consensus.cc:493] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.074043 21608 raft_consensus.cc:3060] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.075436 21576 raft_consensus.cc:493] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.076285 21576 raft_consensus.cc:515] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.081063 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:20.081888 21026 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:20.083158 21608 raft_consensus.cc:515] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.086146 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:20.086585 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:20.087204 21170 raft_consensus.cc:3060] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:20.088047 20991 leader_election.cc:343] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.089785 21576 leader_election.cc:290] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:20.075513 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:20.091498 21023 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.013262 20947 raft_consensus.cc:2468] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:20.101112 21170 raft_consensus.cc:2468] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:20.058133 21222 tablet_bootstrap.cc:654] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:20.110611 20917 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:20.145812 21684 raft_consensus.cc:2804] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.146219 21684 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.146597 21684 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.147796 20989 leader_election.cc:304] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:20.150218 21099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:20.151700 21620 raft_consensus.cc:2804] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.152319 21620 raft_consensus.cc:697] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:20.153250 21620 consensus_queue.cc:237] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.055398 21683 raft_consensus.cc:515] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.158475 21683 leader_election.cc:290] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
W20260709 12:17:20.159317 20916 leader_election.cc:343] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.160109 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:20.115839 21024 raft_consensus.cc:2468] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:20.127576 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:20.132292 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:20.134523 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:20.125665 21518 raft_consensus.cc:515] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:20.137560 21137 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:20.138356 21596 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:20.141350 21220 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:20.172315 21220 ts_tablet_manager.cc:1403] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.347s	user 0.110s	sys 0.132s
I20260709 12:17:20.172369 21137 leader_election.cc:304] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:20.172760 21222 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:20.173326 21222 ts_tablet_manager.cc:1403] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.000s
I20260709 12:17:20.173931 21614 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.174559 21614 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.175076 21614 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.175299 21220 raft_consensus.cc:359] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.176162 21220 raft_consensus.cc:385] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:20.141789 21608 leader_election.cc:290] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:20.176021 21222 raft_consensus.cc:359] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.176458 21312 raft_consensus.cc:2804] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.177321 21312 raft_consensus.cc:697] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:20.178234 21312 consensus_queue.cc:237] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.183708 21608 raft_consensus.cc:493] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.184511 21608 raft_consensus.cc:515] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.176456 21220 raft_consensus.cc:740] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
W20260709 12:17:20.186342 20916 leader_election.cc:343] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.176785 21222 raft_consensus.cc:385] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:20.160777 21097 raft_consensus.cc:3060] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.186887 21222 raft_consensus.cc:740] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
W20260709 12:17:20.187700 20916 leader_election.cc:343] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:20.188753 20916 leader_election.cc:343] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.188545 21602 raft_consensus.cc:493] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.192037 21602 raft_consensus.cc:515] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.195572 21602 leader_election.cc:290] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
W20260709 12:17:20.198679 21139 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 161398us
I20260709 12:17:20.200832 20914 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:20.189767 21220 consensus_queue.cc:260] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.189428 21222 consensus_queue.cc:260] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.210474 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:20.211179 21174 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 0.
I20260709 12:17:20.213379 21220 ts_tablet_manager.cc:1434] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:17:20.208832 21633 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.220130 21633 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.220512 21633 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.223181 21097 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:20.228084 21168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:20.235738 20916 leader_election.cc:304] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:20.236747 21685 raft_consensus.cc:2804] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.237268 21685 raft_consensus.cc:697] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:20.238114 21685 consensus_queue.cc:237] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.240031 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
W20260709 12:17:20.242554 20916 leader_election.cc:343] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.243968 21220 tablet_bootstrap.cc:492] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:20.247723 21222 ts_tablet_manager.cc:1434] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.074s	user 0.009s	sys 0.000s
I20260709 12:17:20.249505 21608 leader_election.cc:290] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:20.249612 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:20.250470 21171 raft_consensus.cc:3060] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.250312 21633 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.252197 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b0a17978446c6bee0b8db24be9e49" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:20.253686 21633 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
W20260709 12:17:20.254278 20989 leader_election.cc:343] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.254503 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:20.256574 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:20.258262 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b0a17978446c6bee0b8db24be9e49" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:20.216514 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a820defdda646daaa5ffa1d6aba18f1" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:20.245954 21614 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.265509 21222 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:20.260783 21097 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.273682 21024 raft_consensus.cc:2393] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:20.266388 20946 raft_consensus.cc:2468] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 0.
I20260709 12:17:20.277932 21518 leader_election.cc:290] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:20.279227 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:20.279942 21024 raft_consensus.cc:3060] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.280861 21023 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:20.282338 20917 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:20.283315 20989 leader_election.cc:304] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:20.283638 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:20.285869 21596 raft_consensus.cc:2804] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.286428 21596 raft_consensus.cc:697] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
W20260709 12:17:20.287111 20916 leader_election.cc:343] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.287364 21596 consensus_queue.cc:237] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:20.285933 21620 raft_consensus.cc:2804] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.294317 21620 raft_consensus.cc:493] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.294687 21620 raft_consensus.cc:3060] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.299307 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:20.300137 21174 raft_consensus.cc:2393] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8997852a67a4e678dfad30c76a76964 in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:20.303495 21171 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:20.306103 21097 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:20.306447 21684 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.309334 21220 tablet_bootstrap.cc:654] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:20.328429 21684 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:20.329166 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:20.332430 21023 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.310029 21614 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:20.310294 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:20.314241 21222 tablet_bootstrap.cc:654] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:20.349938 20946 raft_consensus.cc:2393] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:20.339725 20846 catalog_manager.cc:5697] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:20.335189 21433 consensus_queue.cc:1048] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:17:20.362404 21620 raft_consensus.cc:515] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:20.369382 21139 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 69381.3us
W20260709 12:17:20.374557 20914 leader_election.cc:343] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.375214 20914 leader_election.cc:304] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989
I20260709 12:17:20.378759 21063 leader_election.cc:304] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989
I20260709 12:17:20.379348 20849 catalog_manager.cc:5697] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:20.382730 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:20.383476 21174 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.398731 21174 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
I20260709 12:17:20.412850 20947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b0a17978446c6bee0b8db24be9e49" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:20.413407 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:20.413599 20947 raft_consensus.cc:3060] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:20.437078 21066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 119487us
I20260709 12:17:20.379632 21601 raft_consensus.cc:2749] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:20.439625 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b0a17978446c6bee0b8db24be9e49" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:20.438320 20916 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:20.418484 21587 raft_consensus.cc:3060] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.418128 21620 leader_election.cc:290] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:20.395782 21024 raft_consensus.cc:2468] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
W20260709 12:17:20.450469 21063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 64449us
W20260709 12:17:20.450807 20989 leader_election.cc:343] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.451902 21023 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 1.
W20260709 12:17:20.463738 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 69087us
I20260709 12:17:20.450266 21683 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.469132 20914 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:20.454542 21097 raft_consensus.cc:2393] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate c8997852a67a4e678dfad30c76a76964 in this term.
I20260709 12:17:20.476996 21701 raft_consensus.cc:2804] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.477559 21701 raft_consensus.cc:697] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:20.478422 21701 consensus_queue.cc:237] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.478964 21587 raft_consensus.cc:2749] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:20.483358 20917 leader_election.cc:304] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:20.508462 21220 tablet_bootstrap.cc:492] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:20.509992 21220 ts_tablet_manager.cc:1403] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.266s	user 0.010s	sys 0.023s
I20260709 12:17:20.525138 21220 raft_consensus.cc:359] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.527607 21220 raft_consensus.cc:385] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:20.527930 21220 raft_consensus.cc:740] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:20.528733 21220 consensus_queue.cc:260] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:20.541491 21139 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:20.543216 21692 raft_consensus.cc:2749] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:20.547164 21220 ts_tablet_manager.cc:1434] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:17:20.500581 21685 raft_consensus.cc:2804] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.551564 21685 raft_consensus.cc:697] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:20.561429 21220 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:20.576432 21222 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:20.577018 21222 ts_tablet_manager.cc:1403] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.326s	user 0.025s	sys 0.009s
I20260709 12:17:20.577775 21312 consensus_queue.cc:1048] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:20.578694 21095 raft_consensus.cc:3060] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.579702 21220 tablet_bootstrap.cc:654] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:20.579964 21222 raft_consensus.cc:359] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.580698 21222 raft_consensus.cc:385] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:20.580974 21222 raft_consensus.cc:740] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:20.581722 21222 consensus_queue.cc:260] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.584272 21222 ts_tablet_manager.cc:1434] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:20.585384 21222 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989: Bootstrap starting.
I20260709 12:17:20.591934 20947 raft_consensus.cc:2468] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:20.552804 21685 consensus_queue.cc:237] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:20.594100 21222 tablet_bootstrap.cc:654] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:20.607429 21066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 133473us
W20260709 12:17:20.610520 20917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 127273us
W20260709 12:17:20.742020 20897 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:20.619372 21683 raft_consensus.cc:697] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
W20260709 12:17:20.630724 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 161656us
I20260709 12:17:20.729825 21702 raft_consensus.cc:493] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.749056 21702 raft_consensus.cc:515] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.751719 21702 leader_election.cc:290] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:20.756026 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:20.756840 21683 consensus_queue.cc:237] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.759171 20846 catalog_manager.cc:5697] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:20.761116 21139 leader_election.cc:343] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.773137 21220 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:20.773885 21220 ts_tablet_manager.cc:1403] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.213s	user 0.017s	sys 0.000s
W20260709 12:17:20.783612 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 67521.7us
I20260709 12:17:20.791031 20989 leader_election.cc:304] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: eb45c1a4fe5341368530840de4608989
I20260709 12:17:20.777446 21220 raft_consensus.cc:359] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:20.793785 21220 raft_consensus.cc:385] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:20.793025 21714 raft_consensus.cc:2804] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:20.798228 21701 consensus_queue.cc:1048] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:20.808311 20946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:20.826085 21633 consensus_queue.cc:1048] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:20.832500 21220 raft_consensus.cc:740] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:20.838891 21518 consensus_queue.cc:1048] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:17:20.836076 21714 raft_consensus.cc:697] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:20.853966 21714 consensus_queue.cc:237] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.839679 21220 consensus_queue.cc:260] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:20.857666 21713 consensus_queue.cc:1048] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:20.857893 21692 consensus_queue.cc:1048] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:20.866500 21608 consensus_queue.cc:1048] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:20.882140 20914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 47631.9us
I20260709 12:17:20.838053 20946 raft_consensus.cc:2468] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 0.
I20260709 12:17:20.887091 21222 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989: No bootstrap required, opened a new log
I20260709 12:17:20.888146 21222 ts_tablet_manager.cc:1403] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989: Time spent bootstrapping tablet: real 0.303s	user 0.015s	sys 0.007s
I20260709 12:17:20.894213 21137 leader_election.cc:304] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:20.899973 21692 raft_consensus.cc:2804] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:20.900508 21692 raft_consensus.cc:493] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.900978 21692 raft_consensus.cc:3060] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:20.901163 21433 consensus_queue.cc:1048] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:20.908857 20846 catalog_manager.cc:5697] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:20.926019 20846 catalog_manager.cc:5697] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:20.934965 21220 ts_tablet_manager.cc:1434] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.161s	user 0.011s	sys 0.002s
I20260709 12:17:20.931816 20846 catalog_manager.cc:5697] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:20.935876 20917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 98590.1us
I20260709 12:17:20.937930 20846 catalog_manager.cc:5697] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:20.939576 21222 raft_consensus.cc:359] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.940312 21222 raft_consensus.cc:385] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:20.940562 21222 raft_consensus.cc:740] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Initialized, Role: FOLLOWER
I20260709 12:17:20.943636 21220 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:20.942572 21222 consensus_queue.cc:260] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.956647 21692 raft_consensus.cc:515] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:20.966009 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:20.969029 21222 ts_tablet_manager.cc:1434] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989: Time spent starting tablet: real 0.080s	user 0.005s	sys 0.003s
W20260709 12:17:20.984227 21139 leader_election.cc:343] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:20.968963 20849 catalog_manager.cc:5697] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:20.990259 21614 raft_consensus.cc:493] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.991077 21614 raft_consensus.cc:515] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:20.996206 21091 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:21.014824 21692 leader_election.cc:290] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:21.014668 20945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b0a17978446c6bee0b8db24be9e49" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:21.018060 20945 raft_consensus.cc:2393] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 1: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:21.019711 21614 leader_election.cc:290] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:21.024788 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e5b0a17978446c6bee0b8db24be9e49" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:21.037806 21140 leader_election.cc:304] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:21.038832 21220 tablet_bootstrap.cc:654] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:21.053694 21702 raft_consensus.cc:3060] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:21.091849 21701 consensus_queue.cc:1048] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:17:21.102514 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:21.103231 20944 raft_consensus.cc:3060] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:21.107986 21702 raft_consensus.cc:2749] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:21.124280 21724 consensus_queue.cc:1048] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:21.132045 21220 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:21.132691 21220 ts_tablet_manager.cc:1403] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.189s	user 0.010s	sys 0.003s
I20260709 12:17:21.139853 21724 consensus_queue.cc:1048] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:21.142488 21685 consensus_queue.cc:1048] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:21.170008 20944 raft_consensus.cc:2468] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:21.176666 21220 raft_consensus.cc:359] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:21.179804 21220 raft_consensus.cc:385] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:21.180190 21220 raft_consensus.cc:740] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:21.181032 21220 consensus_queue.cc:260] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:21.183866 21220 ts_tablet_manager.cc:1434] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:17:21.186648 21220 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e: Bootstrap starting.
I20260709 12:17:21.187039 21137 leader_election.cc:304] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:21.187978 21702 raft_consensus.cc:2804] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:21.188529 21702 raft_consensus.cc:697] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:21.189458 21702 consensus_queue.cc:237] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:21.197758 20916 consensus_peers.cc:597] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:21.202399 21220 tablet_bootstrap.cc:654] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:21.230608 21707 consensus_queue.cc:1048] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:21.255044 21220 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e: No bootstrap required, opened a new log
I20260709 12:17:21.255656 21220 ts_tablet_manager.cc:1403] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.006s
I20260709 12:17:21.275553 21220 raft_consensus.cc:359] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:21.276281 21220 raft_consensus.cc:385] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:21.276554 21220 raft_consensus.cc:740] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Initialized, Role: FOLLOWER
I20260709 12:17:21.277175 21220 consensus_queue.cc:260] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:21.279408 21220 ts_tablet_manager.cc:1434] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:17:21.286706 20846 catalog_manager.cc:5697] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 1 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:21.296417 21683 consensus_queue.cc:1048] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:21.319363 21714 consensus_queue.cc:1048] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:21.386273 21713 consensus_queue.cc:1048] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:21.442098 21704 raft_consensus.cc:493] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.442687 21704 raft_consensus.cc:515] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:21.445039 21704 leader_election.cc:290] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:21.446775 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:21.447190 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:21.448966 21063 leader_election.cc:304] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:21.449954 21704 raft_consensus.cc:2749] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:21.536052 21576 raft_consensus.cc:493] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.536537 21576 raft_consensus.cc:515] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:21.538630 21576 leader_election.cc:290] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:21.539686 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:21.541018 20944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16b20f7ffdc94676bd1cbade62c4dcc5" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:21.541895 20944 raft_consensus.cc:2393] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 1: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:21.542830 21063 leader_election.cc:304] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:21.543452 21576 raft_consensus.cc:3060] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:21.549187 21576 raft_consensus.cc:2749] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:21.593099 21692 consensus_queue.cc:1048] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:21.611449 21692 consensus_queue.cc:1048] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:21.679328 21093 raft_consensus.cc:3060] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.064630 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
W20260709 12:17:22.079903 21063 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:17:22.084983 21063 consensus_peers.cc:597] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:22.109447 21063 consensus_peers.cc:597] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:22.125193 21137 consensus_peers.cc:597] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.126979 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:17:22.127297 21137 consensus_peers.cc:597] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.128209 10011 tablet_replica.cc:333] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.129328 10011 raft_consensus.cc:2243] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.130231 10011 raft_consensus.cc:2272] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.133565 10011 tablet_replica.cc:333] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.134290 10011 raft_consensus.cc:2243] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.134755 10011 raft_consensus.cc:2272] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.136701 10011 tablet_replica.cc:333] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.137434 10011 raft_consensus.cc:2243] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.137893 10011 raft_consensus.cc:2272] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.139734 10011 tablet_replica.cc:333] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.140475 10011 raft_consensus.cc:2243] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.140980 10011 raft_consensus.cc:2272] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.143340 10011 tablet_replica.cc:333] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.144112 10011 raft_consensus.cc:2243] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.144603 10011 raft_consensus.cc:2272] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.144881 20989 consensus_peers.cc:597] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.147039 10011 tablet_replica.cc:333] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.147774 10011 raft_consensus.cc:2243] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.148729 10011 raft_consensus.cc:2272] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.149798 20989 consensus_peers.cc:597] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.150785 10011 tablet_replica.cc:333] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.151443 10011 raft_consensus.cc:2243] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.151868 10011 raft_consensus.cc:2272] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.153777 10011 tablet_replica.cc:333] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.154392 10011 raft_consensus.cc:2243] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.155279 10011 raft_consensus.cc:2272] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.157441 10011 tablet_replica.cc:333] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.158150 10011 raft_consensus.cc:2243] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.158672 10011 raft_consensus.cc:2272] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.161137 10011 tablet_replica.cc:333] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.162053 10011 raft_consensus.cc:2243] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.162990 10011 raft_consensus.cc:2272] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.165851 10011 tablet_replica.cc:333] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.166749 10011 raft_consensus.cc:2243] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.167340 10011 raft_consensus.cc:2272] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.170030 10011 tablet_replica.cc:333] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.170990 10011 raft_consensus.cc:2243] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.171998 10011 raft_consensus.cc:2272] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.174474 10011 tablet_replica.cc:333] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.175243 10011 raft_consensus.cc:2243] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.175760 10011 raft_consensus.cc:2272] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.178755 10011 tablet_replica.cc:333] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.179658 10011 raft_consensus.cc:2243] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.180640 10011 raft_consensus.cc:2272] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.183640 10011 tablet_replica.cc:333] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.184504 10011 raft_consensus.cc:2243] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.185531 10011 raft_consensus.cc:2272] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.188320 10011 tablet_replica.cc:333] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.189073 10011 raft_consensus.cc:2243] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.190052 10011 raft_consensus.cc:2272] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.190845 21137 consensus_peers.cc:597] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:22.191483 20989 consensus_peers.cc:597] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.193874 10011 tablet_replica.cc:333] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.194759 10011 raft_consensus.cc:2243] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.195322 10011 raft_consensus.cc:2272] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.198342 10011 tablet_replica.cc:333] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.199084 10011 raft_consensus.cc:2243] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.199608 10011 raft_consensus.cc:2272] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.200094 21137 consensus_peers.cc:597] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.201936 10011 tablet_replica.cc:333] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.202696 10011 raft_consensus.cc:2243] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:17:22.203017 21063 consensus_peers.cc:597] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.203671 10011 raft_consensus.cc:2272] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.206254 10011 tablet_replica.cc:333] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.206848 10011 raft_consensus.cc:2243] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.207336 10011 raft_consensus.cc:2272] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.210186 10011 tablet_replica.cc:333] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.210950 10011 raft_consensus.cc:2243] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.211441 10011 raft_consensus.cc:2272] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.213584 10011 tablet_replica.cc:333] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.214502 10011 raft_consensus.cc:2243] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.215210 10011 raft_consensus.cc:2272] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.217207 10011 tablet_replica.cc:333] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.217883 10011 raft_consensus.cc:2243] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.218353 10011 raft_consensus.cc:2272] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.220145 10011 tablet_replica.cc:333] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.220709 10011 raft_consensus.cc:2243] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.221473 10011 raft_consensus.cc:2272] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.223342 10011 tablet_replica.cc:333] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.223966 10011 raft_consensus.cc:2243] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.224818 10011 raft_consensus.cc:2272] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.227036 10011 tablet_replica.cc:333] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.227624 10011 raft_consensus.cc:2243] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.228124 10011 raft_consensus.cc:2272] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.230654 10011 tablet_replica.cc:333] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.231415 10011 raft_consensus.cc:2243] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.232471 10011 raft_consensus.cc:2272] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.235406 10011 tablet_replica.cc:333] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.236063 10011 raft_consensus.cc:2243] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:22.236464 21063 consensus_peers.cc:597] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.236541 10011 raft_consensus.cc:2272] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.238936 10011 tablet_replica.cc:333] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.239663 10011 raft_consensus.cc:2243] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.240118 10011 raft_consensus.cc:2272] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.242586 10011 tablet_replica.cc:333] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.243129 10011 raft_consensus.cc:2243] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.243870 10011 raft_consensus.cc:2272] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.245841 10011 tablet_replica.cc:333] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.246438 10011 raft_consensus.cc:2243] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.247377 10011 raft_consensus.cc:2272] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.249941 10011 tablet_replica.cc:333] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.251266 10011 raft_consensus.cc:2243] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.251989 10011 raft_consensus.cc:2272] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.253811 10011 tablet_replica.cc:333] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.254453 10011 raft_consensus.cc:2243] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.254998 10011 raft_consensus.cc:2272] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.257200 10011 tablet_replica.cc:333] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.257982 10011 raft_consensus.cc:2243] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.258458 10011 raft_consensus.cc:2272] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.260380 21137 consensus_peers.cc:597] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.261268 10011 tablet_replica.cc:333] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.262200 10011 raft_consensus.cc:2243] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:22.262244 20989 consensus_peers.cc:597] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.262759 10011 raft_consensus.cc:2272] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.264842 10011 tablet_replica.cc:333] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.265442 10011 raft_consensus.cc:2243] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.265962 10011 raft_consensus.cc:2272] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.267971 10011 tablet_replica.cc:333] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.268555 10011 raft_consensus.cc:2243] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.269389 10011 raft_consensus.cc:2272] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.271772 10011 tablet_replica.cc:333] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.272506 10011 raft_consensus.cc:2243] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.273442 10011 raft_consensus.cc:2272] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.273958 21137 consensus_peers.cc:597] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.276003 10011 tablet_replica.cc:333] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.276834 10011 raft_consensus.cc:2243] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.277436 10011 raft_consensus.cc:2272] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.277565 20989 consensus_peers.cc:597] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.279649 10011 tablet_replica.cc:333] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.280211 10011 raft_consensus.cc:2243] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.280921 10011 raft_consensus.cc:2272] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.282902 10011 tablet_replica.cc:333] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.283437 10011 raft_consensus.cc:2243] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.283843 10011 raft_consensus.cc:2272] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.285771 10011 tablet_replica.cc:333] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.286274 10011 raft_consensus.cc:2243] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.286705 10011 raft_consensus.cc:2272] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.288771 10011 tablet_replica.cc:333] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.289441 10011 raft_consensus.cc:2243] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:22.289924 10011 raft_consensus.cc:2272] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.290934 20989 consensus_peers.cc:597] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.292557 10011 tablet_replica.cc:333] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.293092 10011 raft_consensus.cc:2243] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.294056 10011 raft_consensus.cc:2272] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:22.295893 10011 tablet_replica.cc:333] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: stopping tablet replica
I20260709 12:17:22.296324 10011 raft_consensus.cc:2243] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:22.297097 10011 raft_consensus.cc:2272] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:22.307061 21063 consensus_peers.cc:597] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.368498 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
W20260709 12:17:22.391309 20989 consensus_peers.cc:597] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:22.418954 20989 consensus_peers.cc:597] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.454659 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:22.463470 21753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:22.463491 21752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:22.464511 21755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:22.470109 10011 server_base.cc:1061] running on GCE node
W20260709 12:17:22.472152 21137 consensus_peers.cc:597] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.475319 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:22.475719 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:22.475912 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599442475898 us; error 0 us; skew 500 ppm
I20260709 12:17:22.476704 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260709 12:17:22.479012 21137 consensus_peers.cc:597] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.480196 10011 webserver.cc:533] Webserver started at http://127.9.198.193:43527/ using document root <none> and password file <none>
I20260709 12:17:22.480772 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:22.480935 10011 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:17:22.484807 10011 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.000s
W20260709 12:17:22.490298 20989 consensus_peers.cc:597] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:22.494244 21137 consensus_peers.cc:597] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.496250 21761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:22.497386 10011 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:17:22.497825 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "c8997852a67a4e678dfad30c76a76964"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-glnm"
I20260709 12:17:22.498373 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:17:22.519291 20989 consensus_peers.cc:597] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.524987 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:22.526515 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:22.528136 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:22.533178 21770 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:17:22.667774 20989 consensus_peers.cc:597] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:22.826948 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:17:22.827389 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.295s	user 0.008s	sys 0.001s
I20260709 12:17:22.827764 10011 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:17:22.835700 21770 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:22.866827 21770 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.867801 21770 tablet_bootstrap.cc:492] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:22.868470 21770 ts_tablet_manager.cc:1403] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.008s
I20260709 12:17:22.870962 21770 raft_consensus.cc:359] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:22.871615 21770 raft_consensus.cc:740] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.872346 21770 consensus_queue.cc:260] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:22.887579 21770 ts_tablet_manager.cc:1434] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.016s
I20260709 12:17:22.888473 21770 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:22.902329 21770 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.903249 21770 tablet_bootstrap.cc:492] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:22.903891 21770 ts_tablet_manager.cc:1403] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:17:22.906222 21770 raft_consensus.cc:359] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:22.906894 21770 raft_consensus.cc:740] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.907506 21770 consensus_queue.cc:260] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:22.909212 21770 ts_tablet_manager.cc:1434] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:22.910017 21770 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:22.923822 21770 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.924748 21770 tablet_bootstrap.cc:492] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:22.925398 21770 ts_tablet_manager.cc:1403] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 12:17:22.927790 21770 raft_consensus.cc:359] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:22.928453 21770 raft_consensus.cc:740] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.929071 21770 consensus_queue.cc:260] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:22.930925 21770 ts_tablet_manager.cc:1434] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:22.931717 21770 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:22.946485 21770 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.947471 21770 tablet_bootstrap.cc:492] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:22.948129 21770 ts_tablet_manager.cc:1403] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 12:17:22.950634 21770 raft_consensus.cc:359] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:22.951292 21770 raft_consensus.cc:740] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.951954 21770 consensus_queue.cc:260] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:22.953894 21770 ts_tablet_manager.cc:1434] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.004s
I20260709 12:17:22.954663 21770 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:22.987592 21770 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:22.989114 21770 tablet_bootstrap.cc:492] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:22.989869 21770 ts_tablet_manager.cc:1403] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.019s
I20260709 12:17:22.993113 21770 raft_consensus.cc:359] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:22.994059 21770 raft_consensus.cc:740] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.994961 21770 consensus_queue.cc:260] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:22.999686 21770 ts_tablet_manager.cc:1434] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:17:23.001072 21770 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.037835 21770 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.038892 21770 tablet_bootstrap.cc:492] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.039588 21770 ts_tablet_manager.cc:1403] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.023s
I20260709 12:17:23.042151 21770 raft_consensus.cc:359] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.042824 21770 raft_consensus.cc:740] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.043530 21770 consensus_queue.cc:260] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.045410 21770 ts_tablet_manager.cc:1434] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:17:23.046182 21770 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.061295 21770 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.062713 21770 tablet_bootstrap.cc:492] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.063630 21770 ts_tablet_manager.cc:1403] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:17:23.066982 21770 raft_consensus.cc:359] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.067657 21770 raft_consensus.cc:740] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.068333 21770 consensus_queue.cc:260] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.070397 21770 ts_tablet_manager.cc:1434] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:23.071173 21770 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.098405 21770 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.099349 21770 tablet_bootstrap.cc:492] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.100001 21770 ts_tablet_manager.cc:1403] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.011s
I20260709 12:17:23.102406 21770 raft_consensus.cc:359] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.103080 21770 raft_consensus.cc:740] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.103713 21770 consensus_queue.cc:260] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.105952 21770 ts_tablet_manager.cc:1434] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:17:23.106735 21770 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.125972 21770 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.126760 21770 tablet_bootstrap.cc:492] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.127322 21770 ts_tablet_manager.cc:1403] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 12:17:23.129352 21770 raft_consensus.cc:359] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.129957 21770 raft_consensus.cc:740] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.130524 21770 consensus_queue.cc:260] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.132097 21770 ts_tablet_manager.cc:1434] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:17:23.132756 21770 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.141212 10011 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:17:23.141692 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.314s	user 0.290s	sys 0.004s
I20260709 12:17:23.147753 21770 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.148928 21770 tablet_bootstrap.cc:492] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.149683 21770 ts_tablet_manager.cc:1403] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.004s
I20260709 12:17:23.152459 21770 raft_consensus.cc:359] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.153235 21770 raft_consensus.cc:740] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.154320 21770 consensus_queue.cc:260] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.157284 21770 ts_tablet_manager.cc:1434] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:17:23.158334 21770 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.178049 21770 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.179126 21770 tablet_bootstrap.cc:492] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.179885 21770 ts_tablet_manager.cc:1403] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 12:17:23.182597 21770 raft_consensus.cc:359] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.183377 21770 raft_consensus.cc:740] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.184197 21770 consensus_queue.cc:260] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.186612 21770 ts_tablet_manager.cc:1434] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:23.187548 21770 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.211798 21746 raft_consensus.cc:493] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.212507 21746 raft_consensus.cc:515] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.214776 21746 leader_election.cc:290] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.216934 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
W20260709 12:17:23.221282 20989 leader_election.cc:336] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:23.221889 20989 leader_election.cc:304] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:23.222946 21770 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.224041 21770 tablet_bootstrap.cc:492] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.224140 21746 raft_consensus.cc:2749] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:23.226313 21770 ts_tablet_manager.cc:1403] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.039s	user 0.023s	sys 0.001s
I20260709 12:17:23.229673 21770 raft_consensus.cc:359] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.230542 21770 raft_consensus.cc:740] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.231769 21770 consensus_queue.cc:260] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.234508 21770 ts_tablet_manager.cc:1434] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:23.235610 21770 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.242091 21762 raft_consensus.cc:493] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.242780 21762 raft_consensus.cc:515] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.244751 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:41243
I20260709 12:17:23.244882 21844 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:41243 every 8 connection(s)
I20260709 12:17:23.245729 21773 raft_consensus.cc:493] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.246326 21773 raft_consensus.cc:515] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.246696 21762 leader_election.cc:290] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.249619 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.251006 21746 raft_consensus.cc:493] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.251571 21746 raft_consensus.cc:515] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.253196 21773 leader_election.cc:290] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.253875 21746 leader_election.cc:290] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:23.254906 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.256014 21026 raft_consensus.cc:2468] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:23.257084 21140 leader_election.cc:304] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:23.258319 21773 raft_consensus.cc:2804] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.258754 21773 raft_consensus.cc:493] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.258909 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:23.259202 21773 raft_consensus.cc:3060] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.267411 21773 raft_consensus.cc:515] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.270452 21762 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.304141 21692 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.304009 21762 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.304734 21692 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.306264 21762 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.277279 21746 raft_consensus.cc:493] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.307017 21746 raft_consensus.cc:515] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.307703 21692 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.309235 21746 leader_election.cc:290] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:23.285027 21772 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.310115 21772 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.310789 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:23.311379 21026 raft_consensus.cc:3060] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.312245 21772 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.285919 21747 raft_consensus.cc:493] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.312968 21747 raft_consensus.cc:515] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.315197 21747 leader_election.cc:290] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:23.320200 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:23.321221 21026 raft_consensus.cc:2468] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:23.323112 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:23.324517 21140 leader_election.cc:304] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:23.325066 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:23.325619 21772 raft_consensus.cc:2804] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:23.326339 21772 raft_consensus.cc:697] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:23.327566 21704 raft_consensus.cc:493] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.327455 21772 consensus_queue.cc:237] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.328167 21704 raft_consensus.cc:515] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.330394 21704 leader_election.cc:290] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.295478 21774 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.331367 21774 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.331672 21692 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.332469 21692 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.340346 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:23.325764 21098 raft_consensus.cc:2468] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:23.361284 21704 raft_consensus.cc:493] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.382339 21704 raft_consensus.cc:515] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.385128 21704 leader_election.cc:290] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.387221 20991 leader_election.cc:304] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:23.388526 21747 raft_consensus.cc:2804] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.389011 21747 raft_consensus.cc:493] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.389468 21747 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.360404 21762 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.394941 21762 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.360410 21093 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:23.359534 21774 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.365275 21746 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.398249 21746 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.401995 21746 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.379113 21773 leader_election.cc:290] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.405078 21773 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.426841 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.428004 21812 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:23.427896 21773 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.417827 21819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.420608 21814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.429410 21814 raft_consensus.cc:2468] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.422714 21816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.424088 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.431200 21815 raft_consensus.cc:2468] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.424731 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.424993 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.425623 21813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.426033 21818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.362396 21692 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.436664 21704 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.437315 21704 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.439088 20989 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:23.439584 21704 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:23.440253 20989 leader_election.cc:343] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:23.441105 20989 leader_election.cc:343] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.441673 20989 leader_election.cc:304] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:23.444582 21771 raft_consensus.cc:2749] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:23.442299 21774 raft_consensus.cc:2804] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:23.446736 20989 leader_election.cc:343] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.451730 20989 leader_election.cc:304] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
W20260709 12:17:23.453365 20989 leader_election.cc:343] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.455621 21851 raft_consensus.cc:2749] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:23.462182 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.451267 21774 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
W20260709 12:17:23.466480 21063 leader_election.cc:343] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.467011 21063 leader_election.cc:304] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:23.478725 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.479653 21168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.480370 21168 raft_consensus.cc:2468] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.468142 21063 leader_election.cc:304] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:23.469722 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.470920 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.471974 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.481226 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.475848 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:23.476311 21855 raft_consensus.cc:2749] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:23.485424 21098 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:23.466692 21774 raft_consensus.cc:3060] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.473618 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.477998 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.482283 21174 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.468901 21772 raft_consensus.cc:493] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.497305 21139 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:23.481463 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:23.482065 21773 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.502660 21770 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:23.510053 21770 tablet_bootstrap.cc:492] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.511224 21770 ts_tablet_manager.cc:1403] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.276s	user 0.066s	sys 0.078s
I20260709 12:17:23.515659 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.513908 20849 catalog_manager.cc:5697] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 2 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.495846 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.500562 21850 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.504303 21173 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.498051 21813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.504491 21771 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.525966 21856 raft_consensus.cc:493] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.542636 21856 raft_consensus.cc:515] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.527235 21855 raft_consensus.cc:2804] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.509609 21772 raft_consensus.cc:515] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.544299 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.538944 21850 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:23.550132 21139 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 52880.5us
I20260709 12:17:23.550791 21855 raft_consensus.cc:493] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.551671 21863 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.555909 21139 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:23.546257 21762 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.547601 21851 raft_consensus.cc:493] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.562673 21171 raft_consensus.cc:2468] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:23.557909 21855 raft_consensus.cc:3060] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.545089 21771 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.565374 21856 leader_election.cc:290] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.559975 21863 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.573547 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d7127b20aa4482ae857ddb5430b70e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.574115 21863 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.576655 21770 raft_consensus.cc:359] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.582649 21772 leader_election.cc:290] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.583758 21850 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.585691 21746 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.566072 21859 raft_consensus.cc:493] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.589121 21770 raft_consensus.cc:740] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.602696 21770 consensus_queue.cc:260] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.604086 20989 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:23.605484 21770 ts_tablet_manager.cc:1434] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 12:17:23.599964 21856 raft_consensus.cc:493] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.640198 21770 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:23.601126 21850 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
W20260709 12:17:23.639550 21137 leader_election.cc:343] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.642865 21137 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
W20260709 12:17:23.644007 21137 leader_election.cc:343] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:23.644877 21137 leader_election.cc:343] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:23.634467 21140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 52820.2us
I20260709 12:17:23.564401 21851 raft_consensus.cc:515] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.601852 21746 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.666313 21870 raft_consensus.cc:2749] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:23.667454 21746 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:23.667577 21137 leader_election.cc:343] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.667737 21870 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.668569 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.669126 21867 raft_consensus.cc:2804] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.638652 21856 raft_consensus.cc:515] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.669567 21867 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.669992 21867 raft_consensus.cc:3060] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.671160 21856 leader_election.cc:290] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.603324 21859 raft_consensus.cc:515] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.686996 21747 raft_consensus.cc:515] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.685237 21859 leader_election.cc:290] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:23.668707 21137 leader_election.cc:343] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.693603 21137 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:23.641566 21853 raft_consensus.cc:2804] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.668342 21870 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:23.636497 21864 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:23.700366 21137 leader_election.cc:343] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.706398 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.707419 21026 raft_consensus.cc:2468] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.708778 21066 leader_election.cc:304] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:23.710318 21762 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.714388 21762 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.714917 21856 raft_consensus.cc:2804] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.715361 21856 raft_consensus.cc:493] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.715225 21017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.715927 21856 raft_consensus.cc:3060] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.716605 21762 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.720563 21873 raft_consensus.cc:493] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.725558 21816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.698837 21853 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.731328 21853 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.732622 21856 raft_consensus.cc:515] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.735306 21771 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.703527 21864 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.742574 21864 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.703230 21020 raft_consensus.cc:2468] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:23.704921 21774 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.752101 21774 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.757929 21865 raft_consensus.cc:2749] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:23.721333 21873 raft_consensus.cc:515] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.647802 21850 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:23.777576 21139 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 221730us
W20260709 12:17:23.779701 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.196:38535, user_credentials={real_user=slave}} blocked reactor thread for 71489.9us
I20260709 12:17:23.764096 21856 leader_election.cc:290] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.793820 21140 leader_election.cc:304] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:23.778637 21747 leader_election.cc:290] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:23.795586 21871 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.796118 21871 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.796921 21851 leader_election.cc:290] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.798489 21871 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.799503 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.800151 21171 raft_consensus.cc:2468] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:23.801339 21871 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.801911 21871 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:23.786031 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.802800 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.803066 21812 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:23.803514 21169 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:23.804026 21871 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:23.804390 21813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:23.804633 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:23.805197 21173 raft_consensus.cc:3060] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.777334 21772 raft_consensus.cc:493] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.805860 21772 raft_consensus.cc:515] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.807029 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.807693 21026 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.809046 21870 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.809300 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.810159 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:23.810737 21026 raft_consensus.cc:3060] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.811456 21813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.814626 20989 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:23.816063 21879 raft_consensus.cc:2804] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.816486 21879 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.816735 21850 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.816833 21879 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.819669 21846 heartbeater.cc:344] Connected to a master server at 127.9.198.254:35489
I20260709 12:17:23.820104 21066 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:23.820325 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.820140 21846 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:23.821266 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38019ba094624915aa5f69bc244bc955" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:23.821906 21817 raft_consensus.cc:3060] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.822453 21859 raft_consensus.cc:2804] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.822870 21859 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.823264 21859 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.821952 21846 heartbeater.cc:507] Master 127.9.198.254:35489 requested a full tablet report, sending...
W20260709 12:17:23.830935 20989 leader_election.cc:343] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.832010 20989 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
W20260709 12:17:23.832923 20989 leader_election.cc:343] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.833950 21880 raft_consensus.cc:2804] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.836244 21873 leader_election.cc:290] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:23.835961 21880 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.836943 21880 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.843154 21772 leader_election.cc:290] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:23.843886 21063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 375781us
W20260709 12:17:23.844732 21063 leader_election.cc:343] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.845105 21173 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:23.847266 21816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.847890 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
W20260709 12:17:23.848999 20989 leader_election.cc:343] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.850375 21063 leader_election.cc:304] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:23.854279 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.846606 21813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.856714 21859 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.853477 21026 raft_consensus.cc:2468] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:23.841058 21855 raft_consensus.cc:515] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.872718 21066 leader_election.cc:304] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:23.880574 21810 raft_consensus.cc:2468] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
W20260709 12:17:23.872130 21063 leader_election.cc:343] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.883693 21704 raft_consensus.cc:2804] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:23.888788 21704 raft_consensus.cc:697] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:23.864753 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.871959 21816 raft_consensus.cc:2468] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:23.885023 20989 leader_election.cc:304] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
W20260709 12:17:23.891474 20989 leader_election.cc:343] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.885780 21855 leader_election.cc:290] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.902135 21811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.887467 21853 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.903808 21814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.904731 21853 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.889726 21704 consensus_queue.cc:237] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.847443 21879 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.896651 21774 raft_consensus.cc:2804] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.899694 21819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.900990 21818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.904666 21814 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:23.871404 21877 raft_consensus.cc:2804] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.892395 20989 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:23.892458 21850 raft_consensus.cc:2804] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:23.930012 20989 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.880162 21870 raft_consensus.cc:493] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.924960 21063 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:23.917651 21855 raft_consensus.cc:493] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.856782 21864 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.921288 21774 raft_consensus.cc:493] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
W20260709 12:17:23.950073 21140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 156334us
I20260709 12:17:23.872205 21867 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:23.921885 21762 raft_consensus.cc:493] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.935354 21871 raft_consensus.cc:2804] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.951836 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.934338 21877 raft_consensus.cc:493] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.956326 21026 raft_consensus.cc:2468] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:23.946579 21063 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:23.934031 21850 raft_consensus.cc:493] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.955190 21770 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:17:23.955168 21137 leader_election.cc:343] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:23.957702 21137 leader_election.cc:343] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.944242 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
W20260709 12:17:23.958809 21137 leader_election.cc:343] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.949570 21856 raft_consensus.cc:2804] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.961359 21063 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:23.962810 21017 raft_consensus.cc:2393] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:23.963200 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:23.964238 21024 raft_consensus.cc:2393] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:23.951332 21855 raft_consensus.cc:515] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:23.952041 21774 raft_consensus.cc:3060] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.943486 21870 raft_consensus.cc:515] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:23.969667 21770 tablet_bootstrap.cc:492] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:23.970564 21770 ts_tablet_manager.cc:1403] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.332s	user 0.030s	sys 0.002s
I20260709 12:17:23.954496 21692 raft_consensus.cc:493] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.956418 21772 raft_consensus.cc:493] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.956586 21871 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.958684 21850 raft_consensus.cc:3060] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.961280 21872 raft_consensus.cc:2804] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.958051 21877 raft_consensus.cc:3060] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.961263 21856 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.970983 21704 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.957341 21762 raft_consensus.cc:515] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
W20260709 12:17:23.988008 21137 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:23.992884 21137 leader_election.cc:343] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.984110 21871 raft_consensus.cc:3060] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.976976 21853 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.996085 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:23.998544 21174 raft_consensus.cc:2468] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:24.000445 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:24.001166 21095 raft_consensus.cc:2468] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:24.001673 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:24.002393 21864 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.003773 21870 leader_election.cc:290] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:23.997036 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:24.007164 21093 raft_consensus.cc:2393] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:23.984344 21872 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:23.980844 21692 raft_consensus.cc:515] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.009939 21872 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.978832 21880 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.011776 21692 leader_election.cc:290] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.012804 21818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:24.003697 21819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:23.988211 21856 raft_consensus.cc:3060] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.017683 21819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f01e89f8d2d941d494585d3147a3dd2f" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:23.983878 21772 raft_consensus.cc:515] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.024319 21814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.025080 21814 raft_consensus.cc:3060] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.012742 21853 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.025985 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:24.026572 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:24.027956 21853 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:24.028188 21169 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:24.029541 21859 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:23.997915 21817 raft_consensus.cc:2468] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:24.031035 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.031697 21171 raft_consensus.cc:3060] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.031983 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:24.032434 20989 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:24.032665 21097 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:24.033663 21861 raft_consensus.cc:2804] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.034235 21861 raft_consensus.cc:697] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:24.035145 21861 consensus_queue.cc:237] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.035480 21095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:24.025266 21772 leader_election.cc:290] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.012997 21816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:24.038857 21816 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:24.044849 21850 raft_consensus.cc:515] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.045953 21816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.046419 21871 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.047454 21139 leader_election.cc:304] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.075627 21139 leader_election.cc:304] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:24.048688 21774 raft_consensus.cc:515] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.076159 21772 raft_consensus.cc:2804] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.078455 21772 raft_consensus.cc:493] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.078986 21772 raft_consensus.cc:3060] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.079783 21774 leader_election.cc:290] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:24.050555 21818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.080583 21818 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.083709 21816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:23.991436 21704 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.086511 21704 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.051527 21879 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.087242 21871 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.053355 21880 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:24.053746 21819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:24.088541 21819 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:24.055542 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:24.053037 21867 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.089771 21810 raft_consensus.cc:2468] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:24.059590 21872 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.056813 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.090217 21853 raft_consensus.cc:2804] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.057605 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.049677 21770 raft_consensus.cc:359] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.058619 21140 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
W20260709 12:17:24.059581 21137 consensus_peers.cc:597] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:24.093194 21872 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:24.093510 21140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 104860us
W20260709 12:17:24.093693 21137 leader_election.cc:343] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.094722 21140 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
W20260709 12:17:24.095026 21137 leader_election.cc:343] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.063100 21762 leader_election.cc:290] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:24.063120 21814 raft_consensus.cc:2468] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:24.064477 21855 leader_election.cc:290] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:24.072865 21171 raft_consensus.cc:2468] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:24.083941 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:24.090924 21853 raft_consensus.cc:493] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.091022 21100 raft_consensus.cc:2393] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:24.096338 21098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b6872f6d7f14caea68ac416bcb17b27" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:24.056665 21877 raft_consensus.cc:515] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:24.096671 21066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 114094us
I20260709 12:17:24.075910 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
W20260709 12:17:24.098305 20989 leader_election.cc:343] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.105088 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.096783 21140 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.101766 21773 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.116053 21137 leader_election.cc:304] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:24.104568 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.117560 21818 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:24.140084 21817 raft_consensus.cc:2468] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:24.151248 21137 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:24.141474 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:24.152558 21020 raft_consensus.cc:2393] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:24.152644 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:24.153224 21095 raft_consensus.cc:2393] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:24.142593 21097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:24.143610 21889 raft_consensus.cc:3060] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.154106 21097 raft_consensus.cc:3060] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:24.143729 21140 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 47054.8us
I20260709 12:17:24.156423 21877 leader_election.cc:290] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.158030 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:24.145509 21881 raft_consensus.cc:2804] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.160504 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989" is_pre_election: true
I20260709 12:17:24.162562 21881 raft_consensus.cc:493] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.162786 21171 raft_consensus.cc:2393] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 2: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:24.163086 21881 raft_consensus.cc:3060] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.163480 21024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:24.164290 21024 raft_consensus.cc:2393] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:24.165197 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a416937e48a4949a7b5fba8c466eb41" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.165961 21889 raft_consensus.cc:2749] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.166541 21017 raft_consensus.cc:1275] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.169579 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.171221 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:24.168857 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.171875 21023 raft_consensus.cc:3060] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.175172 21174 raft_consensus.cc:2468] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:24.176929 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.178185 21817 raft_consensus.cc:2393] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:24.101969 21692 raft_consensus.cc:2804] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.178948 21854 consensus_queue.cc:1048] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.140786 21770 raft_consensus.cc:740] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:24.179777 21173 raft_consensus.cc:2468] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:24.180497 21770 consensus_queue.cc:260] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.183018 21770 ts_tablet_manager.cc:1434] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.212s	user 0.010s	sys 0.000s
I20260709 12:17:24.183276 21692 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.183866 21692 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.185030 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:24.147214 21139 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.148046 21704 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.191820 21091 raft_consensus.cc:2393] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:24.192270 21881 raft_consensus.cc:515] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.192829 21704 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.148075 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:24.194285 21026 raft_consensus.cc:2393] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
W20260709 12:17:24.149127 20989 leader_election.cc:343] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:24.195268 21139 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:42875, user_credentials={real_user=slave}} blocked reactor thread for 48146.5us
W20260709 12:17:24.195400 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 54341.2us
I20260709 12:17:24.119099 21850 leader_election.cc:290] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:24.197364 21878 raft_consensus.cc:2749] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.143863 21853 raft_consensus.cc:3060] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.132694 21856 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.202152 20989 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.202658 21856 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.203008 20849 ts_manager.cc:194] Re-registered known tserver with Master: c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.203694 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:24.204571 21100 raft_consensus.cc:2393] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:24.205060 21139 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.205485 21880 raft_consensus.cc:2804] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.205880 21854 raft_consensus.cc:2749] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.153380 21169 raft_consensus.cc:2468] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 1.
I20260709 12:17:24.145787 21812 raft_consensus.cc:3060] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.140329 21094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:24.206004 21880 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.148959 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:24.207010 21887 raft_consensus.cc:2804] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.208002 21880 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.208236 21022 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 1.
I20260709 12:17:24.142949 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:24.140700 21093 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.209774 21887 raft_consensus.cc:493] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.209919 21092 raft_consensus.cc:2393] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:24.210321 21887 raft_consensus.cc:3060] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.191411 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033d781150df48e1ba4cefa59d033f42" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:24.151276 21772 raft_consensus.cc:515] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.147012 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fc3437dadf4368a453b79bb7a38482" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.216498 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.216816 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.217686 21169 raft_consensus.cc:2393] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 2: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:24.219249 21772 leader_election.cc:290] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.219996 20989 leader_election.cc:304] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
W20260709 12:17:24.238883 21137 leader_election.cc:343] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.220613 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:24.240207 21092 raft_consensus.cc:3060] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.240588 21887 raft_consensus.cc:515] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.242923 21887 leader_election.cc:290] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:24.243460 21773 raft_consensus.cc:2749] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.225734 21814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c82b37b2b14fcfb1d8c88765a4ba24" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.247690 20989 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.248965 20989 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.250029 21863 raft_consensus.cc:2804] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.250383 21023 raft_consensus.cc:2468] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:24.250699 21867 raft_consensus.cc:2804] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.251094 21774 raft_consensus.cc:2749] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.252732 20989 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:24.251199 21867 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.254319 21867 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.255625 21171 raft_consensus.cc:2468] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:24.228011 21880 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.140245 21098 raft_consensus.cc:2393] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:24.226408 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.258383 21817 raft_consensus.cc:3060] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.259706 21880 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.260229 21092 raft_consensus.cc:2468] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:24.238801 21770 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:24.253553 21774 raft_consensus.cc:2749] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.261427 21812 raft_consensus.cc:2468] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:24.266685 21692 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:24.266986 21853 raft_consensus.cc:515] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.268064 21817 raft_consensus.cc:2468] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:24.268990 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:24.269234 21692 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:24.261691 21139 leader_election.cc:304] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:24.270179 21023 raft_consensus.cc:2393] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
W20260709 12:17:24.270213 21137 leader_election.cc:343] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.270684 21139 leader_election.cc:304] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.250751 21863 raft_consensus.cc:697] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:24.271576 21772 raft_consensus.cc:2804] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.272179 21772 raft_consensus.cc:697] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:24.272760 21854 raft_consensus.cc:2749] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.231788 21097 raft_consensus.cc:2468] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:24.263636 21140 leader_election.cc:304] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.272444 21863 consensus_queue.cc:237] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:24.273803 21063 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.9.198.254:35489, user_credentials={real_user=slave}} blocked reactor thread for 43659.2us
W20260709 12:17:24.275197 21063 leader_election.cc:343] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.194602 21881 leader_election.cc:290] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.275710 21093 raft_consensus.cc:2468] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
W20260709 12:17:24.276292 21063 leader_election.cc:343] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.277675 21814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f00f76eb99ab413589cdbe3aafd60f9e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.278673 21814 raft_consensus.cc:3060] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.279870 21066 leader_election.cc:304] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:24.282768 21856 raft_consensus.cc:2804] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.283593 21856 raft_consensus.cc:493] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.283703 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.284255 21856 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.287297 21773 raft_consensus.cc:2804] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.289204 21773 raft_consensus.cc:697] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:24.290202 21773 consensus_queue.cc:237] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.291049 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:24.273178 21772 consensus_queue.cc:237] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.292951 21137 leader_election.cc:304] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:24.297219 21887 raft_consensus.cc:2804] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.298056 21887 raft_consensus.cc:697] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:24.299018 21887 consensus_queue.cc:237] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:24.304153 21137 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.277318 21063 leader_election.cc:304] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:24.279017 20989 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:24.315704 21704 raft_consensus.cc:2804] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:24.316550 21704 raft_consensus.cc:493] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:24.316782 21774 raft_consensus.cc:2804] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.317458 21774 raft_consensus.cc:697] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:24.318902 21093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:24.317116 21704 raft_consensus.cc:3060] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:24.315788 21063 leader_election.cc:343] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.319936 21774 consensus_queue.cc:237] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
W20260709 12:17:24.321594 21063 leader_election.cc:343] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.308503 21867 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:24.324561 21022 raft_consensus.cc:2393] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 7f62eca1a43e41f2b41f0d621002ac36 in this term.
I20260709 12:17:24.317458 21853 leader_election.cc:290] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.325834 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.326632 21812 raft_consensus.cc:3060] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.326874 21140 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.327785 21856 raft_consensus.cc:515] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.328375 21853 raft_consensus.cc:2749] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.316473 20989 leader_election.cc:304] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:24.329262 21704 raft_consensus.cc:515] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.331295 21093 raft_consensus.cc:2393] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb45c1a4fe5341368530840de4608989 in current term 2: Already voted for candidate 4262e2b320ed4812be271437718fd86e in this term.
I20260709 12:17:24.334362 21856 leader_election.cc:290] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:24.337991 21026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:24.338665 21026 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.345184 21814 raft_consensus.cc:2468] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:24.346767 21137 leader_election.cc:304] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:24.347630 21853 raft_consensus.cc:2804] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.348212 21853 raft_consensus.cc:697] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:24.348248 21861 raft_consensus.cc:2804] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.348999 21861 raft_consensus.cc:697] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:24.349329 21853 consensus_queue.cc:237] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.349970 21861 consensus_queue.cc:237] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.354362 21063 leader_election.cc:304] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:24.355428 21063 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.349588 21867 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:24.348279 21812 raft_consensus.cc:2468] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:24.358697 21877 raft_consensus.cc:2804] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:24.359076 21063 leader_election.cc:343] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.359287 21877 raft_consensus.cc:697] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:24.360162 21063 leader_election.cc:304] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.360467 21877 consensus_queue.cc:237] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.360926 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.361809 21762 raft_consensus.cc:2749] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:24.361387 21063 leader_election.cc:343] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.366052 21137 leader_election.cc:304] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e
I20260709 12:17:24.362016 21169 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.369863 21872 raft_consensus.cc:2749] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.372700 21772 raft_consensus.cc:2804] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.373337 21772 raft_consensus.cc:697] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:24.374331 21772 consensus_queue.cc:237] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.382002 21704 leader_election.cc:290] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:24.385046 21026 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:24.385356 21063 leader_election.cc:304] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.387701 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.390363 21171 raft_consensus.cc:2393] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f62eca1a43e41f2b41f0d621002ac36 in current term 2: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:24.390946 21066 leader_election.cc:304] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:24.393517 21814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.395661 21704 raft_consensus.cc:2804] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.396629 21704 raft_consensus.cc:697] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:24.397866 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.388453 20989 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.406549 21867 raft_consensus.cc:2804] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.407130 21704 consensus_queue.cc:237] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.411828 21867 raft_consensus.cc:697] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:24.412904 21867 consensus_queue.cc:237] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.419179 21856 raft_consensus.cc:2764] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:24.423988 21026 raft_consensus.cc:1275] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.427268 21169 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:24.427453 21063 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:24.429819 21872 consensus_queue.cc:1048] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:24.430423 21856 raft_consensus.cc:2749] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.433493 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.435441 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d53959ed0724fc1905beec27f8a0917" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.435855 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.436440 21812 raft_consensus.cc:2393] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 2: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:24.435524 21770 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:24.440577 21818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.442858 21171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.452275 21169 raft_consensus.cc:2393] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 2: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
I20260709 12:17:24.452757 21171 raft_consensus.cc:3060] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.444983 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7a254b1018240fd8b49a2776fb01a2b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.450470 20991 leader_election.cc:304] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.454466 21861 raft_consensus.cc:2804] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.455238 21861 raft_consensus.cc:697] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:24.456549 21861 consensus_queue.cc:237] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.455469 21770 tablet_bootstrap.cc:492] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:24.463016 21770 ts_tablet_manager.cc:1403] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.224s	user 0.018s	sys 0.011s
W20260709 12:17:24.468077 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 49135.1us
W20260709 12:17:24.469072 20989 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.473460 21171 raft_consensus.cc:2468] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:24.472780 20845 catalog_manager.cc:5697] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:24.479790 20846 catalog_manager.cc:5697] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:24.481163 20845 catalog_manager.cc:5697] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:24.473948 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:24.467185 21770 raft_consensus.cc:359] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.485785 21770 raft_consensus.cc:740] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:24.471596 20989 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:24.476310 21818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.489465 21861 raft_consensus.cc:2749] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:24.493433 20989 leader_election.cc:343] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.478868 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca20df817314411a9f94ee42a3b5220a" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:24.486858 21770 consensus_queue.cc:260] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.499236 21770 ts_tablet_manager.cc:1434] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:17:24.501272 21770 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:24.514163 21812 raft_consensus.cc:2393] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4262e2b320ed4812be271437718fd86e in current term 2: Already voted for candidate eb45c1a4fe5341368530840de4608989 in this term.
W20260709 12:17:24.537418 21063 leader_election.cc:343] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:24.538497 21063 leader_election.cc:343] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:24.539443 21063 consensus_peers.cc:597] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:24.540578 21063 leader_election.cc:343] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:24.546149 20989 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.548007 21876 raft_consensus.cc:2804] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.549145 21876 raft_consensus.cc:697] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:24.553615 21876 consensus_queue.cc:237] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:24.564714 21770 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:24.586388 21171 raft_consensus.cc:1275] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.594605 21871 consensus_queue.cc:1048] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.597724 21063 leader_election.cc:304] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:24.599143 21872 raft_consensus.cc:2749] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.599114 21063 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:24.601589 21704 raft_consensus.cc:2749] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:24.601846 21137 consensus_peers.cc:597] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:24.603032 21063 leader_election.cc:304] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4262e2b320ed4812be271437718fd86e; no voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989
I20260709 12:17:24.604192 21704 raft_consensus.cc:2749] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.605494 21063 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: c8997852a67a4e678dfad30c76a76964
I20260709 12:17:24.606370 21704 raft_consensus.cc:2804] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.607144 21704 raft_consensus.cc:697] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:24.608105 21704 consensus_queue.cc:237] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.609875 21770 tablet_bootstrap.cc:492] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:24.610941 21770 ts_tablet_manager.cc:1403] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.003s
I20260709 12:17:24.614889 20848 catalog_manager.cc:5697] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 2 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.617921 21812 raft_consensus.cc:1275] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.616904 20848 catalog_manager.cc:5697] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 2 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.622185 20848 catalog_manager.cc:5697] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 2 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.624182 20848 catalog_manager.cc:5697] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 2 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.626132 20848 catalog_manager.cc:5697] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 2 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.640115 21876 consensus_queue.cc:1048] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.641136 21770 raft_consensus.cc:359] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.642405 21770 raft_consensus.cc:740] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:24.643234 21770 consensus_queue.cc:260] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.653591 20845 catalog_manager.cc:5697] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.656100 20845 catalog_manager.cc:5697] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:24.678957 20845 catalog_manager.cc:5697] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:24.703548 20989 consensus_peers.cc:597] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:24.704562 20989 consensus_peers.cc:597] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:24.711083 21770 ts_tablet_manager.cc:1434] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.000s
I20260709 12:17:24.712172 21770 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:24.736218 20989 consensus_peers.cc:597] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:24.739085 21022 raft_consensus.cc:1275] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.741165 21887 consensus_queue.cc:1048] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.747964 21817 raft_consensus.cc:1275] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.750681 21863 consensus_queue.cc:1048] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:24.750922 20847 catalog_manager.cc:5697] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.754678 20847 catalog_manager.cc:5697] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:24.760179 20847 catalog_manager.cc:5697] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.762876 21097 raft_consensus.cc:1275] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.764894 21887 consensus_queue.cc:1048] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.763130 20847 catalog_manager.cc:5697] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:24.770220 20847 catalog_manager.cc:5697] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:24.763619 20849 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55172
I20260709 12:17:24.785503 21812 raft_consensus.cc:1275] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.787766 21870 consensus_queue.cc:1048] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:24.788960 21846 heartbeater.cc:499] Master 127.9.198.254:35489 was elected leader, sending a full tablet report...
I20260709 12:17:24.811070 21812 raft_consensus.cc:1275] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.812014 21770 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:24.813265 21770 tablet_bootstrap.cc:492] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:24.813062 21887 consensus_queue.cc:1048] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:24.814954 21770 ts_tablet_manager.cc:1403] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.018s
I20260709 12:17:24.817512 21171 raft_consensus.cc:1275] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:24.821257 20989 consensus_peers.cc:597] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:24.822003 21876 consensus_queue.cc:1048] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.823218 21770 raft_consensus.cc:359] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.824023 21770 raft_consensus.cc:740] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:24.825330 21770 consensus_queue.cc:260] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:24.826228 21093 raft_consensus.cc:1275] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.827852 21770 ts_tablet_manager.cc:1434] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:17:24.828792 21770 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:24.830749 21887 consensus_queue.cc:1048] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
W20260709 12:17:24.849385 21137 consensus_peers.cc:597] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:24.850003 20989 consensus_peers.cc:597] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:24.850921 21093 raft_consensus.cc:1275] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.853363 21871 consensus_queue.cc:1048] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.860461 21026 raft_consensus.cc:1275] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.862020 21887 consensus_queue.cc:1048] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:24.867828 21137 consensus_peers.cc:597] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:24.869433 21137 consensus_peers.cc:597] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:24.882149 21063 consensus_peers.cc:597] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:24.890635 21173 raft_consensus.cc:1275] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.896615 21871 consensus_queue.cc:1048] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:24.898705 21818 raft_consensus.cc:1275] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.912930 21818 raft_consensus.cc:1275] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.913153 21770 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:24.914614 21879 consensus_queue.cc:1048] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.916471 21770 tablet_bootstrap.cc:492] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:24.917490 21770 ts_tablet_manager.cc:1403] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.089s	user 0.011s	sys 0.007s
I20260709 12:17:24.918620 21872 consensus_queue.cc:1048] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:24.919721 21097 raft_consensus.cc:1275] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.923259 21879 consensus_queue.cc:1048] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:24.939720 21022 raft_consensus.cc:1275] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.941493 21872 consensus_queue.cc:1048] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:24.944842 21063 consensus_peers.cc:597] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:24.947643 21770 raft_consensus.cc:359] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.948554 21770 raft_consensus.cc:740] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:24.951016 21770 consensus_queue.cc:260] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:24.981336 21770 ts_tablet_manager.cc:1434] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.000s
I20260709 12:17:24.987543 21770 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:25.082365 21925 log.cc:927] Time spent T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:17:24.994263 21812 raft_consensus.cc:1275] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.165305 21092 raft_consensus.cc:1275] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.197822 21936 consensus_queue.cc:1048] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:25.205328 21137 consensus_peers.cc:597] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.221127 21173 raft_consensus.cc:1275] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.228848 21814 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:25.233987 21063 consensus_peers.cc:597] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:25.244302 21137 consensus_peers.cc:597] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.247875 21814 raft_consensus.cc:1275] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.249471 21704 consensus_queue.cc:1048] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:25.262432 21770 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.263602 21770 tablet_bootstrap.cc:492] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.264349 21770 ts_tablet_manager.cc:1403] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.277s	user 0.018s	sys 0.023s
I20260709 12:17:25.265830 21879 consensus_queue.cc:1048] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.267413 21770 raft_consensus.cc:359] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.268217 21770 raft_consensus.cc:740] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.269104 21770 consensus_queue.cc:260] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.277242 21098 raft_consensus.cc:1275] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.274794 21770 ts_tablet_manager.cc:1434] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:17:25.279136 21770 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:25.290582 20989 consensus_peers.cc:597] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.293601 21942 consensus_queue.cc:1048] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:25.292393 21935 consensus_queue.cc:1048] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:17:25.317119 21817 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.339919 21817 raft_consensus.cc:1275] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:25.354348 21137 consensus_peers.cc:597] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.356339 21174 raft_consensus.cc:1275] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.362123 21770 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.363152 21770 tablet_bootstrap.cc:492] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.363829 21770 ts_tablet_manager.cc:1403] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.085s	user 0.011s	sys 0.009s
I20260709 12:17:25.367690 21170 raft_consensus.cc:1275] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.372025 21770 raft_consensus.cc:359] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.372782 21770 raft_consensus.cc:740] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.373473 21770 consensus_queue.cc:260] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.377884 21939 consensus_queue.cc:1048] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.404763 21770 ts_tablet_manager.cc:1434] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
W20260709 12:17:25.408609 21063 consensus_peers.cc:597] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:25.409332 21063 consensus_peers.cc:597] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.416965 21770 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.419642 21861 consensus_queue.cc:1048] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:25.424872 20989 consensus_peers.cc:597] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:25.425565 20989 consensus_peers.cc:597] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.435470 21863 consensus_queue.cc:1048] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:25.489069 21770 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.492486 21770 tablet_bootstrap.cc:492] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.493212 21770 ts_tablet_manager.cc:1403] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.012s
I20260709 12:17:25.495776 21770 raft_consensus.cc:359] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.498037 21770 raft_consensus.cc:740] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.498908 21770 consensus_queue.cc:260] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.501509 21770 ts_tablet_manager.cc:1434] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:17:25.502329 21770 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.524634 21770 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.525588 21770 tablet_bootstrap.cc:492] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.526230 21770 ts_tablet_manager.cc:1403] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 12:17:25.527943 21770 raft_consensus.cc:359] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.528477 21770 raft_consensus.cc:740] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.529192 21770 consensus_queue.cc:260] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.531646 21770 ts_tablet_manager.cc:1434] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:25.535637 21770 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.555990 21770 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.558185 21770 tablet_bootstrap.cc:492] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.558895 21770 ts_tablet_manager.cc:1403] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 12:17:25.561316 21770 raft_consensus.cc:359] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.562107 21770 raft_consensus.cc:740] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.562783 21770 consensus_queue.cc:260] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.564934 21770 ts_tablet_manager.cc:1434] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
W20260709 12:17:25.565994 21137 consensus_peers.cc:597] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.566390 21770 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:25.575857 20989 consensus_peers.cc:597] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:25.589623 21770 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.590520 21770 tablet_bootstrap.cc:492] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.591187 21770 ts_tablet_manager.cc:1403] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 12:17:25.593485 21770 raft_consensus.cc:359] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.594126 21770 raft_consensus.cc:740] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.594766 21770 consensus_queue.cc:260] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.596819 21770 ts_tablet_manager.cc:1434] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:25.598109 21770 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.610954 21770 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.611955 21770 tablet_bootstrap.cc:492] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.612617 21770 ts_tablet_manager.cc:1403] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.004s
I20260709 12:17:25.614393 21770 raft_consensus.cc:359] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.614931 21770 raft_consensus.cc:740] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.615504 21770 consensus_queue.cc:260] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.617132 21770 ts_tablet_manager.cc:1434] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:17:25.618201 21770 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.637107 21770 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.638024 21770 tablet_bootstrap.cc:492] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.638736 21770 ts_tablet_manager.cc:1403] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 12:17:25.640689 21770 raft_consensus.cc:359] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.641213 21770 raft_consensus.cc:740] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.641825 21770 consensus_queue.cc:260] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.643543 21770 ts_tablet_manager.cc:1434] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:17:25.644586 21770 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.659996 21770 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.660993 21770 tablet_bootstrap.cc:492] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.661713 21770 ts_tablet_manager.cc:1403] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:17:25.665794 21770 raft_consensus.cc:359] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.666426 21770 raft_consensus.cc:740] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.667197 21770 consensus_queue.cc:260] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.669095 21770 ts_tablet_manager.cc:1434] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:17:25.670188 21770 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.682958 21770 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.683862 21770 tablet_bootstrap.cc:492] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.684486 21770 ts_tablet_manager.cc:1403] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:17:25.686378 21770 raft_consensus.cc:359] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.686895 21770 raft_consensus.cc:740] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.687439 21770 consensus_queue.cc:260] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.689227 21770 ts_tablet_manager.cc:1434] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:25.690426 21770 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.695406 21810 raft_consensus.cc:3060] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.718554 21770 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.719650 21770 tablet_bootstrap.cc:492] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.720350 21770 ts_tablet_manager.cc:1403] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.008s
I20260709 12:17:25.723937 21770 raft_consensus.cc:359] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.724910 21770 raft_consensus.cc:740] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.725883 21770 consensus_queue.cc:260] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.728231 21770 ts_tablet_manager.cc:1434] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:17:25.777194 21770 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.834072 21770 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.835115 21770 tablet_bootstrap.cc:492] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.836402 21770 ts_tablet_manager.cc:1403] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.059s	user 0.038s	sys 0.017s
I20260709 12:17:25.838892 21770 raft_consensus.cc:359] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.839622 21770 raft_consensus.cc:740] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.840493 21770 consensus_queue.cc:260] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.842831 21770 ts_tablet_manager.cc:1434] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:17:25.843693 21770 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.858312 21772 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.859009 21772 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.861277 21772 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:25.862964 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:25.863464 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:25.863933 21815 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:25.864241 21092 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:25.865831 21137 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:25.867882 21951 raft_consensus.cc:2804] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.868275 21951 raft_consensus.cc:493] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.868602 21951 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.875440 21770 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.876529 21770 tablet_bootstrap.cc:492] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.877216 21770 ts_tablet_manager.cc:1403] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.012s
I20260709 12:17:25.881567 21951 raft_consensus.cc:515] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.882304 21770 raft_consensus.cc:359] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.883054 21770 raft_consensus.cc:740] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.883769 21770 consensus_queue.cc:260] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.887233 21770 ts_tablet_manager.cc:1434] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:17:25.887316 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:25.888023 21100 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.888114 21770 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.900107 21100 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 3.
I20260709 12:17:25.902143 21951 leader_election.cc:290] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:25.903997 21770 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.905133 21770 tablet_bootstrap.cc:492] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.905557 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91f9400a74934eb4b3269b428057ccef" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:25.906010 21770 ts_tablet_manager.cc:1403] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.003s
I20260709 12:17:25.906456 21810 raft_consensus.cc:3060] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.903889 21139 leader_election.cc:304] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:25.909310 21770 raft_consensus.cc:359] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.910053 21772 raft_consensus.cc:2804] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.910441 21770 raft_consensus.cc:740] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.911371 21772 raft_consensus.cc:697] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:25.911553 21770 consensus_queue.cc:260] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.912564 21772 consensus_queue.cc:237] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:25.913998 21770 ts_tablet_manager.cc:1434] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:25.914884 21770 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.919471 21810 raft_consensus.cc:2468] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 3.
I20260709 12:17:25.932598 20848 catalog_manager.cc:5697] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 3, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 3 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:25.937718 21770 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.939285 21770 tablet_bootstrap.cc:492] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.940045 21770 ts_tablet_manager.cc:1403] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260709 12:17:25.942821 21770 raft_consensus.cc:359] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.943384 21770 raft_consensus.cc:740] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.944060 21770 consensus_queue.cc:260] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.946252 21770 ts_tablet_manager.cc:1434] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:17:25.950230 21770 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.967800 21770 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.969228 21770 tablet_bootstrap.cc:492] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.969957 21770 ts_tablet_manager.cc:1403] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:17:25.972124 21770 raft_consensus.cc:359] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.972615 21770 raft_consensus.cc:740] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.973140 21770 consensus_queue.cc:260] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:25.974682 21770 ts_tablet_manager.cc:1434] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:17:25.975571 21770 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:25.994168 21770 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:25.995198 21770 tablet_bootstrap.cc:492] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:25.995606 21810 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.995795 21770 ts_tablet_manager.cc:1403] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:17:25.998027 21770 raft_consensus.cc:359] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:25.998528 21770 raft_consensus.cc:740] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:25.999159 21770 consensus_queue.cc:260] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.001858 21770 ts_tablet_manager.cc:1434] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:17:26.003072 21770 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.062882 21951 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.064884 21951 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:26.119755 21951 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:26.126757 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36" is_pre_election: true
I20260709 12:17:26.128531 21023 raft_consensus.cc:2468] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:26.126704 21815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:26.134552 21140 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:26.136319 21772 raft_consensus.cc:2804] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.136960 21772 raft_consensus.cc:493] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.137511 21772 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:26.144156 21137 leader_election.cc:343] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:26.157362 21815 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.175283 21770 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.176618 21770 tablet_bootstrap.cc:492] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.177781 21770 ts_tablet_manager.cc:1403] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.175s	user 0.027s	sys 0.002s
I20260709 12:17:26.197896 21770 raft_consensus.cc:359] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:26.198774 21770 raft_consensus.cc:740] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
W20260709 12:17:26.192056 21137 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 57253.5us
I20260709 12:17:26.199504 21770 consensus_queue.cc:260] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:26.216051 21772 raft_consensus.cc:515] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:26.238236 21772 leader_election.cc:290] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 election: Requested vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:26.251737 21812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:26.252616 21812 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.279742 21770 ts_tablet_manager.cc:1434] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.009s
I20260709 12:17:26.282794 21817 raft_consensus.cc:3060] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.289650 21770 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.293537 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3778fc4a90d9419cbd028360957cbc4e" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:26.294358 21023 raft_consensus.cc:3060] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.310672 21092 raft_consensus.cc:1275] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.316725 21023 raft_consensus.cc:2468] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 3.
I20260709 12:17:26.351912 21935 consensus_queue.cc:1048] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:26.372639 21140 leader_election.cc:304] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:26.379294 21936 raft_consensus.cc:2804] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.380004 21936 raft_consensus.cc:697] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:26.376992 21812 raft_consensus.cc:2468] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 3.
I20260709 12:17:26.381248 21936 consensus_queue.cc:237] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:26.418533 21982 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.420379 21982 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:26.427134 21982 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:26.428313 21817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964" is_pre_election: true
I20260709 12:17:26.432463 21817 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 1.
I20260709 12:17:26.442783 21137 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:26.445787 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e" is_pre_election: true
I20260709 12:17:26.447009 21100 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 2.
I20260709 12:17:26.454655 21982 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.455983 21982 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.457999 21982 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.478592 20845 catalog_manager.cc:5697] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 3, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 3 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:26.503434 21770 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.505013 21770 tablet_bootstrap.cc:492] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.505861 21770 ts_tablet_manager.cc:1403] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.217s	user 0.023s	sys 0.006s
I20260709 12:17:26.511562 21810 raft_consensus.cc:1275] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.516490 21982 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:26.518986 21982 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:26.520177 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:26.520898 21092 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.521898 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "eb45c1a4fe5341368530840de4608989" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8997852a67a4e678dfad30c76a76964"
I20260709 12:17:26.522779 21810 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.523445 21935 consensus_queue.cc:1048] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:26.533509 21092 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 3.
I20260709 12:17:26.535161 21139 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:26.536326 21935 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.536877 21935 raft_consensus.cc:697] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Becoming Leader. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:26.537905 21935 consensus_queue.cc:237] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:26.544193 21810 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb45c1a4fe5341368530840de4608989 in term 3.
I20260709 12:17:26.551519 21770 raft_consensus.cc:359] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.553809 21770 raft_consensus.cc:740] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.556401 21770 consensus_queue.cc:260] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.558701 21770 ts_tablet_manager.cc:1434] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.003s
I20260709 12:17:26.559546 21770 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.589944 20847 catalog_manager.cc:5697] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 reported cstate change: term changed from 1 to 3, leader changed from c8997852a67a4e678dfad30c76a76964 (127.9.198.193) to eb45c1a4fe5341368530840de4608989 (127.9.198.196). New cstate: current_term: 3 leader_uuid: "eb45c1a4fe5341368530840de4608989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:26.591670 21770 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.592823 21770 tablet_bootstrap.cc:492] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.593544 21770 ts_tablet_manager.cc:1403] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.034s	user 0.022s	sys 0.003s
I20260709 12:17:26.596468 21770 raft_consensus.cc:359] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.597169 21770 raft_consensus.cc:740] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.597935 21770 consensus_queue.cc:260] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.600211 21770 ts_tablet_manager.cc:1434] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:26.602272 21770 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
W20260709 12:17:26.655062 21137 consensus_peers.cc:597] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:26.655184 21770 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.656530 21770 tablet_bootstrap.cc:492] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.659808 21817 raft_consensus.cc:3060] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.663384 21770 ts_tablet_manager.cc:1403] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.061s	user 0.031s	sys 0.008s
I20260709 12:17:26.666536 21770 raft_consensus.cc:359] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:26.667493 21770 raft_consensus.cc:740] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.668284 21770 consensus_queue.cc:260] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:26.670048 21770 ts_tablet_manager.cc:1434] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:26.674578 21770 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.695449 21770 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.696445 21770 tablet_bootstrap.cc:492] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.697160 21770 ts_tablet_manager.cc:1403] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.001s
I20260709 12:17:26.699719 21770 raft_consensus.cc:359] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.700271 21770 raft_consensus.cc:740] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.700907 21770 consensus_queue.cc:260] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.702898 21770 ts_tablet_manager.cc:1434] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.004s
I20260709 12:17:26.703740 21770 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.717474 21770 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.718344 21770 tablet_bootstrap.cc:492] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.718986 21770 ts_tablet_manager.cc:1403] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 12:17:26.721271 21770 raft_consensus.cc:359] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.721832 21770 raft_consensus.cc:740] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.722338 21770 consensus_queue.cc:260] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.723948 21770 ts_tablet_manager.cc:1434] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:26.724830 21770 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.746380 21770 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.747454 21770 tablet_bootstrap.cc:492] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.748090 21770 ts_tablet_manager.cc:1403] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 12:17:26.750447 21770 raft_consensus.cc:359] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:26.751084 21770 raft_consensus.cc:740] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.751715 21770 consensus_queue.cc:260] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:26.753818 21770 ts_tablet_manager.cc:1434] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:17:26.754678 21770 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.771250 21770 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.772261 21770 tablet_bootstrap.cc:492] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.772977 21770 ts_tablet_manager.cc:1403] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.001s
I20260709 12:17:26.775020 21770 raft_consensus.cc:359] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.775651 21770 raft_consensus.cc:740] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.776206 21770 consensus_queue.cc:260] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:26.778185 21770 ts_tablet_manager.cc:1434] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:17:26.779450 21770 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.780269 21817 raft_consensus.cc:1275] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.783994 21936 consensus_queue.cc:1048] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:26.797734 21770 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.799022 21770 tablet_bootstrap.cc:492] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.799837 21770 ts_tablet_manager.cc:1403] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.001s
I20260709 12:17:26.803406 21770 raft_consensus.cc:359] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:26.804318 21770 raft_consensus.cc:740] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.805037 21770 consensus_queue.cc:260] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:26.807322 21770 ts_tablet_manager.cc:1434] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:26.808341 21770 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Bootstrap starting.
I20260709 12:17:26.825922 21023 raft_consensus.cc:1275] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.827600 21982 consensus_queue.cc:1048] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:26.837307 21770 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:26.838404 21770 tablet_bootstrap.cc:492] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Bootstrap complete.
I20260709 12:17:26.839108 21770 ts_tablet_manager.cc:1403] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.003s
I20260709 12:17:26.842062 21770 raft_consensus.cc:359] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:26.842965 21770 raft_consensus.cc:740] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Initialized, Role: FOLLOWER
I20260709 12:17:26.843917 21770 consensus_queue.cc:260] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:26.846374 21770 ts_tablet_manager.cc:1434] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:17:26.884353 21817 raft_consensus.cc:3060] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.042850 21100 raft_consensus.cc:1275] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.044186 21935 consensus_queue.cc:1048] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.055203 21817 raft_consensus.cc:1275] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Refusing update from remote peer eb45c1a4fe5341368530840de4608989: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.057049 21982 consensus_queue.cc:1048] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.083202 21817 raft_consensus.cc:3060] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.240064 21817 raft_consensus.cc:3060] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.479374 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:33.541577 21174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1"
dest_uuid: "eb45c1a4fe5341368530840de4608989"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:17:33.542412 21174 raft_consensus.cc:606] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:33.555692 21020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a47e447bb0004896847c1e74485df964"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:33.556306 21020 raft_consensus.cc:606] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:33.558876 21020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:33.559289 21020 raft_consensus.cc:606] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Received request to transfer leadership to TS 4262e2b320ed4812be271437718fd86e
I20260709 12:17:33.561403 21020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:33.561880 21020 raft_consensus.cc:606] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:33.564056 21020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:33.564498 21020 raft_consensus.cc:606] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:33.566785 21020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:33.567305 21020 raft_consensus.cc:606] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Received request to transfer leadership to TS 4262e2b320ed4812be271437718fd86e
I20260709 12:17:33.568379 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:33.570221 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:33.571084 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:33.613617 21945 raft_consensus.cc:993] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:33.614279 22024 raft_consensus.cc:1081] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:33.616804 21817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a47e447bb0004896847c1e74485df964"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:33.617379 21817 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:33.617803 21817 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.625478 21817 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:33.627856 21817 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:33.650990 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:33.651649 21174 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.654872 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a47e447bb0004896847c1e74485df964" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:33.655504 21020 raft_consensus.cc:3055] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:33.655812 21020 raft_consensus.cc:740] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:33.656553 21020 consensus_queue.cc:260] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:33.657852 21020 raft_consensus.cc:3060] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.660352 21174 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:33.661449 21764 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:33.663370 22033 raft_consensus.cc:2804] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:33.664518 21020 raft_consensus.cc:2468] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:33.664664 22033 raft_consensus.cc:697] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:33.665913 22033 consensus_queue.cc:237] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:33.675830 20845 catalog_manager.cc:5697] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 2 to 3, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:33.690955 22025 raft_consensus.cc:993] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:33.691636 22020 raft_consensus.cc:1081] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:33.693061 21817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37558
I20260709 12:17:33.693522 21817 raft_consensus.cc:493] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:33.693859 21817 raft_consensus.cc:3060] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.699334 21817 raft_consensus.cc:515] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:33.702222 21817 leader_election.cc:290] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:33.704656 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:33.705428 21174 raft_consensus.cc:3055] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:33.705793 21174 raft_consensus.cc:740] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:33.706566 21174 consensus_queue.cc:260] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:33.707747 21174 raft_consensus.cc:3060] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.715656 21174 raft_consensus.cc:2468] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:33.716768 21764 leader_election.cc:304] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:33.717262 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21144a10cc4945629d70fdd253d940d1" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:33.717779 22033 raft_consensus.cc:2804] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:33.717988 21091 raft_consensus.cc:3060] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.718358 22033 raft_consensus.cc:697] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:33.719280 22033 consensus_queue.cc:237] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:33.722215 22024 raft_consensus.cc:993] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:33.722779 22024 raft_consensus.cc:1081] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Signalling peer 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:33.724789 21100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:43338
I20260709 12:17:33.725310 21100 raft_consensus.cc:493] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:33.725647 21100 raft_consensus.cc:3060] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.726048 21091 raft_consensus.cc:2468] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:33.730496 20845 catalog_manager.cc:5697] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from eb45c1a4fe5341368530840de4608989 (127.9.198.196) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:33.732345 21100 raft_consensus.cc:515] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:33.735538 21945 raft_consensus.cc:993] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:33.736263 22024 raft_consensus.cc:1081] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Signalling peer 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:33.738401 21092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:43338
I20260709 12:17:33.738507 21020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:33.739009 21092 raft_consensus.cc:493] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:33.739181 21020 raft_consensus.cc:3055] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:33.742298 21092 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.738536 21100 leader_election.cc:290] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:33.742550 21020 raft_consensus.cc:740] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:33.739702 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b79db6c634bd4743b1cb15b5310dd4f8" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:33.745482 21174 raft_consensus.cc:3060] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.745862 21020 consensus_queue.cc:260] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:33.747641 21020 raft_consensus.cc:3060] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.758261 21174 raft_consensus.cc:2468] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:33.761332 21092 raft_consensus.cc:515] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:33.761998 21063 leader_election.cc:304] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:33.763723 21092 leader_election.cc:290] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:33.756043 21020 raft_consensus.cc:2468] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:33.766456 21939 raft_consensus.cc:2804] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:33.766641 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:33.767001 21939 raft_consensus.cc:697] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:33.767336 21023 raft_consensus.cc:3055] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:33.767674 21023 raft_consensus.cc:740] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:33.768342 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:33.768445 21023 consensus_queue.cc:260] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:33.769014 21174 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.769858 21023 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.767813 21939 consensus_queue.cc:237] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:33.776443 21174 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:33.778475 21023 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:33.779472 21063 leader_election.cc:304] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:33.780954 21704 raft_consensus.cc:2804] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:33.781586 21704 raft_consensus.cc:697] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:33.782729 21704 consensus_queue.cc:237] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:33.797483 20845 catalog_manager.cc:5697] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:33.813805 20845 catalog_manager.cc:5697] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.011498 22024 raft_consensus.cc:993] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:34.012044 21945 raft_consensus.cc:1081] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:34.013715 21810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:34.014344 21810 raft_consensus.cc:493] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:34.014748 21810 raft_consensus.cc:3060] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.022384 21810 raft_consensus.cc:515] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.026376 21810 leader_election.cc:290] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:34.026600 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:34.027253 21100 raft_consensus.cc:3060] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.027509 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d701ef5192bf4f11a84c7b7fa0660c67" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:34.028139 21022 raft_consensus.cc:3055] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:34.028460 21022 raft_consensus.cc:740] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:34.029191 21022 consensus_queue.cc:260] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.030447 21022 raft_consensus.cc:3060] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.033800 21100 raft_consensus.cc:2468] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:34.034997 21766 leader_election.cc:304] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:34.035948 22033 raft_consensus.cc:2804] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:34.036454 22033 raft_consensus.cc:697] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:34.037303 21022 raft_consensus.cc:2468] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:34.037389 22033 consensus_queue.cc:237] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.049530 20845 catalog_manager.cc:5697] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.051105 22026 raft_consensus.cc:993] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:34.053884 21945 raft_consensus.cc:1081] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:34.058905 21810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:34.059634 21810 raft_consensus.cc:493] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:34.060077 21810 raft_consensus.cc:3060] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.063246 21022 raft_consensus.cc:1275] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:34.069545 21810 raft_consensus.cc:515] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.072067 21810 leader_election.cc:290] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:34.073659 22033 consensus_queue.cc:1048] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.076273 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:34.077018 21100 raft_consensus.cc:3060] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.081743 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5353702fdb44369a77c62ee46fa7fcd" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:34.082428 21022 raft_consensus.cc:3055] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:34.082754 21022 raft_consensus.cc:740] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:34.083508 21022 consensus_queue.cc:260] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.084879 21022 raft_consensus.cc:3060] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.086122 21100 raft_consensus.cc:2468] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:34.087718 21766 leader_election.cc:304] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:34.089852 22033 raft_consensus.cc:2804] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:34.090405 22033 raft_consensus.cc:697] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:34.091332 22033 consensus_queue.cc:237] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.110065 21022 raft_consensus.cc:2468] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:34.126432 21170 raft_consensus.cc:1275] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:34.129665 22033 consensus_queue.cc:1048] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.129168 20845 catalog_manager.cc:5697] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.220536 21022 raft_consensus.cc:1275] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.221848 22043 consensus_queue.cc:1048] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.231223 21170 raft_consensus.cc:1275] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.232757 21704 consensus_queue.cc:1048] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.283774 21022 raft_consensus.cc:1275] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.285337 21939 consensus_queue.cc:1048] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.291600 21170 raft_consensus.cc:1275] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.296046 22033 consensus_queue.cc:1048] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.297366 21170 raft_consensus.cc:1275] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.298779 21704 consensus_queue.cc:1048] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:34.318902 21092 raft_consensus.cc:1275] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.320317 22051 consensus_queue.cc:1048] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:34.490514 21100 raft_consensus.cc:1275] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.491709 22051 consensus_queue.cc:1048] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.501659 21022 raft_consensus.cc:1275] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.503443 22051 consensus_queue.cc:1048] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.615511 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:34.644109 21022 raft_consensus.cc:1275] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.645387 22034 consensus_queue.cc:1048] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.657142 21023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:34.657750 21023 raft_consensus.cc:606] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Received request to transfer leadership to TS 4262e2b320ed4812be271437718fd86e
I20260709 12:17:34.660458 21174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435"
dest_uuid: "eb45c1a4fe5341368530840de4608989"
mode: GRACEFUL
new_leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:17:34.661005 21174 raft_consensus.cc:606] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Received request to transfer leadership to TS 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:34.663707 21022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:34.664266 21022 raft_consensus.cc:606] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:34.667349 21023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:34.667860 21023 raft_consensus.cc:606] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:34.670771 21100 raft_consensus.cc:1275] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:34.672906 22033 consensus_queue.cc:1048] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:34.673096 21022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5569aee047642449df010d452ed73fa"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:34.673727 21022 raft_consensus.cc:606] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:34.675393 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:34.677094 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:34.677917 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:34.735667 22046 raft_consensus.cc:993] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:34.736225 22046 raft_consensus.cc:1081] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:34.737597 21810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:34.738078 21810 raft_consensus.cc:493] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:34.738308 21810 raft_consensus.cc:3060] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.743307 21810 raft_consensus.cc:515] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.744998 21810 leader_election.cc:290] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:34.745883 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:34.746069 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c1d07d1b1344a5ad97b6f5a36d343a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:34.746619 21022 raft_consensus.cc:3055] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:34.746703 21100 raft_consensus.cc:3060] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.747007 21022 raft_consensus.cc:740] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:34.747830 21022 consensus_queue.cc:260] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.749131 21022 raft_consensus.cc:3060] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.754258 21100 raft_consensus.cc:2468] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:34.755594 21766 leader_election.cc:304] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:34.756457 22033 raft_consensus.cc:2804] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:34.756928 22033 raft_consensus.cc:697] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:34.758123 21022 raft_consensus.cc:2468] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:34.757779 22033 consensus_queue.cc:237] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:34.769091 20847 catalog_manager.cc:5697] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.853715 22048 raft_consensus.cc:993] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989: : Instructing follower 7f62eca1a43e41f2b41f0d621002ac36 to start an election
I20260709 12:17:34.854271 22048 raft_consensus.cc:1081] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Signalling peer 7f62eca1a43e41f2b41f0d621002ac36 to start an election
I20260709 12:17:34.856480 21022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
 from {username='slave'} at 127.0.0.1:34544
I20260709 12:17:34.857082 21022 raft_consensus.cc:493] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:34.857481 21022 raft_consensus.cc:3060] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.866334 21022 raft_consensus.cc:515] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:34.871305 21022 leader_election.cc:290] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:34.872018 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:34.872735 21173 raft_consensus.cc:3055] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:34.873098 21173 raft_consensus.cc:740] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:34.872817 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9137ae28b54fb58af02339dca65435" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:34.873616 21092 raft_consensus.cc:3060] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.873843 21173 consensus_queue.cc:260] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:34.874730 21173 raft_consensus.cc:3060] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.880163 21092 raft_consensus.cc:2468] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:34.882294 20991 leader_election.cc:304] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:34.883491 22046 raft_consensus.cc:2804] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:34.886314 21173 raft_consensus.cc:2468] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 2.
I20260709 12:17:34.887567 22046 raft_consensus.cc:697] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:34.888648 22046 consensus_queue.cc:237] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:34.899569 20847 catalog_manager.cc:5697] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 1 to 2, leader changed from eb45c1a4fe5341368530840de4608989 (127.9.198.196) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:35.086514 22072 raft_consensus.cc:993] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:35.087044 22024 raft_consensus.cc:1081] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:35.088632 21817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5569aee047642449df010d452ed73fa"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:35.089294 21817 raft_consensus.cc:493] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:35.089699 21817 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.092424 22072 raft_consensus.cc:993] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:35.092940 22024 raft_consensus.cc:1081] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Signalling peer 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:35.094455 21092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:43338
I20260709 12:17:35.095001 21092 raft_consensus.cc:493] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:35.095335 21092 raft_consensus.cc:3060] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:35.097744 21817 raft_consensus.cc:515] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:35.099831 21817 leader_election.cc:290] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:35.100409 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:35.100711 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5569aee047642449df010d452ed73fa" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:35.101083 21022 raft_consensus.cc:3055] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:35.101279 21100 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.101445 21022 raft_consensus.cc:740] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:35.102442 21022 consensus_queue.cc:260] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:35.103083 21092 raft_consensus.cc:515] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:35.103650 21022 raft_consensus.cc:3060] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.104926 21092 leader_election.cc:290] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:35.106349 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:35.106628 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0468c28c130f44858399b670b2b9cb8b" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:35.106971 21023 raft_consensus.cc:3055] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:35.107254 21174 raft_consensus.cc:3060] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:35.107358 21023 raft_consensus.cc:740] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:35.108076 21100 raft_consensus.cc:2468] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:35.108294 21023 consensus_queue.cc:260] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:35.109179 21766 leader_election.cc:304] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:35.109622 21023 raft_consensus.cc:3060] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:35.110167 22064 raft_consensus.cc:2804] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:35.110675 22064 raft_consensus.cc:697] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:35.111472 22064 consensus_queue.cc:237] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:35.114379 21174 raft_consensus.cc:2468] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:35.115407 21023 raft_consensus.cc:2468] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:35.115998 21063 leader_election.cc:304] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:35.116884 22043 raft_consensus.cc:2804] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:35.117451 22043 raft_consensus.cc:697] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:35.118366 22043 consensus_queue.cc:237] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:35.118903 21022 raft_consensus.cc:2468] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:35.125778 20847 catalog_manager.cc:5697] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 2 to 3, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:35.138979 20845 catalog_manager.cc:5697] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:35.160311 22072 raft_consensus.cc:993] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:35.160912 22024 raft_consensus.cc:1081] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:35.162578 21817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:35.162997 21817 raft_consensus.cc:493] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:35.163223 21817 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.170935 21817 raft_consensus.cc:515] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:35.172659 21817 leader_election.cc:290] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:35.173393 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:35.173678 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a29e7a2e40b345b09d50b1699ce33f47" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:35.174144 21022 raft_consensus.cc:3055] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:35.174192 21100 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.174558 21022 raft_consensus.cc:740] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:35.175361 21022 consensus_queue.cc:260] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:35.176577 21022 raft_consensus.cc:3060] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.181757 21100 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:35.182921 21766 leader_election.cc:304] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:35.183809 22064 raft_consensus.cc:2804] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:35.183773 21022 raft_consensus.cc:2468] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:35.184605 22064 raft_consensus.cc:697] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:35.185590 22064 consensus_queue.cc:237] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:35.206384 20845 catalog_manager.cc:5697] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 2 to 3, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:35.301717 21100 raft_consensus.cc:1275] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.303411 22064 consensus_queue.cc:1048] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:35.319044 21023 raft_consensus.cc:1275] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.320228 22064 consensus_queue.cc:1048] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:35.478097 21100 raft_consensus.cc:1275] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.479830 22024 consensus_queue.cc:1048] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:35.499619 21173 raft_consensus.cc:1275] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.502413 22046 consensus_queue.cc:1048] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:17:35.535163 21092 raft_consensus.cc:1275] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:35.537084 22064 consensus_queue.cc:1048] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:35.551981 21023 raft_consensus.cc:1275] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:35.553382 22033 consensus_queue.cc:1048] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:35.586668 21022 raft_consensus.cc:1275] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.587996 22043 consensus_queue.cc:1048] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:35.598116 21173 raft_consensus.cc:1275] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.599484 22043 consensus_queue.cc:1048] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:35.689070 21092 raft_consensus.cc:1275] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:35.690296 22033 consensus_queue.cc:1048] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:35.699765 21023 raft_consensus.cc:1275] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:35.705984 22033 consensus_queue.cc:1048] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:35.728655 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:35.774538 21173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536"
dest_uuid: "eb45c1a4fe5341368530840de4608989"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:17:35.775151 21173 raft_consensus.cc:606] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:35.791702 21092 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:42772
I20260709 12:17:35.792305 21092 raft_consensus.cc:606] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:35.795171 21173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b"
dest_uuid: "eb45c1a4fe5341368530840de4608989"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:17:35.795755 21173 raft_consensus.cc:606] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:35.798527 21174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca"
dest_uuid: "eb45c1a4fe5341368530840de4608989"
mode: GRACEFUL
new_leader_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:17:35.799149 21174 raft_consensus.cc:606] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Received request to transfer leadership to TS 4262e2b320ed4812be271437718fd86e
I20260709 12:17:35.801805 21023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:35.802362 21023 raft_consensus.cc:606] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:35.803640 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:35.805497 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:35.806264 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:36.050717 22043 raft_consensus.cc:993] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.051273 22043 raft_consensus.cc:1081] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.053159 21817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37542
I20260709 12:17:36.053736 21817 raft_consensus.cc:493] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:36.054181 21817 raft_consensus.cc:3060] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.061764 21817 raft_consensus.cc:515] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:36.064640 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:36.065255 21100 raft_consensus.cc:3055] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:36.065586 21100 raft_consensus.cc:740] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:36.066474 21100 consensus_queue.cc:260] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:36.067584 21100 raft_consensus.cc:3060] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.091758 21817 leader_election.cc:290] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:36.091898 21100 raft_consensus.cc:2468] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:36.093300 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698227e80f3c48a28b91fa4df79955cf" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:36.093677 22048 raft_consensus.cc:993] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989: : Instructing follower 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:36.094040 21174 raft_consensus.cc:3060] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.094225 22048 raft_consensus.cc:1081] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Signalling peer 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:36.100663 21766 leader_election.cc:304] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:36.100847 21174 raft_consensus.cc:2468] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:36.101536 22051 raft_consensus.cc:2804] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:36.103422 22051 raft_consensus.cc:697] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:36.104516 22051 consensus_queue.cc:237] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:36.107887 22025 raft_consensus.cc:993] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.108811 21100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:43346
I20260709 12:17:36.109989 21100 raft_consensus.cc:493] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:36.110693 21100 raft_consensus.cc:3060] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.110147 22097 raft_consensus.cc:1081] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.116734 21817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37558
I20260709 12:17:36.117233 21817 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:36.117527 21817 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.119506 21100 raft_consensus.cc:515] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:36.121994 21100 leader_election.cc:290] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:36.123574 21817 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:36.125856 21817 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 4 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:36.153553 21092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:36.155838 21092 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.156782 20847 catalog_manager.cc:5697] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from 4262e2b320ed4812be271437718fd86e (127.9.198.195) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:36.159879 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:36.161024 21022 raft_consensus.cc:3060] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:36.276268 21137 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 44230.3us
I20260709 12:17:36.288611 22097 raft_consensus.cc:993] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.289179 21022 raft_consensus.cc:2468] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:36.289518 22097 raft_consensus.cc:1081] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.305145 21174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1474ed8c5944a89ade3d329705003ca" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:36.306147 21174 raft_consensus.cc:3055] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:36.306542 21174 raft_consensus.cc:740] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:36.306804 22095 raft_consensus.cc:993] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.307544 22095 raft_consensus.cc:1081] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:36.307826 21174 consensus_queue.cc:260] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:36.310148 21819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:36.311187 21819 raft_consensus.cc:493] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:36.311329 21174 raft_consensus.cc:3060] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.311786 21819 raft_consensus.cc:3060] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.314203 21092 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 4.
I20260709 12:17:36.320315 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a4ea8a219940a3852084f1d3451536" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:36.321406 21173 raft_consensus.cc:3055] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:36.321851 21173 raft_consensus.cc:740] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:36.325769 21173 consensus_queue.cc:260] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:36.329994 21173 raft_consensus.cc:3060] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.330921 21766 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:36.336341 22064 raft_consensus.cc:2804] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:36.337114 22064 raft_consensus.cc:697] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 4 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:36.358425 21066 leader_election.cc:304] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:36.360517 22043 raft_consensus.cc:2804] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:36.376240 21767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 54395.6us
I20260709 12:17:36.389802 22064 consensus_queue.cc:237] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:36.394596 21812 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37558
I20260709 12:17:36.395455 21812 raft_consensus.cc:493] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:36.396051 21812 raft_consensus.cc:3060] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.379724 21174 raft_consensus.cc:2468] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:36.406574 22043 raft_consensus.cc:697] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
W20260709 12:17:36.413769 21066 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:43813, user_credentials={real_user=slave}} blocked reactor thread for 53104.3us
I20260709 12:17:36.419768 21026 raft_consensus.cc:1240] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Rejecting Update request from peer eb45c1a4fe5341368530840de4608989 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:36.423878 22043 consensus_queue.cc:237] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:36.441164 21173 raft_consensus.cc:2468] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 4.
I20260709 12:17:36.451309 21819 raft_consensus.cc:515] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:36.466459 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:36.467133 21091 raft_consensus.cc:3060] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.469089 21018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb0e4b78baa443b7a59d8155afb1dc1a" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:36.469933 21018 raft_consensus.cc:3055] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:36.470358 21018 raft_consensus.cc:740] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:36.471513 21018 consensus_queue.cc:260] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:36.475736 21819 leader_election.cc:290] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:36.477833 21018 raft_consensus.cc:3060] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.486726 21091 raft_consensus.cc:2468] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:36.488372 21766 leader_election.cc:304] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:36.491535 22051 raft_consensus.cc:2804] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:36.492084 22051 raft_consensus.cc:697] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:36.493093 22051 consensus_queue.cc:237] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:36.511448 21812 raft_consensus.cc:515] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:36.514674 21173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:36.517140 21173 raft_consensus.cc:3055] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:36.517619 21173 raft_consensus.cc:740] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:36.518793 21173 consensus_queue.cc:260] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:36.520135 21173 raft_consensus.cc:3060] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.526185 21018 raft_consensus.cc:2468] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:36.530023 21173 raft_consensus.cc:2468] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:36.545774 21764 leader_election.cc:304] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:36.544469 20847 catalog_manager.cc:5697] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from eb45c1a4fe5341368530840de4608989 (127.9.198.196) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:36.549409 20848 catalog_manager.cc:5697] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 3 to 4, leader changed from eb45c1a4fe5341368530840de4608989 (127.9.198.196) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 4 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:36.551122 20848 catalog_manager.cc:5697] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:36.552410 22051 raft_consensus.cc:2804] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:36.554334 22051 raft_consensus.cc:697] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:36.557829 22051 consensus_queue.cc:237] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:36.563923 21812 leader_election.cc:290] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:36.565009 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775061823aca4c7c887f4f14e7ceb63b" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:36.565750 21023 raft_consensus.cc:3060] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.577708 21023 raft_consensus.cc:2468] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:36.588745 20848 catalog_manager.cc:5697] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from eb45c1a4fe5341368530840de4608989 (127.9.198.196) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:36.624871 21091 raft_consensus.cc:1275] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.626062 22051 consensus_queue.cc:1048] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:36.642346 21173 raft_consensus.cc:1275] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.644825 22108 consensus_queue.cc:1048] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:36.843603 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:36.884594 21173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5"
dest_uuid: "eb45c1a4fe5341368530840de4608989"
mode: GRACEFUL
new_leader_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:17:36.885334 21173 raft_consensus.cc:606] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Received request to transfer leadership to TS 4262e2b320ed4812be271437718fd86e
I20260709 12:17:36.888051 21173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3"
dest_uuid: "eb45c1a4fe5341368530840de4608989"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:51668
I20260709 12:17:36.888617 21173 raft_consensus.cc:606] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:36.891527 21092 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
mode: GRACEFUL
new_leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
 from {username='slave'} at 127.0.0.1:42772
I20260709 12:17:36.892091 21092 raft_consensus.cc:606] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Received request to transfer leadership to TS 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:36.893343 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:36.894958 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:36.896227 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:36.900800 21091 raft_consensus.cc:1275] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.903417 22108 consensus_queue.cc:1048] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:36.928154 21092 raft_consensus.cc:1275] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 4 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:36.930284 22108 consensus_queue.cc:1048] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:36.942233 21023 raft_consensus.cc:1275] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.944087 22108 consensus_queue.cc:1048] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:36.962898 21174 raft_consensus.cc:1275] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.973623 22111 consensus_queue.cc:1048] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:36.981602 22083 raft_consensus.cc:993] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e: : Instructing follower 7f62eca1a43e41f2b41f0d621002ac36 to start an election
I20260709 12:17:36.982489 22083 raft_consensus.cc:1081] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Signalling peer 7f62eca1a43e41f2b41f0d621002ac36 to start an election
I20260709 12:17:36.979761 21018 raft_consensus.cc:1275] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.991926 22110 consensus_queue.cc:1048] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:36.994796 21023 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
 from {username='slave'} at 127.0.0.1:34554
I20260709 12:17:36.995464 21023 raft_consensus.cc:493] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:36.995905 21023 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.004357 21023 raft_consensus.cc:515] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:37.012578 21170 raft_consensus.cc:1275] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:37.026582 21023 leader_election.cc:290] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 3 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:37.033890 22051 consensus_queue.cc:1048] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:37.047050 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:37.047787 21091 raft_consensus.cc:3055] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:37.051167 21091 raft_consensus.cc:740] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:37.052093 21091 consensus_queue.cc:260] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:37.079696 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8d45e86f72047658bf7a60b30614c22" candidate_uuid: "7f62eca1a43e41f2b41f0d621002ac36" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:37.080610 21170 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.091424 21170 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 3.
I20260709 12:17:37.108006 21091 raft_consensus.cc:3060] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.132114 21091 raft_consensus.cc:2468] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62eca1a43e41f2b41f0d621002ac36 in term 3.
I20260709 12:17:37.133260 20989 leader_election.cc:304] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:37.137904 22101 raft_consensus.cc:2804] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:37.138408 22101 raft_consensus.cc:697] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 LEADER]: Becoming Leader. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:37.139278 22101 consensus_queue.cc:237] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:37.140487 21023 raft_consensus.cc:1275] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:37.142339 22064 consensus_queue.cc:1048] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.162526 21170 raft_consensus.cc:1275] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:37.164129 22108 consensus_queue.cc:1048] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.178205 20847 catalog_manager.cc:5697] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 reported cstate change: term changed from 2 to 3, leader changed from 4262e2b320ed4812be271437718fd86e (127.9.198.195) to 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "7f62eca1a43e41f2b41f0d621002ac36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:37.203650 22107 raft_consensus.cc:993] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:37.205617 22098 raft_consensus.cc:1081] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:37.207738 21817 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37558
I20260709 12:17:37.208350 21817 raft_consensus.cc:493] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:37.208627 21817 raft_consensus.cc:3060] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.214188 21817 raft_consensus.cc:515] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:37.215675 21817 leader_election.cc:290] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Requested vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:37.216578 21091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:37.217026 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981cbd4c8dab4902a1cd993cb45cd9c3" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:37.217314 21091 raft_consensus.cc:3060] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.217731 21170 raft_consensus.cc:3055] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:37.218076 21170 raft_consensus.cc:740] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:37.218822 21170 consensus_queue.cc:260] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:37.219825 21170 raft_consensus.cc:3060] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.225029 21091 raft_consensus.cc:2468] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:37.226294 21766 leader_election.cc:304] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:37.226791 21170 raft_consensus.cc:2468] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 2.
I20260709 12:17:37.227169 22108 raft_consensus.cc:2804] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:37.227782 22108 raft_consensus.cc:697] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:37.228682 22108 consensus_queue.cc:237] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:37.239192 22107 raft_consensus.cc:993] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989: : Instructing follower 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:37.239704 22098 raft_consensus.cc:1081] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Signalling peer 4262e2b320ed4812be271437718fd86e to start an election
I20260709 12:17:37.239845 20847 catalog_manager.cc:5697] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 1 to 2, leader changed from eb45c1a4fe5341368530840de4608989 (127.9.198.196) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:37.241279 21091 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
 from {username='slave'} at 127.0.0.1:43346
I20260709 12:17:37.241894 21091 raft_consensus.cc:493] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:37.242244 21091 raft_consensus.cc:3060] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.248371 21091 raft_consensus.cc:515] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:37.251173 21091 leader_election.cc:290] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535)
I20260709 12:17:37.251683 21022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:37.252301 21022 raft_consensus.cc:3060] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.252422 21170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a760357a94c70a8de37a7358e7ab5" candidate_uuid: "4262e2b320ed4812be271437718fd86e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:37.253099 21170 raft_consensus.cc:3055] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:37.253619 21170 raft_consensus.cc:740] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb45c1a4fe5341368530840de4608989, State: Running, Role: LEADER
I20260709 12:17:37.254522 21170 consensus_queue.cc:260] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:37.255733 21170 raft_consensus.cc:3060] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.259384 21022 raft_consensus.cc:2468] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:37.260538 21066 leader_election.cc:304] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36; no voters: 
I20260709 12:17:37.261387 22131 raft_consensus.cc:2804] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:37.261857 21170 raft_consensus.cc:2468] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4262e2b320ed4812be271437718fd86e in term 2.
I20260709 12:17:37.261974 22131 raft_consensus.cc:697] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Leader. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:37.263106 22131 consensus_queue.cc:237] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:37.275591 20847 catalog_manager.cc:5697] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e reported cstate change: term changed from 1 to 2, leader changed from eb45c1a4fe5341368530840de4608989 (127.9.198.196) to 4262e2b320ed4812be271437718fd86e (127.9.198.195). New cstate: current_term: 2 leader_uuid: "4262e2b320ed4812be271437718fd86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:37.596900 21170 raft_consensus.cc:1275] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:37.598125 22130 consensus_queue.cc:1048] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:37.616775 21091 raft_consensus.cc:1275] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Refusing update from remote peer 7f62eca1a43e41f2b41f0d621002ac36: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:37.618984 22101 consensus_queue.cc:1048] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:37.640291 21170 raft_consensus.cc:1275] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:37.643148 22108 consensus_queue.cc:1048] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.653748 21091 raft_consensus.cc:1275] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:37.655285 22108 consensus_queue.cc:1048] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.841846 21170 raft_consensus.cc:1275] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:37.843907 22111 consensus_queue.cc:1048] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.866662 21018 raft_consensus.cc:1275] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Refusing update from remote peer 4262e2b320ed4812be271437718fd86e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:37.879434 22131 consensus_queue.cc:1048] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
W20260709 12:17:37.895730 20989 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:41243, user_credentials={real_user=slave}} blocked reactor thread for 53865us
I20260709 12:17:37.973145 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:38.179708 21100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01"
dest_uuid: "4262e2b320ed4812be271437718fd86e"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:42772
I20260709 12:17:38.180298 21100 raft_consensus.cc:606] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:38.183450 21023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6"
dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
mode: GRACEFUL
new_leader_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:17:38.184136 21023 raft_consensus.cc:606] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Received request to transfer leadership to TS c8997852a67a4e678dfad30c76a76964
I20260709 12:17:38.185756 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:38.187093 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:38.187856 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:38.339036 22131 raft_consensus.cc:993] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:38.339658 22155 raft_consensus.cc:1081] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:38.341444 21810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37542
I20260709 12:17:38.341998 21810 raft_consensus.cc:493] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:38.342327 21810 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.348641 21810 raft_consensus.cc:515] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:38.350309 21810 leader_election.cc:290] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Requested vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:38.351218 21100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4262e2b320ed4812be271437718fd86e"
I20260709 12:17:38.351886 21100 raft_consensus.cc:3055] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:38.351835 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81cad3626c3646ed896f5fbda5206d01" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:38.352398 21100 raft_consensus.cc:740] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4262e2b320ed4812be271437718fd86e, State: Running, Role: LEADER
I20260709 12:17:38.352541 21023 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.353283 21100 consensus_queue.cc:260] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:38.354676 21100 raft_consensus.cc:3060] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.359700 21023 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:38.360891 21767 leader_election.cc:304] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964; no voters: 
I20260709 12:17:38.361624 22051 raft_consensus.cc:2804] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:38.361883 21100 raft_consensus.cc:2468] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:38.362222 22051 raft_consensus.cc:697] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:38.363130 22051 consensus_queue.cc:237] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } }
I20260709 12:17:38.375428 22101 raft_consensus.cc:993] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36: : Instructing follower c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:38.376029 22095 raft_consensus.cc:1081] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Signalling peer c8997852a67a4e678dfad30c76a76964 to start an election
I20260709 12:17:38.375352 20849 catalog_manager.cc:5697] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 2 to 3, leader changed from 4262e2b320ed4812be271437718fd86e (127.9.198.195) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:38.378134 21810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6"
dest_uuid: "c8997852a67a4e678dfad30c76a76964"
 from {username='slave'} at 127.0.0.1:37566
I20260709 12:17:38.378764 21810 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:38.379161 21810 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.389372 21810 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:38.391233 21810 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Requested vote from peers eb45c1a4fe5341368530840de4608989 (127.9.198.196:38535), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:38.392319 21023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f62eca1a43e41f2b41f0d621002ac36"
I20260709 12:17:38.393123 21023 raft_consensus.cc:3055] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:38.393438 21023 raft_consensus.cc:740] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f62eca1a43e41f2b41f0d621002ac36, State: Running, Role: LEADER
I20260709 12:17:38.393772 21169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c32186e2dec443e685ff412c8bda31a6" candidate_uuid: "c8997852a67a4e678dfad30c76a76964" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb45c1a4fe5341368530840de4608989"
I20260709 12:17:38.394421 21169 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.394282 21023 consensus_queue.cc:260] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:38.395646 21023 raft_consensus.cc:3060] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.402487 21169 raft_consensus.cc:2468] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:38.404014 21764 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8997852a67a4e678dfad30c76a76964, eb45c1a4fe5341368530840de4608989; no voters: 
I20260709 12:17:38.404184 21023 raft_consensus.cc:2468] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8997852a67a4e678dfad30c76a76964 in term 3.
I20260709 12:17:38.405128 22051 raft_consensus.cc:2804] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:38.405611 22051 raft_consensus.cc:697] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Becoming Leader. State: Replica: c8997852a67a4e678dfad30c76a76964, State: Running, Role: LEADER
I20260709 12:17:38.406493 22051 consensus_queue.cc:237] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:38.418951 20849 catalog_manager.cc:5697] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 reported cstate change: term changed from 2 to 3, leader changed from 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194) to c8997852a67a4e678dfad30c76a76964 (127.9.198.193). New cstate: current_term: 3 leader_uuid: "c8997852a67a4e678dfad30c76a76964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:38.883978 21023 raft_consensus.cc:1275] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:38.885277 22051 consensus_queue.cc:1048] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:38.902107 21092 raft_consensus.cc:1275] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:38.906960 22051 consensus_queue.cc:1048] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:17:38.942703 21121 debug-util.cc:398] Leaking SignalData structure 0x7b08001f3640 after lost signal to thread 10014
W20260709 12:17:38.944038 21121 debug-util.cc:398] Leaking SignalData structure 0x7b08003c97c0 after lost signal to thread 20883
W20260709 12:17:38.946810 21121 debug-util.cc:398] Leaking SignalData structure 0x7b0800130b60 after lost signal to thread 21050
W20260709 12:17:38.948060 21121 debug-util.cc:398] Leaking SignalData structure 0x7b0800126a20 after lost signal to thread 21124
W20260709 12:17:38.955560 21121 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7180 after lost signal to thread 21198
W20260709 12:17:38.956764 21121 debug-util.cc:398] Leaking SignalData structure 0x7b0800374ce0 after lost signal to thread 21844
I20260709 12:17:38.988374 21169 raft_consensus.cc:1275] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:38.993007 21023 raft_consensus.cc:1275] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Refusing update from remote peer c8997852a67a4e678dfad30c76a76964: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:38.994885 22064 consensus_queue.cc:1048] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:38.996234 22051 consensus_queue.cc:1048] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:39.228133 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:39.288859 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:39.290607 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:39.291294 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:39.345584 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:39.399735 10011 tablet_server.cc:179] TabletServer@127.9.198.193:41243 shutting down...
W20260709 12:17:39.402979 21063 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 60 similar messages]
W20260709 12:17:39.412603 21063 consensus_peers.cc:597] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.429780 21137 consensus_peers.cc:597] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.450424 21063 consensus_peers.cc:597] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.458599 21063 consensus_peers.cc:597] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.481428 21063 consensus_peers.cc:597] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.501657 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:39.505097 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.506278 10011 raft_consensus.cc:2243] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.507498 10011 raft_consensus.cc:2272] T c0c1d07d1b1344a5ad97b6f5a36d343a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.510805 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.511809 10011 raft_consensus.cc:2243] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.512320 10011 raft_consensus.cc:2272] T 5041cbf53c7e4c588bb5a0b699284c33 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.515385 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.516250 10011 raft_consensus.cc:2243] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.516741 10011 raft_consensus.cc:2272] T 395043703c424af7b41072d0e92012a1 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.519860 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.520759 10011 raft_consensus.cc:2243] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.521384 10011 raft_consensus.cc:2272] T 98d7127b20aa4482ae857ddb5430b70e P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.524420 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.525193 10011 raft_consensus.cc:2243] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.525864 10011 raft_consensus.cc:2272] T 6181c8aa4b7a4d2fa8163b486fec512b P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.529556 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.530433 10011 raft_consensus.cc:2243] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.531015 10011 raft_consensus.cc:2272] T 3778fc4a90d9419cbd028360957cbc4e P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.532806 20989 consensus_peers.cc:597] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.534400 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.535300 10011 raft_consensus.cc:2243] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.536271 10011 raft_consensus.cc:2272] T eb0e4b78baa443b7a59d8155afb1dc1a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.539562 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.540549 10011 raft_consensus.cc:2243] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.541105 10011 raft_consensus.cc:2272] T e65504fa010146ef8fef1ff07dfcff69 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.543088 20989 consensus_peers.cc:597] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.544842 21137 consensus_peers.cc:597] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.545239 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.546422 10011 raft_consensus.cc:2243] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.547022 10011 raft_consensus.cc:2272] T 17ab161162c04ac8b68e594a1fbabe36 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.550311 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.551636 10011 raft_consensus.cc:2243] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.552577 10011 raft_consensus.cc:2272] T 775061823aca4c7c887f4f14e7ceb63b P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.555681 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.556545 10011 raft_consensus.cc:2243] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.557689 10011 raft_consensus.cc:2272] T a47e447bb0004896847c1e74485df964 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.560892 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.561791 10011 raft_consensus.cc:2243] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.562393 10011 raft_consensus.cc:2272] T 4a416937e48a4949a7b5fba8c466eb41 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.564179 21063 consensus_peers.cc:597] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.565647 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.566783 10011 raft_consensus.cc:2243] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.567355 10011 raft_consensus.cc:2272] T ae962db47c4e4fc6a10172d6395e113e P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.570672 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.571627 10011 raft_consensus.cc:2243] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.572263 10011 raft_consensus.cc:2272] T 91f9400a74934eb4b3269b428057ccef P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.573137 21137 consensus_peers.cc:597] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.575678 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.576772 10011 raft_consensus.cc:2243] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.577374 10011 raft_consensus.cc:2272] T fc706075ca374b5eb2cc3867a4f7a456 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.580744 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.581727 10011 raft_consensus.cc:2243] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.582756 10011 raft_consensus.cc:2272] T 81cad3626c3646ed896f5fbda5206d01 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.586879 10011 tablet_replica.cc:333] stopping tablet replica
W20260709 12:17:39.586736 21137 consensus_peers.cc:597] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.587932 10011 raft_consensus.cc:2243] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.588522 10011 raft_consensus.cc:2272] T 89fda386048b4838a8b10fbe2d602568 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.592193 10011 tablet_replica.cc:333] stopping tablet replica
W20260709 12:17:39.592614 21063 consensus_peers.cc:597] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.593194 10011 raft_consensus.cc:2243] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.594107 10011 raft_consensus.cc:2272] T d4fc3437dadf4368a453b79bb7a38482 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.597573 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.598834 10011 raft_consensus.cc:2243] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.600028 10011 raft_consensus.cc:2272] T 21144a10cc4945629d70fdd253d940d1 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.602586 20989 consensus_peers.cc:597] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.603751 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.604892 10011 raft_consensus.cc:2243] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.605583 10011 raft_consensus.cc:2272] T 38019ba094624915aa5f69bc244bc955 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.609095 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.610062 10011 raft_consensus.cc:2243] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.611217 10011 raft_consensus.cc:2272] T 698227e80f3c48a28b91fa4df79955cf P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.614571 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.615504 10011 raft_consensus.cc:2243] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.616050 10011 raft_consensus.cc:2272] T f85cfddd6d98445c951b81e38136c039 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.619287 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.620209 10011 raft_consensus.cc:2243] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.620754 10011 raft_consensus.cc:2272] T e842b1b7f85a4cd9a991bb35684df6b9 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.624264 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.625140 10011 raft_consensus.cc:2243] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.626207 10011 raft_consensus.cc:2272] T c32186e2dec443e685ff412c8bda31a6 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.629992 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.630910 10011 raft_consensus.cc:2243] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.631987 10011 raft_consensus.cc:2272] T 981cbd4c8dab4902a1cd993cb45cd9c3 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.634974 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.635773 10011 raft_consensus.cc:2243] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.636327 10011 raft_consensus.cc:2272] T 033d781150df48e1ba4cefa59d033f42 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.637218 20989 consensus_peers.cc:597] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.639559 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.640573 10011 raft_consensus.cc:2243] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.641721 10011 raft_consensus.cc:2272] T d701ef5192bf4f11a84c7b7fa0660c67 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.644662 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.645519 10011 raft_consensus.cc:2243] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.646629 10011 raft_consensus.cc:2272] T 03a4ea8a219940a3852084f1d3451536 P c8997852a67a4e678dfad30c76a76964 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.649806 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.650687 10011 raft_consensus.cc:2243] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.651991 10011 raft_consensus.cc:2272] T e5569aee047642449df010d452ed73fa P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.655118 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.656025 10011 raft_consensus.cc:2243] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.656574 10011 raft_consensus.cc:2272] T 352e00dbe52446aca63ec1b6cc70f554 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.659878 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.660765 10011 raft_consensus.cc:2243] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.661275 10011 raft_consensus.cc:2272] T 807ff01cdab14b62885871bdce0d3cd8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.664412 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.665238 10011 raft_consensus.cc:2243] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.666358 10011 raft_consensus.cc:2272] T a29e7a2e40b345b09d50b1699ce33f47 P c8997852a67a4e678dfad30c76a76964 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.669770 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.670666 10011 raft_consensus.cc:2243] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.671276 10011 raft_consensus.cc:2272] T 9d53959ed0724fc1905beec27f8a0917 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.674419 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.675285 10011 raft_consensus.cc:2243] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.675858 10011 raft_consensus.cc:2272] T 044d3214b94a448394941bed6d2204a8 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.679041 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.679901 10011 raft_consensus.cc:2243] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.680500 10011 raft_consensus.cc:2272] T f00f76eb99ab413589cdbe3aafd60f9e P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.682130 21137 consensus_peers.cc:597] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.683766 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.684769 10011 raft_consensus.cc:2243] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.685320 10011 raft_consensus.cc:2272] T b3ddbde7cec2484da93f2b7075232308 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.688347 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.689148 10011 raft_consensus.cc:2243] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.689777 10011 raft_consensus.cc:2272] T ca20df817314411a9f94ee42a3b5220a P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.692873 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.693727 10011 raft_consensus.cc:2243] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.694283 10011 raft_consensus.cc:2272] T a8c82b37b2b14fcfb1d8c88765a4ba24 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.697484 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.698457 10011 raft_consensus.cc:2243] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.699312 10011 raft_consensus.cc:2272] T f01e89f8d2d941d494585d3147a3dd2f P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.701965 21137 consensus_peers.cc:597] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.702844 21137 consensus_peers.cc:597] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.702996 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.703733 10011 raft_consensus.cc:2243] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.704133 10011 raft_consensus.cc:2272] T 3e5b0a17978446c6bee0b8db24be9e49 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.706488 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.707036 10011 raft_consensus.cc:2243] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.707518 10011 raft_consensus.cc:2272] T c7a254b1018240fd8b49a2776fb01a2b P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.709928 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.710484 10011 raft_consensus.cc:2243] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.710939 10011 raft_consensus.cc:2272] T 9b6872f6d7f14caea68ac416bcb17b27 P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:39.713601 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.714156 10011 raft_consensus.cc:2243] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:39.715201 10011 raft_consensus.cc:2272] T d5353702fdb44369a77c62ee46fa7fcd P c8997852a67a4e678dfad30c76a76964 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.717031 20989 consensus_peers.cc:597] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.718497 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.719426 10011 raft_consensus.cc:2243] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.719899 10011 raft_consensus.cc:2272] T 181a9cec50b94395aac6bd156652a9f2 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.722395 20989 consensus_peers.cc:597] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.723047 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:39.723855 10011 raft_consensus.cc:2243] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:39.724382 10011 raft_consensus.cc:2272] T 16b20f7ffdc94676bd1cbade62c4dcc5 P c8997852a67a4e678dfad30c76a76964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:39.729336 21137 consensus_peers.cc:597] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.751627 21137 consensus_peers.cc:597] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.794677 20989 consensus_peers.cc:597] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.802192 21137 consensus_peers.cc:597] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.803679 20989 consensus_peers.cc:597] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.809962 21137 connection.cc:573] client connection to 127.9.198.193:41243 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:17:39.817533 21137 consensus_peers.cc:597] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.829144 10011 tablet_server.cc:196] TabletServer@127.9.198.193:41243 shutdown complete.
W20260709 12:17:39.830605 20989 consensus_peers.cc:597] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.855675 21137 consensus_peers.cc:597] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.865823 21137 consensus_peers.cc:597] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.908994 20989 consensus_peers.cc:597] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.912184 21063 consensus_peers.cc:597] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.918555 21137 consensus_peers.cc:597] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:39.954044 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
W20260709 12:17:39.974597 21140 consensus_peers.cc:597] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:39.975382 21140 consensus_peers.cc:597] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.016964 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:40.018069 10011 tablet_replica.cc:333] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
W20260709 12:17:40.018131 21066 consensus_peers.cc:597] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.019203 10011 raft_consensus.cc:2243] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.020241 10011 raft_consensus.cc:2272] T f01e89f8d2d941d494585d3147a3dd2f P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.023561 10011 tablet_replica.cc:333] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.024703 10011 raft_consensus.cc:2243] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.025782 10011 raft_consensus.cc:2272] T 3e5b0a17978446c6bee0b8db24be9e49 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.028723 10011 tablet_replica.cc:333] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.029567 10011 raft_consensus.cc:2243] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.030210 10011 raft_consensus.cc:2272] T 0468c28c130f44858399b670b2b9cb8b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.033612 10011 tablet_replica.cc:333] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.034530 10011 raft_consensus.cc:2243] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.035149 10011 raft_consensus.cc:2272] T fc706075ca374b5eb2cc3867a4f7a456 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.039971 10011 tablet_replica.cc:333] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.040901 10011 raft_consensus.cc:2243] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.041522 10011 raft_consensus.cc:2272] T 7a820defdda646daaa5ffa1d6aba18f1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.041935 21140 consensus_peers.cc:597] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.045908 10011 tablet_replica.cc:333] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.046906 10011 raft_consensus.cc:2243] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.048066 10011 raft_consensus.cc:2272] T f85cfddd6d98445c951b81e38136c039 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.051857 10011 tablet_replica.cc:333] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.052762 10011 raft_consensus.cc:2243] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.053369 10011 raft_consensus.cc:2272] T 505a760357a94c70a8de37a7358e7ab5 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.056646 10011 tablet_replica.cc:333] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.057542 10011 raft_consensus.cc:2243] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.058832 10011 raft_consensus.cc:2272] T 38019ba094624915aa5f69bc244bc955 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.062144 10011 tablet_replica.cc:333] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.063060 10011 raft_consensus.cc:2243] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.063658 10011 raft_consensus.cc:2272] T 81cad3626c3646ed896f5fbda5206d01 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.066562 10011 tablet_replica.cc:333] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.067351 10011 raft_consensus.cc:2243] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.067962 10011 raft_consensus.cc:2272] T 395043703c424af7b41072d0e92012a1 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.072702 10011 tablet_replica.cc:333] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.073578 10011 raft_consensus.cc:2243] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.074254 10011 raft_consensus.cc:2272] T b79db6c634bd4743b1cb15b5310dd4f8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.077419 10011 tablet_replica.cc:333] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.078286 10011 raft_consensus.cc:2243] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.078913 10011 raft_consensus.cc:2272] T 98d7127b20aa4482ae857ddb5430b70e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.082248 10011 tablet_replica.cc:333] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.083195 10011 raft_consensus.cc:2243] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.083865 10011 raft_consensus.cc:2272] T d5353702fdb44369a77c62ee46fa7fcd P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.087183 10011 tablet_replica.cc:333] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.088075 10011 raft_consensus.cc:2243] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.088819 10011 raft_consensus.cc:2272] T eb0e4b78baa443b7a59d8155afb1dc1a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.091755 10011 tablet_replica.cc:333] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.092988 10011 raft_consensus.cc:2243] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:17:40.093494 21140 consensus_peers.cc:597] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.094148 10011 raft_consensus.cc:2272] T 033d781150df48e1ba4cefa59d033f42 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.094340 21066 consensus_peers.cc:597] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.095045 21066 consensus_peers.cc:597] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.097457 10011 tablet_replica.cc:333] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.098599 10011 raft_consensus.cc:2243] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.099193 10011 raft_consensus.cc:2272] T d701ef5192bf4f11a84c7b7fa0660c67 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.102551 10011 tablet_replica.cc:333] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.103358 10011 raft_consensus.cc:2243] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.103888 10011 raft_consensus.cc:2272] T b3ddbde7cec2484da93f2b7075232308 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.106515 10011 tablet_replica.cc:333] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.107285 10011 raft_consensus.cc:2243] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.107873 10011 raft_consensus.cc:2272] T e5569aee047642449df010d452ed73fa P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.110966 10011 tablet_replica.cc:333] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.111779 10011 raft_consensus.cc:2243] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.112396 10011 raft_consensus.cc:2272] T c0c1d07d1b1344a5ad97b6f5a36d343a P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.115448 10011 tablet_replica.cc:333] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.116225 10011 raft_consensus.cc:2243] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.116803 10011 raft_consensus.cc:2272] T d1474ed8c5944a89ade3d329705003ca P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.119167 21066 consensus_peers.cc:597] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.119719 10011 tablet_replica.cc:333] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.120652 10011 raft_consensus.cc:2243] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.121300 10011 raft_consensus.cc:2272] T a47e447bb0004896847c1e74485df964 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.124444 10011 tablet_replica.cc:333] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.125339 10011 raft_consensus.cc:2243] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.125916 10011 raft_consensus.cc:2272] T 775061823aca4c7c887f4f14e7ceb63b P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.128666 10011 tablet_replica.cc:333] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.129478 10011 raft_consensus.cc:2243] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.130137 10011 raft_consensus.cc:2272] T f00f76eb99ab413589cdbe3aafd60f9e P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.133119 10011 tablet_replica.cc:333] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.133963 10011 raft_consensus.cc:2243] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.135082 10011 raft_consensus.cc:2272] T 044d3214b94a448394941bed6d2204a8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.138399 10011 tablet_replica.cc:333] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.139212 10011 raft_consensus.cc:2243] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.140277 10011 raft_consensus.cc:2272] T d1c050ac450e451d9a27af7cf32b7edc P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.143375 10011 tablet_replica.cc:333] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.144145 10011 raft_consensus.cc:2243] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.144709 10011 raft_consensus.cc:2272] T 741c5b82a89142c2840a00c76ede6243 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.147734 10011 tablet_replica.cc:333] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.148572 10011 raft_consensus.cc:2243] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.149137 10011 raft_consensus.cc:2272] T 5041cbf53c7e4c588bb5a0b699284c33 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.152020 10011 tablet_replica.cc:333] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.152844 10011 raft_consensus.cc:2243] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.153939 10011 raft_consensus.cc:2272] T 4a416937e48a4949a7b5fba8c466eb41 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.156608 10011 tablet_replica.cc:333] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.157203 10011 raft_consensus.cc:2243] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.158325 10011 raft_consensus.cc:2272] T e842b1b7f85a4cd9a991bb35684df6b9 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.161110 10011 tablet_replica.cc:333] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.161926 10011 raft_consensus.cc:2243] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.163066 10011 raft_consensus.cc:2272] T 7a9137ae28b54fb58af02339dca65435 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.165794 10011 tablet_replica.cc:333] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.166535 10011 raft_consensus.cc:2243] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.167091 10011 raft_consensus.cc:2272] T c32186e2dec443e685ff412c8bda31a6 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.169596 21066 consensus_peers.cc:597] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.169813 10011 tablet_replica.cc:333] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.170696 10011 raft_consensus.cc:2243] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.177397 10011 raft_consensus.cc:2272] T 3778fc4a90d9419cbd028360957cbc4e P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.181342 10011 tablet_replica.cc:333] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.182269 10011 raft_consensus.cc:2243] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.187438 10011 raft_consensus.cc:2272] T d4fc3437dadf4368a453b79bb7a38482 P 7f62eca1a43e41f2b41f0d621002ac36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.190492 10011 tablet_replica.cc:333] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.191300 10011 raft_consensus.cc:2243] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.191941 10011 raft_consensus.cc:2272] T 89fda386048b4838a8b10fbe2d602568 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.192231 21066 consensus_peers.cc:597] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.197149 10011 tablet_replica.cc:333] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.198171 10011 raft_consensus.cc:2243] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.199187 10011 raft_consensus.cc:2272] T f8d45e86f72047658bf7a60b30614c22 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.202008 10011 tablet_replica.cc:333] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.202643 10011 raft_consensus.cc:2243] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.203613 10011 raft_consensus.cc:2272] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.205940 10011 tablet_replica.cc:333] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.206650 10011 raft_consensus.cc:2243] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.207656 10011 raft_consensus.cc:2272] T 6181c8aa4b7a4d2fa8163b486fec512b P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.210480 10011 tablet_replica.cc:333] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
W20260709 12:17:40.210829 21066 consensus_peers.cc:597] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.211195 10011 raft_consensus.cc:2243] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.211820 10011 raft_consensus.cc:2272] T 72e531cd0b844ef99e5360fe8adabf3e P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.214634 10011 tablet_replica.cc:333] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.215406 10011 raft_consensus.cc:2243] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.216404 10011 raft_consensus.cc:2272] T 17ab161162c04ac8b68e594a1fbabe36 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.218036 21140 consensus_peers.cc:597] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.219245 10011 tablet_replica.cc:333] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.220139 10011 raft_consensus.cc:2243] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.221251 10011 raft_consensus.cc:2272] T fe6f9c0020a4484ca89737318ee6cc3c P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.224123 10011 tablet_replica.cc:333] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.224828 10011 raft_consensus.cc:2243] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.225375 10011 raft_consensus.cc:2272] T 93ad3b0b41544f61bdc6f80d66c26adf P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.228119 10011 tablet_replica.cc:333] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.228865 10011 raft_consensus.cc:2243] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.229434 10011 raft_consensus.cc:2272] T a29e7a2e40b345b09d50b1699ce33f47 P 7f62eca1a43e41f2b41f0d621002ac36 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.232208 10011 tablet_replica.cc:333] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.232838 10011 raft_consensus.cc:2243] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.233793 10011 raft_consensus.cc:2272] T 807ff01cdab14b62885871bdce0d3cd8 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.237485 10011 tablet_replica.cc:333] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36: stopping tablet replica
I20260709 12:17:40.238011 10011 raft_consensus.cc:2243] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.238456 10011 raft_consensus.cc:2272] T aaf04654bfa04b2d8d24ec0581483a23 P 7f62eca1a43e41f2b41f0d621002ac36 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.335143 21066 consensus_peers.cc:597] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.340008 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
W20260709 12:17:40.342420 21140 consensus_peers.cc:597] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.351763 21140 consensus_peers.cc:597] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.365499 21066 consensus_peers.cc:597] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.369385 21066 consensus_peers.cc:597] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.388842 21066 consensus_peers.cc:597] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.443807 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
W20260709 12:17:40.451818 21139 consensus_peers.cc:597] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.454718 21139 consensus_peers.cc:597] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.486984 21139 consensus_peers.cc:597] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:40.494674 21140 consensus_peers.cc:597] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.530745 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:40.533921 10011 tablet_replica.cc:333] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.534883 10011 raft_consensus.cc:2243] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.535485 10011 raft_consensus.cc:2272] T 16b20f7ffdc94676bd1cbade62c4dcc5 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.539811 10011 tablet_replica.cc:333] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.540570 10011 raft_consensus.cc:2243] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.541538 10011 raft_consensus.cc:2272] T 0468c28c130f44858399b670b2b9cb8b P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.543859 10011 tablet_replica.cc:333] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.544574 10011 raft_consensus.cc:2243] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.545084 10011 raft_consensus.cc:2272] T 7a820defdda646daaa5ffa1d6aba18f1 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.547680 10011 tablet_replica.cc:333] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
W20260709 12:17:40.547942 21139 consensus_peers.cc:597] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.548532 10011 raft_consensus.cc:2243] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.549046 10011 raft_consensus.cc:2272] T ca20df817314411a9f94ee42a3b5220a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.551358 10011 tablet_replica.cc:333] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.551952 10011 raft_consensus.cc:2243] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.552392 10011 raft_consensus.cc:2272] T f85cfddd6d98445c951b81e38136c039 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.554466 10011 tablet_replica.cc:333] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.555109 10011 raft_consensus.cc:2243] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.555915 10011 raft_consensus.cc:2272] T 352e00dbe52446aca63ec1b6cc70f554 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.557962 10011 tablet_replica.cc:333] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.558526 10011 raft_consensus.cc:2243] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.559333 10011 raft_consensus.cc:2272] T b79db6c634bd4743b1cb15b5310dd4f8 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.561299 10011 tablet_replica.cc:333] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.562004 10011 raft_consensus.cc:2243] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.562855 10011 raft_consensus.cc:2272] T 395043703c424af7b41072d0e92012a1 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.564857 10011 tablet_replica.cc:333] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.565464 10011 raft_consensus.cc:2243] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.565922 10011 raft_consensus.cc:2272] T 181a9cec50b94395aac6bd156652a9f2 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.568027 10011 tablet_replica.cc:333] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.568567 10011 raft_consensus.cc:2243] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.569437 10011 raft_consensus.cc:2272] T 741c5b82a89142c2840a00c76ede6243 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.571467 10011 tablet_replica.cc:333] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.572121 10011 raft_consensus.cc:2243] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.572546 10011 raft_consensus.cc:2272] T 81cad3626c3646ed896f5fbda5206d01 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.574633 10011 tablet_replica.cc:333] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.575181 10011 raft_consensus.cc:2243] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.575634 10011 raft_consensus.cc:2272] T d5353702fdb44369a77c62ee46fa7fcd P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.578042 10011 tablet_replica.cc:333] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.578842 10011 raft_consensus.cc:2243] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.579381 10011 raft_consensus.cc:2272] T 91f9400a74934eb4b3269b428057ccef P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.581257 21139 consensus_peers.cc:597] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.581864 10011 tablet_replica.cc:333] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.582540 10011 raft_consensus.cc:2243] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.583385 10011 raft_consensus.cc:2272] T 505a760357a94c70a8de37a7358e7ab5 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.585423 10011 tablet_replica.cc:333] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.586058 10011 raft_consensus.cc:2243] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.586860 10011 raft_consensus.cc:2272] T e65504fa010146ef8fef1ff07dfcff69 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.589586 10011 tablet_replica.cc:333] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.590444 10011 raft_consensus.cc:2243] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.591058 10011 raft_consensus.cc:2272] T c0c1d07d1b1344a5ad97b6f5a36d343a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.593827 10011 tablet_replica.cc:333] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.594411 10011 raft_consensus.cc:2243] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.595194 10011 raft_consensus.cc:2272] T d1474ed8c5944a89ade3d329705003ca P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.597360 10011 tablet_replica.cc:333] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.598002 10011 raft_consensus.cc:2243] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.598433 10011 raft_consensus.cc:2272] T 03a4ea8a219940a3852084f1d3451536 P 4262e2b320ed4812be271437718fd86e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.600346 10011 tablet_replica.cc:333] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.600921 10011 raft_consensus.cc:2243] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.601398 10011 raft_consensus.cc:2272] T e5569aee047642449df010d452ed73fa P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.603745 10011 tablet_replica.cc:333] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.604481 10011 raft_consensus.cc:2243] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.604979 10011 raft_consensus.cc:2272] T eb0e4b78baa443b7a59d8155afb1dc1a P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.607625 10011 tablet_replica.cc:333] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.608387 10011 raft_consensus.cc:2243] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.608918 10011 raft_consensus.cc:2272] T ae962db47c4e4fc6a10172d6395e113e P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.611909 10011 tablet_replica.cc:333] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.612771 10011 raft_consensus.cc:2243] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.615010 10011 raft_consensus.cc:2272] T c7a254b1018240fd8b49a2776fb01a2b P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.618137 10011 tablet_replica.cc:333] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.618781 10011 raft_consensus.cc:2243] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.619715 10011 raft_consensus.cc:2272] T 9b6872f6d7f14caea68ac416bcb17b27 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.622762 10011 tablet_replica.cc:333] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.623654 10011 raft_consensus.cc:2243] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.624223 10011 raft_consensus.cc:2272] T d1c050ac450e451d9a27af7cf32b7edc P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.627194 10011 tablet_replica.cc:333] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.628003 10011 raft_consensus.cc:2243] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.628552 10011 raft_consensus.cc:2272] T 044d3214b94a448394941bed6d2204a8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.631507 10011 tablet_replica.cc:333] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.632333 10011 raft_consensus.cc:2243] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.632956 10011 raft_consensus.cc:2272] T a29e7a2e40b345b09d50b1699ce33f47 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.635419 10011 tablet_replica.cc:333] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.636000 10011 raft_consensus.cc:2243] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.636466 10011 raft_consensus.cc:2272] T 807ff01cdab14b62885871bdce0d3cd8 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.638831 10011 tablet_replica.cc:333] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.639405 10011 raft_consensus.cc:2243] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.639819 10011 raft_consensus.cc:2272] T 9d53959ed0724fc1905beec27f8a0917 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.641992 10011 tablet_replica.cc:333] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.642623 10011 raft_consensus.cc:2243] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.643047 10011 raft_consensus.cc:2272] T f8d45e86f72047658bf7a60b30614c22 P 4262e2b320ed4812be271437718fd86e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.645043 10011 tablet_replica.cc:333] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.645591 10011 raft_consensus.cc:2243] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.646476 10011 raft_consensus.cc:2272] T 89fda386048b4838a8b10fbe2d602568 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.648674 10011 tablet_replica.cc:333] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.649250 10011 raft_consensus.cc:2243] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.649760 10011 raft_consensus.cc:2272] T 1b836c6e345b4404b5fbc1f5d42b5a15 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.651934 10011 tablet_replica.cc:333] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.652470 10011 raft_consensus.cc:2243] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.652890 10011 raft_consensus.cc:2272] T 033d781150df48e1ba4cefa59d033f42 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.655145 10011 tablet_replica.cc:333] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.655694 10011 raft_consensus.cc:2243] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.656131 10011 raft_consensus.cc:2272] T d701ef5192bf4f11a84c7b7fa0660c67 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.658654 10011 tablet_replica.cc:333] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.659291 10011 raft_consensus.cc:2243] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.659721 10011 raft_consensus.cc:2272] T fe6f9c0020a4484ca89737318ee6cc3c P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.662375 10011 tablet_replica.cc:333] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.662974 10011 raft_consensus.cc:2243] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.663849 10011 raft_consensus.cc:2272] T 93ad3b0b41544f61bdc6f80d66c26adf P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.666466 10011 tablet_replica.cc:333] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.667174 10011 raft_consensus.cc:2243] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.667665 10011 raft_consensus.cc:2272] T 981cbd4c8dab4902a1cd993cb45cd9c3 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.670351 10011 tablet_replica.cc:333] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.671085 10011 raft_consensus.cc:2243] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.672164 10011 raft_consensus.cc:2272] T f01e89f8d2d941d494585d3147a3dd2f P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.674721 10011 tablet_replica.cc:333] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.675380 10011 raft_consensus.cc:2243] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.675895 10011 raft_consensus.cc:2272] T a8c82b37b2b14fcfb1d8c88765a4ba24 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.678354 10011 tablet_replica.cc:333] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.679054 10011 raft_consensus.cc:2243] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.680033 10011 raft_consensus.cc:2272] T 72e531cd0b844ef99e5360fe8adabf3e P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.680495 22180 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:40.680938 22098 raft_consensus.cc:493] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:40.681123 22180 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:40.681504 22098 raft_consensus.cc:515] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:40.683027 10011 tablet_replica.cc:333] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.683830 10011 raft_consensus.cc:2243] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.684487 10011 raft_consensus.cc:2272] T 17ab161162c04ac8b68e594a1fbabe36 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.683662 22180 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:40.683707 22098 leader_election.cc:290] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
I20260709 12:17:40.687258 10011 tablet_replica.cc:333] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.688086 10011 raft_consensus.cc:2243] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:40.688627 21139 leader_election.cc:336] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:17:40.688685 10011 raft_consensus.cc:2272] T 7a9137ae28b54fb58af02339dca65435 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.689466 21140 leader_election.cc:336] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:40.692261 10011 tablet_replica.cc:333] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.693023 10011 raft_consensus.cc:2243] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:40.693451 21137 leader_election.cc:336] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:40.693579 10011 raft_consensus.cc:2272] T 698227e80f3c48a28b91fa4df79955cf P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.694020 21137 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:40.694856 22098 raft_consensus.cc:2749] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:40.695034 21137 leader_election.cc:336] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:40.695592 21137 leader_election.cc:304] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:40.696329 22098 raft_consensus.cc:2749] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:40.696590 10011 tablet_replica.cc:333] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.697268 10011 raft_consensus.cc:2243] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.697780 10011 raft_consensus.cc:2272] T 21144a10cc4945629d70fdd253d940d1 P 4262e2b320ed4812be271437718fd86e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.699806 10011 tablet_replica.cc:333] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e: stopping tablet replica
I20260709 12:17:40.700307 10011 raft_consensus.cc:2243] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.701179 10011 raft_consensus.cc:2272] T aaf04654bfa04b2d8d24ec0581483a23 P 4262e2b320ed4812be271437718fd86e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.733759 21139 consensus_peers.cc:597] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.788439 22098 raft_consensus.cc:493] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:40.788985 22098 raft_consensus.cc:515] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:40.790930 22098 leader_election.cc:290] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:40.794880 21140 leader_election.cc:336] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:40.796022 21137 leader_election.cc:336] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:40.796386 21137 leader_election.cc:304] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:40.797001 22098 raft_consensus.cc:2749] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:40.835529 22098 raft_consensus.cc:493] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:40.836040 22098 raft_consensus.cc:515] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:40.837848 22098 leader_election.cc:290] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:40.841799 21139 leader_election.cc:336] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:40.842316 21137 leader_election.cc:336] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:40.842705 21137 leader_election.cc:304] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:40.843314 22098 raft_consensus.cc:2749] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:40.933382 21139 consensus_peers.cc:597] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.983309 22154 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:40.983834 22154 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:40.985869 22154 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:40.990245 21137 leader_election.cc:336] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:40.991868 21140 leader_election.cc:336] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:40.992268 21140 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:40.992908 22154 raft_consensus.cc:2749] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.994765 22154 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:40.995184 22154 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:40.998027 22154 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:41.002775 21139 leader_election.cc:336] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:41.007196 21137 leader_election.cc:336] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:41.007658 21137 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.008342 22154 raft_consensus.cc:2749] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:41.030308 21139 consensus_peers.cc:597] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:41.042155 22154 raft_consensus.cc:493] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:41.042855 22154 raft_consensus.cc:515] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:41.045616 22154 leader_election.cc:290] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:41.050733 21137 leader_election.cc:336] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:41.051460 21139 leader_election.cc:336] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.051877 21139 leader_election.cc:304] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.052668 22154 raft_consensus.cc:2749] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:41.348004 22154 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.348619 22154 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:41.350306 22180 raft_consensus.cc:493] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.350687 22154 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:41.350899 22180 raft_consensus.cc:515] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:41.353060 22180 leader_election.cc:290] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:41.354030 22180 raft_consensus.cc:493] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.354532 22180 raft_consensus.cc:515] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
W20260709 12:17:41.355648 21140 leader_election.cc:336] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.358031 22180 leader_election.cc:290] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.359158 21137 leader_election.cc:336] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:41.359647 21137 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
W20260709 12:17:41.360103 21140 leader_election.cc:336] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.360340 22180 raft_consensus.cc:2749] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:41.361157 21139 leader_election.cc:336] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:41.363682 21137 leader_election.cc:336] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:41.363780 21140 leader_election.cc:336] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.364133 21137 leader_election.cc:304] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.364224 21140 leader_election.cc:304] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.364907 22180 raft_consensus.cc:2749] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.365011 22154 raft_consensus.cc:2749] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.399688 22154 raft_consensus.cc:493] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.400228 22154 raft_consensus.cc:515] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.402202 22154 leader_election.cc:290] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.406117 21139 leader_election.cc:336] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:41.406801 21140 leader_election.cc:336] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.407258 21140 leader_election.cc:304] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.408066 22154 raft_consensus.cc:2749] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.417361 22154 raft_consensus.cc:493] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.417999 22154 raft_consensus.cc:515] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.420598 22154 leader_election.cc:290] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.425964 21140 leader_election.cc:336] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:41.427502 21139 leader_election.cc:336] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.427983 21139 leader_election.cc:304] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.428791 22180 raft_consensus.cc:2749] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:41.440465 22180 raft_consensus.cc:493] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.441002 22180 raft_consensus.cc:515] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:41.442883 22180 leader_election.cc:290] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:41.447114 21137 leader_election.cc:336] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:41.448864 21140 leader_election.cc:336] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.449311 21140 leader_election.cc:304] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.450098 22180 raft_consensus.cc:2749] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.515928 22180 raft_consensus.cc:493] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.516459 22180 raft_consensus.cc:515] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:41.518471 22180 leader_election.cc:290] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:41.523641 21140 leader_election.cc:336] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:41.524181 21137 leader_election.cc:336] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:41.524612 21137 leader_election.cc:304] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.525312 22180 raft_consensus.cc:2749] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.599609 22180 raft_consensus.cc:493] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.600095 22180 raft_consensus.cc:515] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.602008 22180 leader_election.cc:290] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.606703 21140 leader_election.cc:336] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:41.607262 21139 leader_election.cc:336] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.607645 21139 leader_election.cc:304] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.608266 22180 raft_consensus.cc:2749] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.628986 22180 raft_consensus.cc:493] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.629590 22180 raft_consensus.cc:515] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.631983 22180 leader_election.cc:290] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.635773 21140 leader_election.cc:336] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:41.636420 21139 leader_election.cc:336] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.636873 21139 leader_election.cc:304] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.637604 22180 raft_consensus.cc:2749] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.641767 22180 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.642206 22180 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:41.644124 22180 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:41.648432 21137 leader_election.cc:336] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:41.649192 21140 leader_election.cc:336] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.649605 21140 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.650300 22180 raft_consensus.cc:2749] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:41.681810 22180 raft_consensus.cc:493] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.682359 22180 raft_consensus.cc:515] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:41.684603 22180 leader_election.cc:290] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:41.688546 21139 leader_election.cc:336] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:41.689163 21137 leader_election.cc:336] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:41.689606 21137 leader_election.cc:304] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.690315 22154 raft_consensus.cc:2749] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.696596 22154 raft_consensus.cc:493] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.697036 22154 raft_consensus.cc:515] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.698483 22180 raft_consensus.cc:493] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:41.699019 22154 leader_election.cc:290] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:41.699046 22180 raft_consensus.cc:515] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.701712 22180 leader_election.cc:290] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.703851 21139 leader_election.cc:336] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:41.704351 21140 leader_election.cc:336] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.704772 21140 leader_election.cc:304] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.705454 22180 raft_consensus.cc:2749] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:41.706192 21139 leader_election.cc:336] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:41.708199 21140 leader_election.cc:336] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.708557 21140 leader_election.cc:304] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.709141 22180 raft_consensus.cc:2749] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.722919 22180 raft_consensus.cc:493] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.723450 22180 raft_consensus.cc:515] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.725504 22180 leader_election.cc:290] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.730101 21140 leader_election.cc:336] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:41.730667 21139 leader_election.cc:336] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.731062 21139 leader_election.cc:304] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.731720 22180 raft_consensus.cc:2749] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.757601 22180 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.758349 22180 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:41.764231 22180 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:41.767321 21137 leader_election.cc:336] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:41.768468 21139 leader_election.cc:336] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.768939 21139 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.769759 22180 raft_consensus.cc:2749] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:41.770884 10011 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 21121)
W20260709 12:17:41.793955 21137 consensus_peers.cc:597] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:41.832767 22180 raft_consensus.cc:493] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.833432 22180 raft_consensus.cc:515] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.835721 22180 leader_election.cc:290] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:41.839511 21140 leader_election.cc:336] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:41.840163 21139 leader_election.cc:336] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.840559 21139 leader_election.cc:304] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:41.841213 22180 raft_consensus.cc:2749] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:41.843495 22180 raft_consensus.cc:493] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.844048 22180 raft_consensus.cc:515] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:41.846203 22180 leader_election.cc:290] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:41.851383 21137 leader_election.cc:336] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:41.851948 21139 leader_election.cc:336] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.852404 21139 leader_election.cc:304] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:41.853520 22180 raft_consensus.cc:2749] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:41.924902 21137 consensus_peers.cc:597] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:41.987689 22180 raft_consensus.cc:493] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.988183 22180 raft_consensus.cc:515] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.990178 22180 leader_election.cc:290] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:41.990851 22180 raft_consensus.cc:493] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.991356 22180 raft_consensus.cc:515] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:41.993705 22180 leader_election.cc:290] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:41.993832 22154 raft_consensus.cc:493] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:41.995075 22154 raft_consensus.cc:515] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
W20260709 12:17:41.996480 21139 leader_election.cc:336] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:41.997267 22154 leader_election.cc:290] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:41.997577 21140 leader_election.cc:336] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:41.999864 21140 leader_election.cc:304] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:42.000613 22154 raft_consensus.cc:2749] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:42.000243 21139 leader_election.cc:336] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:42.001878 22154 raft_consensus.cc:493] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:42.002408 22154 raft_consensus.cc:515] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:42.004567 22154 leader_election.cc:290] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:42.005115 21137 leader_election.cc:336] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:42.005437 21140 leader_election.cc:336] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:42.005995 21139 leader_election.cc:336] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:42.006038 21140 leader_election.cc:304] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:42.006582 21139 leader_election.cc:304] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:42.006892 22180 raft_consensus.cc:2749] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:42.007334 22154 raft_consensus.cc:2749] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:42.011701 21139 leader_election.cc:336] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:42.012533 22154 raft_consensus.cc:493] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:42.013187 22154 raft_consensus.cc:515] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
W20260709 12:17:42.013453 21140 leader_election.cc:336] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:42.014039 21140 leader_election.cc:304] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:42.014832 22180 raft_consensus.cc:2749] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:42.015229 22154 leader_election.cc:290] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:42.019021 21140 leader_election.cc:336] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:42.019716 21139 leader_election.cc:336] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:42.020133 21139 leader_election.cc:304] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:42.020702 22154 raft_consensus.cc:2749] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:42.124325 21137 consensus_peers.cc:597] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.155103 21137 consensus_peers.cc:597] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.178923 21137 consensus_peers.cc:597] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.218531 21137 consensus_peers.cc:597] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.257506 21137 consensus_peers.cc:597] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:42.285715 22154 raft_consensus.cc:493] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:42.286166 22154 raft_consensus.cc:515] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:42.287678 22154 leader_election.cc:290] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:42.291688 21139 leader_election.cc:336] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:42.292780 21137 leader_election.cc:336] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:42.293200 21137 leader_election.cc:304] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:42.294106 22098 raft_consensus.cc:2749] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:42.296293 21137 consensus_peers.cc:597] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.298583 21140 consensus_peers.cc:597] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.300012 21137 consensus_peers.cc:597] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.376124 21137 consensus_peers.cc:597] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.388118 21137 consensus_peers.cc:597] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.432847 21140 consensus_peers.cc:597] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.483325 21137 consensus_peers.cc:597] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.522364 21137 consensus_peers.cc:597] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:42.652002 22154 raft_consensus.cc:493] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:42.652724 22154 raft_consensus.cc:515] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:42.655601 22154 leader_election.cc:290] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:42.659191 21137 leader_election.cc:336] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:42.659984 21140 leader_election.cc:336] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:42.660516 21140 leader_election.cc:304] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:42.661343 22154 raft_consensus.cc:2749] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:42.699510 21140 consensus_peers.cc:597] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.701948 21137 consensus_peers.cc:597] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 -> Peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Couldn't send request to peer c8997852a67a4e678dfad30c76a76964. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.753373 21140 consensus_peers.cc:597] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.771306 10011 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 21121)
W20260709 12:17:42.854212 21140 consensus_peers.cc:597] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.860566 21140 consensus_peers.cc:597] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.917454 21140 consensus_peers.cc:597] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:42.923002 22098 raft_consensus.cc:493] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:42.923689 22098 raft_consensus.cc:515] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:42.925788 22098 leader_election.cc:290] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:42.929217 21140 leader_election.cc:336] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:42.930094 21137 leader_election.cc:336] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:42.930553 21137 leader_election.cc:304] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:42.931293 22098 raft_consensus.cc:2749] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:42.939162 21140 consensus_peers.cc:597] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 -> Peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Couldn't send request to peer 7f62eca1a43e41f2b41f0d621002ac36. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.950548 21139 consensus_peers.cc:597] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:42.985827 21139 consensus_peers.cc:597] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:42.997766 22098 raft_consensus.cc:493] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:42.998348 22098 raft_consensus.cc:515] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:43.000409 22098 leader_election.cc:290] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:43.004349 21139 leader_election.cc:336] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.004945 21137 leader_election.cc:336] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.005317 21137 leader_election.cc:304] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.006000 22098 raft_consensus.cc:2749] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:43.034157 21139 consensus_peers.cc:597] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:43.041567 22098 raft_consensus.cc:493] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:43.042071 22098 raft_consensus.cc:515] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:43.043790 22098 leader_election.cc:290] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:43.047219 21139 leader_election.cc:336] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.047870 21137 leader_election.cc:336] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.048324 21137 leader_election.cc:304] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.049041 22098 raft_consensus.cc:2749] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.053941 22098 raft_consensus.cc:493] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.054409 22098 raft_consensus.cc:515] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.056011 22098 leader_election.cc:290] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:43.056910 22098 raft_consensus.cc:493] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:43.057440 22098 raft_consensus.cc:515] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:43.059636 22098 leader_election.cc:290] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:43.061017 21140 leader_election.cc:336] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.062602 21139 leader_election.cc:336] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.063141 21139 leader_election.cc:304] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
W20260709 12:17:43.063575 21140 leader_election.cc:336] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.064077 22098 raft_consensus.cc:2749] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:43.067399 21137 leader_election.cc:336] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.067848 21137 leader_election.cc:304] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.068552 22098 raft_consensus.cc:2749] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.071327 22098 raft_consensus.cc:493] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c8997852a67a4e678dfad30c76a76964)
I20260709 12:17:43.071710 22098 raft_consensus.cc:515] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:43.073386 22098 leader_election.cc:290] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:43.077142 21137 leader_election.cc:336] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:43.077791 21139 leader_election.cc:336] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.078287 21139 leader_election.cc:304] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.079042 22098 raft_consensus.cc:2749] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.094697 22098 raft_consensus.cc:493] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.095342 22098 raft_consensus.cc:515] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:43.098408 22098 leader_election.cc:290] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:43.101819 21140 leader_election.cc:336] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.102478 21137 leader_election.cc:336] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.102814 21137 leader_election.cc:304] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.103400 22098 raft_consensus.cc:2749] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.129124 22098 raft_consensus.cc:493] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.129594 22098 raft_consensus.cc:515] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.131157 22098 leader_election.cc:290] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.136204 21140 leader_election.cc:336] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.142590 21139 leader_election.cc:336] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.142984 21139 leader_election.cc:304] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.143654 22098 raft_consensus.cc:2749] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:43.149905 21139 consensus_peers.cc:597] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:43.214715 21139 consensus_peers.cc:597] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:43.243525 22098 raft_consensus.cc:493] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.244124 22098 raft_consensus.cc:515] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.246294 22098 leader_election.cc:290] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.251155 21140 leader_election.cc:336] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.255789 21139 leader_election.cc:336] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.259124 22193 raft_consensus.cc:493] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.259770 22193 raft_consensus.cc:515] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.259312 21139 leader_election.cc:304] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.261621 22098 raft_consensus.cc:2749] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.263149 22193 leader_election.cc:290] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.269377 21140 leader_election.cc:336] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.273430 21139 leader_election.cc:336] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.273887 21139 leader_election.cc:304] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.274564 22193 raft_consensus.cc:2749] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.389354 22098 raft_consensus.cc:493] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.389868 22098 raft_consensus.cc:515] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.391731 22098 leader_election.cc:290] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.393524 21139 consensus_peers.cc:597] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:43.395447 21140 leader_election.cc:336] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.399206 21139 leader_election.cc:336] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.399590 21139 leader_election.cc:304] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.400185 22098 raft_consensus.cc:2749] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.451324 22098 raft_consensus.cc:493] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.451787 22098 raft_consensus.cc:515] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:43.453433 22098 leader_election.cc:290] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:43.457571 21140 leader_election.cc:336] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.458119 21137 leader_election.cc:336] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.458520 21137 leader_election.cc:304] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.459125 22098 raft_consensus.cc:2749] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:43.495162 21139 consensus_peers.cc:597] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:43.553983 22098 raft_consensus.cc:493] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.554584 22098 raft_consensus.cc:515] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:43.556960 22098 leader_election.cc:290] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:43.560899 21139 leader_election.cc:336] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.561570 21137 leader_election.cc:336] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.561986 21137 leader_election.cc:304] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.562641 22098 raft_consensus.cc:2749] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.585464 22193 raft_consensus.cc:493] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.586076 22193 raft_consensus.cc:515] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.588147 22193 leader_election.cc:290] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.592445 21139 leader_election.cc:336] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.593120 21140 leader_election.cc:336] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.593479 21140 leader_election.cc:304] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.594141 22193 raft_consensus.cc:2749] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.613324 22193 raft_consensus.cc:493] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.613881 22193 raft_consensus.cc:515] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
W20260709 12:17:43.614256 21139 consensus_peers.cc:597] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 -> Peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Couldn't send request to peer 4262e2b320ed4812be271437718fd86e. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:43.616067 22193 leader_election.cc:290] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:43.621656 21140 leader_election.cc:336] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.622315 21137 leader_election.cc:336] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.622761 21137 leader_election.cc:304] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.623471 22193 raft_consensus.cc:2749] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.646239 22193 raft_consensus.cc:493] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.646821 22193 raft_consensus.cc:515] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.649895 22193 leader_election.cc:290] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.654663 21139 leader_election.cc:336] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.660941 21140 leader_election.cc:336] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.661660 21140 leader_election.cc:304] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.662597 22193 raft_consensus.cc:2749] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.671211 22193 raft_consensus.cc:493] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.671799 22193 raft_consensus.cc:515] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:43.674140 22193 leader_election.cc:290] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
W20260709 12:17:43.677984 21137 leader_election.cc:336] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:43.679016 21140 leader_election.cc:336] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.679442 21140 leader_election.cc:304] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.680178 22193 raft_consensus.cc:2749] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.682108 22193 raft_consensus.cc:493] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.682680 22193 raft_consensus.cc:515] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.684918 22193 leader_election.cc:290] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.689950 21139 leader_election.cc:336] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.692898 21140 leader_election.cc:336] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.693311 21140 leader_election.cc:304] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.693974 22193 raft_consensus.cc:2749] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.719735 22193 raft_consensus.cc:493] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.720274 22193 raft_consensus.cc:515] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.722425 22193 leader_election.cc:290] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.726229 21140 leader_election.cc:336] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.726991 21139 leader_election.cc:336] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.727557 21139 leader_election.cc:304] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.728427 22193 raft_consensus.cc:2749] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.729148 22098 raft_consensus.cc:493] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f62eca1a43e41f2b41f0d621002ac36)
I20260709 12:17:43.729792 22098 raft_consensus.cc:515] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } } peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } }
I20260709 12:17:43.732030 22098 leader_election.cc:290] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243), 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813)
I20260709 12:17:43.733680 22098 raft_consensus.cc:493] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.734217 22098 raft_consensus.cc:515] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.736218 22098 leader_election.cc:290] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.737417 21137 leader_election.cc:336] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
W20260709 12:17:43.738046 21140 leader_election.cc:336] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.738795 21140 leader_election.cc:304] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 7f62eca1a43e41f2b41f0d621002ac36, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.739945 22098 raft_consensus.cc:2749] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:43.741624 21139 leader_election.cc:336] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.743191 21140 leader_election.cc:336] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.743614 21140 leader_election.cc:304] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.744290 22098 raft_consensus.cc:2749] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:43.771956 10011 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 21121)
I20260709 12:17:43.790596 22098 raft_consensus.cc:493] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.791239 22098 raft_consensus.cc:515] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } } peers { permanent_uuid: "c8997852a67a4e678dfad30c76a76964" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41243 } }
I20260709 12:17:43.793147 22098 leader_election.cc:290] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875), c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243)
W20260709 12:17:43.797263 21139 leader_election.cc:336] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.798025 21137 leader_election.cc:336] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c8997852a67a4e678dfad30c76a76964 (127.9.198.193:41243): Network error: Client connection negotiation failed: client connection to 127.9.198.193:41243: connect: Connection refused (error 111)
I20260709 12:17:43.798396 21137 leader_election.cc:304] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, c8997852a67a4e678dfad30c76a76964
I20260709 12:17:43.799074 22098 raft_consensus.cc:2749] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.865307 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:17:43.893381 22098 raft_consensus.cc:493] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.894014 22098 raft_consensus.cc:515] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.895829 22098 leader_election.cc:290] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.899963 21139 leader_election.cc:336] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
W20260709 12:17:43.901259 21140 leader_election.cc:336] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.901859 21140 leader_election.cc:304] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.902702 22098 raft_consensus.cc:2749] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.951465 10011 tablet_server.cc:179] TabletServer@127.9.198.196:0 shutting down...
I20260709 12:17:43.964105 22098 raft_consensus.cc:493] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
I20260709 12:17:43.964951 22098 raft_consensus.cc:515] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
I20260709 12:17:43.967598 22098 leader_election.cc:290] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
I20260709 12:17:43.970129 22098 raft_consensus.cc:493] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4262e2b320ed4812be271437718fd86e)
W20260709 12:17:43.970611 21139 leader_election.cc:336] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.970737 22098 raft_consensus.cc:515] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62eca1a43e41f2b41f0d621002ac36" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 43813 } } peers { permanent_uuid: "4262e2b320ed4812be271437718fd86e" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 42875 } } peers { permanent_uuid: "eb45c1a4fe5341368530840de4608989" member_type: VOTER last_known_addr { host: "127.9.198.196" port: 38535 } }
W20260709 12:17:43.973078 21140 leader_election.cc:336] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
I20260709 12:17:43.973544 21140 leader_election.cc:304] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.974399 22193 raft_consensus.cc:2749] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:43.974768 22098 leader_election.cc:290] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813), 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875)
W20260709 12:17:43.978870 21140 leader_election.cc:336] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f62eca1a43e41f2b41f0d621002ac36 (127.9.198.194:43813): Network error: Client connection negotiation failed: client connection to 127.9.198.194:43813: connect: Connection refused (error 111)
W20260709 12:17:43.981618 21139 leader_election.cc:336] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4262e2b320ed4812be271437718fd86e (127.9.198.195:42875): Network error: Client connection negotiation failed: client connection to 127.9.198.195:42875: connect: Connection refused (error 111)
I20260709 12:17:43.982286 21139 leader_election.cc:304] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb45c1a4fe5341368530840de4608989; no voters: 4262e2b320ed4812be271437718fd86e, 7f62eca1a43e41f2b41f0d621002ac36
I20260709 12:17:43.983083 22098 raft_consensus.cc:2749] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.999431 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:44.000512 10011 tablet_replica.cc:333] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.001335 10011 raft_consensus.cc:2243] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.002331 10011 raft_consensus.cc:2272] T 16b20f7ffdc94676bd1cbade62c4dcc5 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.004679 10011 tablet_replica.cc:333] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.005311 10011 raft_consensus.cc:2243] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.005815 10011 raft_consensus.cc:2272] T 0468c28c130f44858399b670b2b9cb8b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.007858 10011 tablet_replica.cc:333] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.008455 10011 raft_consensus.cc:2243] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.008899 10011 raft_consensus.cc:2272] T 03a4ea8a219940a3852084f1d3451536 P eb45c1a4fe5341368530840de4608989 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.011160 10011 tablet_replica.cc:333] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.011839 10011 raft_consensus.cc:2243] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.012691 10011 raft_consensus.cc:2272] T fc706075ca374b5eb2cc3867a4f7a456 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.015369 10011 tablet_replica.cc:333] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.016007 10011 raft_consensus.cc:2243] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.016844 10011 raft_consensus.cc:2272] T 7a820defdda646daaa5ffa1d6aba18f1 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.018960 10011 tablet_replica.cc:333] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.019572 10011 raft_consensus.cc:2243] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.020064 10011 raft_consensus.cc:2272] T b79db6c634bd4743b1cb15b5310dd4f8 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.022187 10011 tablet_replica.cc:333] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.022821 10011 raft_consensus.cc:2243] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.023308 10011 raft_consensus.cc:2272] T 352e00dbe52446aca63ec1b6cc70f554 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.025449 10011 tablet_replica.cc:333] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.026057 10011 raft_consensus.cc:2243] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.027336 10011 raft_consensus.cc:2272] T 181a9cec50b94395aac6bd156652a9f2 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.029524 10011 tablet_replica.cc:333] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.030318 10011 raft_consensus.cc:2243] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.031345 10011 raft_consensus.cc:2272] T 98d7127b20aa4482ae857ddb5430b70e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.033941 10011 tablet_replica.cc:333] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.034556 10011 raft_consensus.cc:2243] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.035444 10011 raft_consensus.cc:2272] T 91f9400a74934eb4b3269b428057ccef P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.037686 10011 tablet_replica.cc:333] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.038272 10011 raft_consensus.cc:2243] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.039068 10011 raft_consensus.cc:2272] T ca20df817314411a9f94ee42a3b5220a P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.041092 10011 tablet_replica.cc:333] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.041779 10011 raft_consensus.cc:2243] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.042573 10011 raft_consensus.cc:2272] T b3ddbde7cec2484da93f2b7075232308 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.044697 10011 tablet_replica.cc:333] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.045320 10011 raft_consensus.cc:2243] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.045835 10011 raft_consensus.cc:2272] T f8d45e86f72047658bf7a60b30614c22 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.048049 10011 tablet_replica.cc:333] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.048686 10011 raft_consensus.cc:2243] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.049582 10011 raft_consensus.cc:2272] T d4fc3437dadf4368a453b79bb7a38482 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.051781 10011 tablet_replica.cc:333] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.052412 10011 raft_consensus.cc:2243] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.052837 10011 raft_consensus.cc:2272] T 505a760357a94c70a8de37a7358e7ab5 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.054867 10011 tablet_replica.cc:333] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.055428 10011 raft_consensus.cc:2243] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.055841 10011 raft_consensus.cc:2272] T e65504fa010146ef8fef1ff07dfcff69 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.058228 10011 tablet_replica.cc:333] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.058840 10011 raft_consensus.cc:2243] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.059300 10011 raft_consensus.cc:2272] T d1474ed8c5944a89ade3d329705003ca P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.061322 10011 tablet_replica.cc:333] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.062016 10011 raft_consensus.cc:2243] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.062520 10011 raft_consensus.cc:2272] T 72e531cd0b844ef99e5360fe8adabf3e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.064584 10011 tablet_replica.cc:333] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.065274 10011 raft_consensus.cc:2243] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.065780 10011 raft_consensus.cc:2272] T a47e447bb0004896847c1e74485df964 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.067824 10011 tablet_replica.cc:333] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.068413 10011 raft_consensus.cc:2243] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.068888 10011 raft_consensus.cc:2272] T 775061823aca4c7c887f4f14e7ceb63b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.071009 10011 tablet_replica.cc:333] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.071662 10011 raft_consensus.cc:2243] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.072136 10011 raft_consensus.cc:2272] T 4a416937e48a4949a7b5fba8c466eb41 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.074316 10011 tablet_replica.cc:333] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.074905 10011 raft_consensus.cc:2243] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.075744 10011 raft_consensus.cc:2272] T ae962db47c4e4fc6a10172d6395e113e P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.077863 10011 tablet_replica.cc:333] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.078444 10011 raft_consensus.cc:2243] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.079277 10011 raft_consensus.cc:2272] T 9d53959ed0724fc1905beec27f8a0917 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.081260 10011 tablet_replica.cc:333] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.081919 10011 raft_consensus.cc:2243] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.082383 10011 raft_consensus.cc:2272] T 1b836c6e345b4404b5fbc1f5d42b5a15 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.084744 10011 tablet_replica.cc:333] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.085302 10011 raft_consensus.cc:2243] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.085762 10011 raft_consensus.cc:2272] T 6181c8aa4b7a4d2fa8163b486fec512b P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.087688 10011 tablet_replica.cc:333] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.088241 10011 raft_consensus.cc:2243] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.088676 10011 raft_consensus.cc:2272] T d1c050ac450e451d9a27af7cf32b7edc P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.090715 10011 tablet_replica.cc:333] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.091307 10011 raft_consensus.cc:2243] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.091724 10011 raft_consensus.cc:2272] T 741c5b82a89142c2840a00c76ede6243 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.093770 10011 tablet_replica.cc:333] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.094319 10011 raft_consensus.cc:2243] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.095106 10011 raft_consensus.cc:2272] T 5041cbf53c7e4c588bb5a0b699284c33 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.097225 10011 tablet_replica.cc:333] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.097990 10011 raft_consensus.cc:2243] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.098510 10011 raft_consensus.cc:2272] T c7a254b1018240fd8b49a2776fb01a2b P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.101029 10011 tablet_replica.cc:333] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.101665 10011 raft_consensus.cc:2243] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.102138 10011 raft_consensus.cc:2272] T 9b6872f6d7f14caea68ac416bcb17b27 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.104369 10011 tablet_replica.cc:333] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.104939 10011 raft_consensus.cc:2243] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.105357 10011 raft_consensus.cc:2272] T e842b1b7f85a4cd9a991bb35684df6b9 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.107357 10011 tablet_replica.cc:333] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.107892 10011 raft_consensus.cc:2243] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.108296 10011 raft_consensus.cc:2272] T c32186e2dec443e685ff412c8bda31a6 P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.110152 10011 tablet_replica.cc:333] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.110697 10011 raft_consensus.cc:2243] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.111145 10011 raft_consensus.cc:2272] T 7a9137ae28b54fb58af02339dca65435 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.113019 10011 tablet_replica.cc:333] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.113574 10011 raft_consensus.cc:2243] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.114471 10011 raft_consensus.cc:2272] T f00f76eb99ab413589cdbe3aafd60f9e P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.116580 10011 tablet_replica.cc:333] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.117164 10011 raft_consensus.cc:2243] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.117609 10011 raft_consensus.cc:2272] T fe6f9c0020a4484ca89737318ee6cc3c P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.119529 10011 tablet_replica.cc:333] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.120069 10011 raft_consensus.cc:2243] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.120479 10011 raft_consensus.cc:2272] T 93ad3b0b41544f61bdc6f80d66c26adf P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.122340 10011 tablet_replica.cc:333] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.122869 10011 raft_consensus.cc:2243] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.123279 10011 raft_consensus.cc:2272] T 981cbd4c8dab4902a1cd993cb45cd9c3 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.125258 10011 tablet_replica.cc:333] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.125885 10011 raft_consensus.cc:2243] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.126345 10011 raft_consensus.cc:2272] T 3e5b0a17978446c6bee0b8db24be9e49 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.128365 10011 tablet_replica.cc:333] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.128947 10011 raft_consensus.cc:2243] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.129812 10011 raft_consensus.cc:2272] T a8c82b37b2b14fcfb1d8c88765a4ba24 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.131857 10011 tablet_replica.cc:333] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.132449 10011 raft_consensus.cc:2243] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.132815 10011 raft_consensus.cc:2272] T 698227e80f3c48a28b91fa4df79955cf P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.134900 10011 tablet_replica.cc:333] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.135391 10011 raft_consensus.cc:2243] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.135785 10011 raft_consensus.cc:2272] T 38019ba094624915aa5f69bc244bc955 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.137662 10011 tablet_replica.cc:333] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.138110 10011 raft_consensus.cc:2243] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.138475 10011 raft_consensus.cc:2272] T 21144a10cc4945629d70fdd253d940d1 P eb45c1a4fe5341368530840de4608989 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.140244 10011 tablet_replica.cc:333] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.140673 10011 raft_consensus.cc:2243] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:44.141055 10011 raft_consensus.cc:2272] T aaf04654bfa04b2d8d24ec0581483a23 P eb45c1a4fe5341368530840de4608989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.142867 10011 tablet_replica.cc:333] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989: stopping tablet replica
I20260709 12:17:44.143383 10011 raft_consensus.cc:2243] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.144196 10011 raft_consensus.cc:2272] T 3778fc4a90d9419cbd028360957cbc4e P eb45c1a4fe5341368530840de4608989 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.216220 10011 tablet_server.cc:196] TabletServer@127.9.198.196:0 shutdown complete.
I20260709 12:17:44.286530 10011 master.cc:562] Master@127.9.198.254:35489 shutting down...
I20260709 12:17:44.317428 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.318164 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.318504 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P 83b03a0385cf48c2bd37621ea07c4cac: stopping tablet replica
I20260709 12:17:44.342314 10011 master.cc:584] Master@127.9.198.254:35489 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (35533 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:17:44.435047 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:44497
I20260709 12:17:44.436183 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.442349 22194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.442655 22195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.443979 22197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.447834 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:44.456632 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:44.456903 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.457036 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599464457023 us; error 0 us; skew 500 ppm
I20260709 12:17:44.457593 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.460548 10011 webserver.cc:533] Webserver started at http://127.9.198.254:39081/ using document root <none> and password file <none>
I20260709 12:17:44.461117 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.461323 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.461670 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.462845 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "b58eaca590ba4825acfb32537e46c13c"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.467969 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:17:44.471864 22202 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.472693 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:17:44.473028 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "b58eaca590ba4825acfb32537e46c13c"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.473347 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.519153 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.520495 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.562009 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:44497
I20260709 12:17:44.562111 22253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:44497 every 8 connection(s)
I20260709 12:17:44.566187 22254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.578369 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c: Bootstrap starting.
I20260709 12:17:44.583647 22254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.588485 22254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c: No bootstrap required, opened a new log
I20260709 12:17:44.590886 22254 raft_consensus.cc:359] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58eaca590ba4825acfb32537e46c13c" member_type: VOTER }
I20260709 12:17:44.591289 22254 raft_consensus.cc:385] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:44.591475 22254 raft_consensus.cc:740] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58eaca590ba4825acfb32537e46c13c, State: Initialized, Role: FOLLOWER
I20260709 12:17:44.592133 22254 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58eaca590ba4825acfb32537e46c13c" member_type: VOTER }
I20260709 12:17:44.592579 22254 raft_consensus.cc:399] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:44.592773 22254 raft_consensus.cc:493] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:44.592993 22254 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:44.598033 22254 raft_consensus.cc:515] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58eaca590ba4825acfb32537e46c13c" member_type: VOTER }
I20260709 12:17:44.598533 22254 leader_election.cc:304] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b58eaca590ba4825acfb32537e46c13c; no voters: 
I20260709 12:17:44.599633 22254 leader_election.cc:290] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:44.600044 22257 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:44.601557 22257 raft_consensus.cc:697] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 1 LEADER]: Becoming Leader. State: Replica: b58eaca590ba4825acfb32537e46c13c, State: Running, Role: LEADER
I20260709 12:17:44.602387 22257 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58eaca590ba4825acfb32537e46c13c" member_type: VOTER }
I20260709 12:17:44.602794 22254 sys_catalog.cc:565] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:44.606989 22258 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b58eaca590ba4825acfb32537e46c13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58eaca590ba4825acfb32537e46c13c" member_type: VOTER } }
I20260709 12:17:44.607622 22258 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:44.612116 22259 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58eaca590ba4825acfb32537e46c13c. Latest consensus state: current_term: 1 leader_uuid: "b58eaca590ba4825acfb32537e46c13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58eaca590ba4825acfb32537e46c13c" member_type: VOTER } }
I20260709 12:17:44.612700 22259 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:44.616441 22265 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:44.624523 22265 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:44.625761 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:17:44.634593 22265 catalog_manager.cc:1383] Generated new cluster ID: fda74715ae89437c961f4a048c91b945
I20260709 12:17:44.634904 22265 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:44.658085 22265 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:44.659585 22265 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:44.677492 22265 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c: Generated new TSK 0
I20260709 12:17:44.678272 22265 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:44.692627 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.699687 22275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.701153 22276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.703265 22278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.703861 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:44.705273 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:44.705533 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.705780 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599464705758 us; error 0 us; skew 500 ppm
I20260709 12:17:44.706478 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.709138 10011 webserver.cc:533] Webserver started at http://127.9.198.193:43013/ using document root <none> and password file <none>
I20260709 12:17:44.709793 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.710037 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.710345 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.711601 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "18929f381fe84136b29bfbcc3c9aad3f"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.717311 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:17:44.721369 22283 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.722317 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:17:44.722648 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "18929f381fe84136b29bfbcc3c9aad3f"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.722965 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.764475 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.765945 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.767690 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:44.770365 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:44.770624 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.770903 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:44.771095 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.817216 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:42209
I20260709 12:17:44.817330 22345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:42209 every 8 connection(s)
I20260709 12:17:44.822907 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.831617 22351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.834717 22353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.834990 22350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.836851 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:44.838279 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:44.838577 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.838810 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599464838789 us; error 0 us; skew 500 ppm
I20260709 12:17:44.839546 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.841143 22346 heartbeater.cc:344] Connected to a master server at 127.9.198.254:44497
I20260709 12:17:44.841559 22346 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:44.842446 22346 heartbeater.cc:507] Master 127.9.198.254:44497 requested a full tablet report, sending...
I20260709 12:17:44.842742 10011 webserver.cc:533] Webserver started at http://127.9.198.194:39961/ using document root <none> and password file <none>
I20260709 12:17:44.843513 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.843783 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.844138 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.844918 22219 ts_manager.cc:194] Registered new tserver with Master: 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209)
I20260709 12:17:44.845477 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "5823e004c86c4257b8b0e25da0fb8102"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.846912 22219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37776
I20260709 12:17:44.851004 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:17:44.855057 22358 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.855919 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:17:44.856264 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "5823e004c86c4257b8b0e25da0fb8102"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.856602 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.871646 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.873031 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.874810 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:44.877441 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:44.877732 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:17:44.878010 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:44.878211 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.926321 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:33705
I20260709 12:17:44.926471 22420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:33705 every 8 connection(s)
I20260709 12:17:44.932101 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.940912 22424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.943843 22425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.945587 22427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.947593 22421 heartbeater.cc:344] Connected to a master server at 127.9.198.254:44497
I20260709 12:17:44.947913 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:44.948102 22421 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:44.949079 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:17:44.949105 22421 heartbeater.cc:507] Master 127.9.198.254:44497 requested a full tablet report, sending...
W20260709 12:17:44.949441 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.949818 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599464949789 us; error 0 us; skew 500 ppm
I20260709 12:17:44.950675 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.951953 22219 ts_manager.cc:194] Registered new tserver with Master: 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:44.953860 10011 webserver.cc:533] Webserver started at http://127.9.198.195:45959/ using document root <none> and password file <none>
I20260709 12:17:44.954296 22219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37788
I20260709 12:17:44.954672 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.955006 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.955396 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.956741 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "d2c2eb9733334ac587cd465847499c71"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.961937 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:17:44.965755 22432 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.966749 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:17:44.967131 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "d2c2eb9733334ac587cd465847499c71"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:44.967568 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.984684 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.986121 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.987824 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:44.990666 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:44.990957 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.991254 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:44.991483 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:45.036294 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:34033
I20260709 12:17:45.036428 22494 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:34033 every 8 connection(s)
I20260709 12:17:45.051527 22495 heartbeater.cc:344] Connected to a master server at 127.9.198.254:44497
I20260709 12:17:45.051949 22495 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:45.052717 22495 heartbeater.cc:507] Master 127.9.198.254:44497 requested a full tablet report, sending...
I20260709 12:17:45.054801 22219 ts_manager.cc:194] Registered new tserver with Master: d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.055037 10011 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014918859s
I20260709 12:17:45.056242 22219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37800
I20260709 12:17:45.085024 22219 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37806:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:45.088320 22219 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:45.175836 22385 tablet_service.cc:1511] Processing CreateTablet for tablet f12134c8100e45b8b2321b8c6b1e342e (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:17:45.177238 22385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12134c8100e45b8b2321b8c6b1e342e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.178751 22383 tablet_service.cc:1511] Processing CreateTablet for tablet dd63024784244f99b9dd7cb8d9fd67c9 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:17:45.180375 22383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd63024784244f99b9dd7cb8d9fd67c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.182646 22384 tablet_service.cc:1511] Processing CreateTablet for tablet 5a0dae4b76494f108367efbd4676b0de (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:17:45.183473 22386 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf4075e9de7430bbbf1d3d362a5b585 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:17:45.186790 22311 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf4075e9de7430bbbf1d3d362a5b585 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:17:45.187381 22310 tablet_service.cc:1511] Processing CreateTablet for tablet f12134c8100e45b8b2321b8c6b1e342e (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:17:45.188215 22382 tablet_service.cc:1511] Processing CreateTablet for tablet 050a9774d2d84e8686f55f7198286712 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:17:45.190555 22309 tablet_service.cc:1511] Processing CreateTablet for tablet 5a0dae4b76494f108367efbd4676b0de (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:17:45.194681 22384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a0dae4b76494f108367efbd4676b0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.196096 22378 tablet_service.cc:1511] Processing CreateTablet for tablet 50f9c35d2acf470c9b81f6d5e71b16e5 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:17:45.196859 22307 tablet_service.cc:1511] Processing CreateTablet for tablet 050a9774d2d84e8686f55f7198286712 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:17:45.194247 22308 tablet_service.cc:1511] Processing CreateTablet for tablet dd63024784244f99b9dd7cb8d9fd67c9 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:17:45.197690 22378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f9c35d2acf470c9b81f6d5e71b16e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.198724 22382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050a9774d2d84e8686f55f7198286712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.202241 22304 tablet_service.cc:1511] Processing CreateTablet for tablet 0fc7aca87fcd41549f8f2e90a42cb925 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:17:45.203114 22303 tablet_service.cc:1511] Processing CreateTablet for tablet 50f9c35d2acf470c9b81f6d5e71b16e5 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:17:45.200174 22306 tablet_service.cc:1511] Processing CreateTablet for tablet b939523530634117855680eb8e3713ea (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:17:45.201608 22377 tablet_service.cc:1511] Processing CreateTablet for tablet 899030d87cdd4814a01f394782e836b6 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:17:45.206188 22377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899030d87cdd4814a01f394782e836b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.206488 22302 tablet_service.cc:1511] Processing CreateTablet for tablet 899030d87cdd4814a01f394782e836b6 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:17:45.210145 22386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf4075e9de7430bbbf1d3d362a5b585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.213914 22311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf4075e9de7430bbbf1d3d362a5b585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.214722 22302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899030d87cdd4814a01f394782e836b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.190738 22380 tablet_service.cc:1511] Processing CreateTablet for tablet e41a3923298149d5bb746f9af9916a17 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:17:45.192348 22381 tablet_service.cc:1511] Processing CreateTablet for tablet b939523530634117855680eb8e3713ea (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:17:45.191680 22379 tablet_service.cc:1511] Processing CreateTablet for tablet 0fc7aca87fcd41549f8f2e90a42cb925 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:17:45.218804 22380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41a3923298149d5bb746f9af9916a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.219457 22381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b939523530634117855680eb8e3713ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.221841 22379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fc7aca87fcd41549f8f2e90a42cb925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.222121 22308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd63024784244f99b9dd7cb8d9fd67c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.225438 22306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b939523530634117855680eb8e3713ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.227010 22307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050a9774d2d84e8686f55f7198286712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.230139 22310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12134c8100e45b8b2321b8c6b1e342e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.202109 22305 tablet_service.cc:1511] Processing CreateTablet for tablet e41a3923298149d5bb746f9af9916a17 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:17:45.231993 22305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41a3923298149d5bb746f9af9916a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.234134 22303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f9c35d2acf470c9b81f6d5e71b16e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.235038 22304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fc7aca87fcd41549f8f2e90a42cb925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.238221 22309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a0dae4b76494f108367efbd4676b0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.287412 22510 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:45.297117 22510 tablet_bootstrap.cc:654] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.318269 22455 tablet_service.cc:1511] Processing CreateTablet for tablet b939523530634117855680eb8e3713ea (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:17:45.318770 22460 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf4075e9de7430bbbf1d3d362a5b585 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:17:45.320004 22455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b939523530634117855680eb8e3713ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.321149 22459 tablet_service.cc:1511] Processing CreateTablet for tablet f12134c8100e45b8b2321b8c6b1e342e (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:17:45.322664 22459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12134c8100e45b8b2321b8c6b1e342e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.322850 22454 tablet_service.cc:1511] Processing CreateTablet for tablet e41a3923298149d5bb746f9af9916a17 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:17:45.324204 22454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41a3923298149d5bb746f9af9916a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.324625 22458 tablet_service.cc:1511] Processing CreateTablet for tablet 5a0dae4b76494f108367efbd4676b0de (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:17:45.326077 22458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a0dae4b76494f108367efbd4676b0de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.330874 22457 tablet_service.cc:1511] Processing CreateTablet for tablet dd63024784244f99b9dd7cb8d9fd67c9 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:17:45.332437 22457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd63024784244f99b9dd7cb8d9fd67c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.334877 22456 tablet_service.cc:1511] Processing CreateTablet for tablet 050a9774d2d84e8686f55f7198286712 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:17:45.336377 22456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050a9774d2d84e8686f55f7198286712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.347163 22460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf4075e9de7430bbbf1d3d362a5b585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.355398 22453 tablet_service.cc:1511] Processing CreateTablet for tablet 0fc7aca87fcd41549f8f2e90a42cb925 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:17:45.357164 22453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fc7aca87fcd41549f8f2e90a42cb925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.360158 22452 tablet_service.cc:1511] Processing CreateTablet for tablet 50f9c35d2acf470c9b81f6d5e71b16e5 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:17:45.362691 22452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50f9c35d2acf470c9b81f6d5e71b16e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.363651 22451 tablet_service.cc:1511] Processing CreateTablet for tablet 899030d87cdd4814a01f394782e836b6 (DEFAULT_TABLE table=test-workload [id=4b272d1c85874f11a5a07be9836f96e7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:17:45.365111 22451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899030d87cdd4814a01f394782e836b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:45.373200 22513 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.386569 22513 tablet_bootstrap.cc:654] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.399495 22513 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:45.401412 22513 ts_tablet_manager.cc:1403] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.000s
I20260709 12:17:45.411476 22513 raft_consensus.cc:359] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.415939 22513 raft_consensus.cc:385] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.416814 22513 raft_consensus.cc:740] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.418543 22513 consensus_queue.cc:260] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.425796 22511 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:45.427858 22495 heartbeater.cc:499] Master 127.9.198.254:44497 was elected leader, sending a full tablet report...
I20260709 12:17:45.432024 22516 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.432473 22510 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.433125 22510 ts_tablet_manager.cc:1403] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.146s	user 0.026s	sys 0.039s
I20260709 12:17:45.432631 22516 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.435992 22516 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:45.436760 22510 raft_consensus.cc:359] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.437471 22510 raft_consensus.cc:385] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.443591 22510 raft_consensus.cc:740] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.444674 22510 consensus_queue.cc:260] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.479779 22511 tablet_bootstrap.cc:654] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.488001 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
W20260709 12:17:45.490314 22433 leader_election.cc:343] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.493240 22513 ts_tablet_manager.cc:1434] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.091s	user 0.022s	sys 0.045s
I20260709 12:17:45.497571 22513 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.511027 22510 ts_tablet_manager.cc:1434] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.078s	user 0.013s	sys 0.005s
I20260709 12:17:45.511933 22421 heartbeater.cc:499] Master 127.9.198.254:44497 was elected leader, sending a full tablet report...
I20260709 12:17:45.512302 22510 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:45.514262 22513 tablet_bootstrap.cc:654] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.519951 22510 tablet_bootstrap.cc:654] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.525964 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
W20260709 12:17:45.528084 22434 leader_election.cc:343] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.528813 22434 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:45.530774 22515 raft_consensus.cc:2749] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:45.532001 22511 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
W20260709 12:17:45.541332 22496 tablet.cc:2406] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:17:45.542347 22511 ts_tablet_manager.cc:1403] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.006s
I20260709 12:17:45.551707 22511 raft_consensus.cc:359] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.552711 22510 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.553552 22510 ts_tablet_manager.cc:1403] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.021s
I20260709 12:17:45.556211 22513 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:45.556861 22513 ts_tablet_manager.cc:1403] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.060s	user 0.023s	sys 0.022s
I20260709 12:17:45.557047 22510 raft_consensus.cc:359] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.557931 22510 raft_consensus.cc:385] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.558277 22510 raft_consensus.cc:740] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.559144 22510 consensus_queue.cc:260] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.553020 22511 raft_consensus.cc:385] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.560089 22511 raft_consensus.cc:740] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.560753 22513 raft_consensus.cc:359] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.561124 22511 consensus_queue.cc:260] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.561616 22513 raft_consensus.cc:385] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.561951 22510 ts_tablet_manager.cc:1434] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:45.562114 22513 raft_consensus.cc:740] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.563230 22510 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:45.563377 22513 consensus_queue.cc:260] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.567222 22513 ts_tablet_manager.cc:1434] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:17:45.569868 22510 tablet_bootstrap.cc:654] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.572786 22513 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.574996 22511 ts_tablet_manager.cc:1434] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.032s	user 0.010s	sys 0.005s
I20260709 12:17:45.575994 22346 heartbeater.cc:499] Master 127.9.198.254:44497 was elected leader, sending a full tablet report...
I20260709 12:17:45.587153 22513 tablet_bootstrap.cc:654] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.597293 22511 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:45.605225 22511 tablet_bootstrap.cc:654] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.610692 22513 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:45.611006 22510 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.611291 22513 ts_tablet_manager.cc:1403] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.000s
I20260709 12:17:45.611724 22510 ts_tablet_manager.cc:1403] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.011s
W20260709 12:17:45.613291 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:45.614799 22513 raft_consensus.cc:359] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.615615 22513 raft_consensus.cc:385] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.615926 22513 raft_consensus.cc:740] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.615844 22510 raft_consensus.cc:359] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.616734 22510 raft_consensus.cc:385] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.617057 22510 raft_consensus.cc:740] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.617019 22513 consensus_queue.cc:260] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.618130 22510 consensus_queue.cc:260] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.619526 22513 ts_tablet_manager.cc:1434] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:17:45.620632 22513 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.620735 22511 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:45.622452 22510 ts_tablet_manager.cc:1434] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:17:45.624101 22511 ts_tablet_manager.cc:1403] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.004s
I20260709 12:17:45.624801 22510 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:45.627485 22513 tablet_bootstrap.cc:654] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.628321 22511 raft_consensus.cc:359] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.629102 22511 raft_consensus.cc:385] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.629416 22511 raft_consensus.cc:740] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.630355 22511 consensus_queue.cc:260] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.636685 22511 ts_tablet_manager.cc:1434] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:17:45.637952 22511 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:45.643554 22510 tablet_bootstrap.cc:654] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.646719 22511 tablet_bootstrap.cc:654] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.652614 22513 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:45.653116 22513 ts_tablet_manager.cc:1403] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.008s
I20260709 12:17:45.653486 22511 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:45.654112 22511 ts_tablet_manager.cc:1403] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:17:45.655215 22510 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.655788 22510 ts_tablet_manager.cc:1403] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.006s
I20260709 12:17:45.656108 22513 raft_consensus.cc:359] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.656601 22511 raft_consensus.cc:359] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.656947 22513 raft_consensus.cc:385] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.657368 22511 raft_consensus.cc:385] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.657416 22513 raft_consensus.cc:740] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.657899 22511 raft_consensus.cc:740] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.658659 22513 consensus_queue.cc:260] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.658782 22511 consensus_queue.cc:260] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.659015 22510 raft_consensus.cc:359] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.659806 22510 raft_consensus.cc:385] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.660163 22510 raft_consensus.cc:740] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.661150 22510 consensus_queue.cc:260] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.661430 22511 ts_tablet_manager.cc:1434] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:17:45.661430 22513 ts_tablet_manager.cc:1434] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.005s
I20260709 12:17:45.662966 22513 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.663532 22510 ts_tablet_manager.cc:1434] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:17:45.664695 22510 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:45.667423 22511 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:45.670936 22513 tablet_bootstrap.cc:654] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.672025 22510 tablet_bootstrap.cc:654] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.676582 22511 tablet_bootstrap.cc:654] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.681610 22510 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.682175 22510 ts_tablet_manager.cc:1403] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 12:17:45.685182 22510 raft_consensus.cc:359] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.685945 22510 raft_consensus.cc:385] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.686007 22518 raft_consensus.cc:493] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.686328 22510 raft_consensus.cc:740] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.686566 22518 raft_consensus.cc:515] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.687788 22510 consensus_queue.cc:260] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.689863 22511 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:45.689937 22522 raft_consensus.cc:493] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.690671 22511 ts_tablet_manager.cc:1403] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.009s
I20260709 12:17:45.703069 22518 leader_election.cc:290] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.703254 22511 raft_consensus.cc:359] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.704435 22511 raft_consensus.cc:385] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.704859 22511 raft_consensus.cc:740] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.705018 22515 raft_consensus.cc:493] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.690660 22522 raft_consensus.cc:515] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.705664 22515 raft_consensus.cc:515] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.706027 22511 consensus_queue.cc:260] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.696893 22513 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:45.708811 22515 leader_election.cc:290] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:45.710103 22522 leader_election.cc:290] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.710673 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:45.715230 22510 ts_tablet_manager.cc:1434] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.033s	user 0.023s	sys 0.001s
I20260709 12:17:45.718701 22513 ts_tablet_manager.cc:1403] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.008s
I20260709 12:17:45.723070 22396 raft_consensus.cc:2468] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 0.
I20260709 12:17:45.724927 22433 leader_election.cc:304] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:45.725863 22515 raft_consensus.cc:2804] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.726274 22515 raft_consensus.cc:493] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.726701 22515 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.726056 22513 raft_consensus.cc:359] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.739466 22513 raft_consensus.cc:385] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.739835 22513 raft_consensus.cc:740] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.740633 22513 consensus_queue.cc:260] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.742590 22510 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:45.743824 22515 raft_consensus.cc:515] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.746227 22513 ts_tablet_manager.cc:1434] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 12:17:45.747553 22513 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.750824 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
I20260709 12:17:45.752728 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
W20260709 12:17:45.754118 22434 leader_election.cc:343] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.761152 22522 raft_consensus.cc:493] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.761998 22522 raft_consensus.cc:515] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.764262 22513 tablet_bootstrap.cc:654] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.764642 22522 leader_election.cc:290] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.765537 22515 leader_election.cc:290] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
W20260709 12:17:45.766323 22434 leader_election.cc:343] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.768769 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:45.770947 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:45.771819 22396 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.771693 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
W20260709 12:17:45.774200 22284 leader_election.cc:343] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.776950 22510 tablet_bootstrap.cc:654] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.777379 22511 ts_tablet_manager.cc:1434] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.078s	user 0.012s	sys 0.012s
W20260709 12:17:45.775100 22284 leader_election.cc:343] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.782586 22511 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:45.783695 22396 raft_consensus.cc:2468] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 1.
I20260709 12:17:45.785025 22433 leader_election.cc:304] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:45.788564 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
W20260709 12:17:45.791164 22360 leader_election.cc:343] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.791841 22511 tablet_bootstrap.cc:654] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.793234 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:45.793035 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:45.794171 22469 raft_consensus.cc:2468] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 0.
I20260709 12:17:45.786721 22515 raft_consensus.cc:2804] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.796943 22515 raft_consensus.cc:697] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:45.798350 22515 consensus_queue.cc:237] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
W20260709 12:17:45.807451 22285 leader_election.cc:343] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.810081 22510 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.810264 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:45.810777 22510 ts_tablet_manager.cc:1403] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.068s	user 0.013s	sys 0.000s
I20260709 12:17:45.810983 22511 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:45.811581 22511 ts_tablet_manager.cc:1403] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.013s
I20260709 12:17:45.808631 22285 leader_election.cc:304] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f; no voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71
I20260709 12:17:45.812501 22536 raft_consensus.cc:2749] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:17:45.812922 22360 leader_election.cc:343] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.814620 22285 leader_election.cc:304] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:45.815582 22536 raft_consensus.cc:2804] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.815973 22536 raft_consensus.cc:493] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.816344 22536 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.816102 22510 raft_consensus.cc:359] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.817067 22510 raft_consensus.cc:385] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.817420 22510 raft_consensus.cc:740] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.816576 22216 catalog_manager.cc:5697] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.817696 22511 raft_consensus.cc:359] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.818504 22511 raft_consensus.cc:385] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.818789 22511 raft_consensus.cc:740] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.818667 22360 leader_election.cc:304] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102; no voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71
I20260709 12:17:45.819840 22511 consensus_queue.cc:260] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.821537 22513 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:45.822216 22513 ts_tablet_manager.cc:1403] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.075s	user 0.025s	sys 0.009s
I20260709 12:17:45.822271 22511 ts_tablet_manager.cc:1434] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:17:45.823467 22511 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:45.826042 22532 raft_consensus.cc:2749] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:45.828303 22513 raft_consensus.cc:359] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:45.829545 22513 raft_consensus.cc:385] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.830029 22513 raft_consensus.cc:740] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.830260 22511 tablet_bootstrap.cc:654] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.831224 22513 consensus_queue.cc:260] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:45.834586 22513 ts_tablet_manager.cc:1434] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:17:45.835844 22513 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.840651 22536 raft_consensus.cc:515] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.837982 22510 consensus_queue.cc:260] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.844625 22510 ts_tablet_manager.cc:1434] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.001s
I20260709 12:17:45.845346 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:45.846192 22510 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
W20260709 12:17:45.848086 22284 leader_election.cc:343] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.851502 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:45.852401 22469 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.861790 22536 leader_election.cc:290] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Requested vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.862432 22510 tablet_bootstrap.cc:654] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.866899 22513 tablet_bootstrap.cc:654] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.868854 22469 raft_consensus.cc:2468] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:45.870438 22511 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:45.870458 22285 leader_election.cc:304] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:45.871351 22511 ts_tablet_manager.cc:1403] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.048s	user 0.022s	sys 0.024s
I20260709 12:17:45.872251 22536 raft_consensus.cc:2804] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.873204 22532 raft_consensus.cc:493] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.873934 22532 raft_consensus.cc:515] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.874959 22511 raft_consensus.cc:359] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:45.875903 22511 raft_consensus.cc:385] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.876206 22511 raft_consensus.cc:740] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.876567 22532 leader_election.cc:290] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.877164 22518 raft_consensus.cc:493] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.877226 22511 consensus_queue.cc:260] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:45.878579 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
I20260709 12:17:45.879379 22321 raft_consensus.cc:2468] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 0.
I20260709 12:17:45.880506 22511 ts_tablet_manager.cc:1434] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:17:45.881830 22511 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:45.882001 22360 leader_election.cc:304] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:45.878166 22518 raft_consensus.cc:515] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.883324 22532 raft_consensus.cc:2804] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.883930 22532 raft_consensus.cc:493] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.884284 22532 raft_consensus.cc:3060] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.885164 22518 leader_election.cc:290] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.887434 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12134c8100e45b8b2321b8c6b1e342e" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
I20260709 12:17:45.888159 22321 raft_consensus.cc:2468] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 0.
I20260709 12:17:45.889735 22360 leader_election.cc:304] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:45.873909 22536 raft_consensus.cc:697] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Becoming Leader. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Running, Role: LEADER
I20260709 12:17:45.893384 22522 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.893926 22522 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.892988 22518 raft_consensus.cc:2804] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.895218 22518 raft_consensus.cc:493] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.895576 22518 raft_consensus.cc:3060] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.896276 22513 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:45.896489 22522 leader_election.cc:290] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.897235 22513 ts_tablet_manager.cc:1403] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.006s
I20260709 12:17:45.898998 22511 tablet_bootstrap.cc:654] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.897311 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:45.899740 22396 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 0.
I20260709 12:17:45.901211 22284 leader_election.cc:304] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:45.902396 22522 raft_consensus.cc:2804] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.902843 22522 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.903208 22522 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.904310 22513 raft_consensus.cc:359] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.892295 22536 consensus_queue.cc:237] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.905603 22513 raft_consensus.cc:385] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.917022 22513 raft_consensus.cc:740] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.918026 22513 consensus_queue.cc:260] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.919174 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:45.920869 22513 ts_tablet_manager.cc:1434] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
W20260709 12:17:45.921406 22360 leader_election.cc:343] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.922205 22513 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:45.912262 22532 raft_consensus.cc:515] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.926091 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12134c8100e45b8b2321b8c6b1e342e" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:45.915506 22510 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.927170 22470 raft_consensus.cc:2468] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 0.
I20260709 12:17:45.927606 22532 leader_election.cc:290] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.928053 22510 ts_tablet_manager.cc:1403] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.082s	user 0.023s	sys 0.027s
I20260709 12:17:45.930154 22321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:45.931247 22321 raft_consensus.cc:3060] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.933032 22510 raft_consensus.cc:359] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.934441 22510 raft_consensus.cc:385] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.934928 22510 raft_consensus.cc:740] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.935649 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:45.936117 22510 consensus_queue.cc:260] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
W20260709 12:17:45.942935 22360 leader_election.cc:343] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.947059 22510 ts_tablet_manager.cc:1434] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.000s
I20260709 12:17:45.948640 22510 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:45.954125 22513 tablet_bootstrap.cc:654] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.954668 22532 raft_consensus.cc:493] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.955282 22532 raft_consensus.cc:515] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.958118 22532 leader_election.cc:290] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:45.965026 22510 tablet_bootstrap.cc:654] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.967016 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
W20260709 12:17:45.969728 22360 leader_election.cc:343] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.976577 22510 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:45.977437 22510 ts_tablet_manager.cc:1403] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.000s
I20260709 12:17:45.984724 22515 raft_consensus.cc:493] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.985715 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:45.985617 22515 raft_consensus.cc:515] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:45.986858 22470 raft_consensus.cc:2468] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 0.
I20260709 12:17:45.988530 22360 leader_election.cc:304] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:45.989681 22532 raft_consensus.cc:2804] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.990046 22532 raft_consensus.cc:493] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.990406 22532 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.991453 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
I20260709 12:17:45.992497 22320 raft_consensus.cc:2468] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 0.
I20260709 12:17:45.994321 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:45.994199 22434 leader_election.cc:304] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:45.995611 22516 raft_consensus.cc:2804] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.995797 22518 raft_consensus.cc:515] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:45.996232 22516 raft_consensus.cc:493] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.996841 22516 raft_consensus.cc:3060] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.984632 22513 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:46.000797 22513 ts_tablet_manager.cc:1403] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.006s
I20260709 12:17:46.001610 22515 leader_election.cc:290] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209)
I20260709 12:17:46.007941 22513 raft_consensus.cc:359] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.008944 22513 raft_consensus.cc:385] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.010213 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12134c8100e45b8b2321b8c6b1e342e" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:46.011286 22320 raft_consensus.cc:3060] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.016430 22321 raft_consensus.cc:2468] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:46.010254 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.009356 22513 raft_consensus.cc:740] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.018921 22532 raft_consensus.cc:515] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.002316 22510 raft_consensus.cc:359] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
W20260709 12:17:45.999421 22433 leader_election.cc:343] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:46.024871 22510 raft_consensus.cc:385] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.025275 22510 raft_consensus.cc:740] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.027064 22513 consensus_queue.cc:260] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.030160 22360 leader_election.cc:304] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: d2c2eb9733334ac587cd465847499c71
I20260709 12:17:46.025840 22469 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 0.
I20260709 12:17:46.031091 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12134c8100e45b8b2321b8c6b1e342e" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:46.033895 22470 raft_consensus.cc:3060] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.036516 22320 raft_consensus.cc:2468] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:46.039656 22532 leader_election.cc:290] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.040344 22532 raft_consensus.cc:2804] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.040912 22532 raft_consensus.cc:697] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:46.041059 22511 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:46.041792 22511 ts_tablet_manager.cc:1403] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.160s	user 0.068s	sys 0.043s
I20260709 12:17:46.041988 22532 consensus_queue.cc:237] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.043188 22216 catalog_manager.cc:5697] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193). New cstate: current_term: 1 leader_uuid: "18929f381fe84136b29bfbcc3c9aad3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.045307 22516 raft_consensus.cc:515] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:46.051096 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:46.053894 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:46.055274 22516 leader_election.cc:290] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 election: Requested vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209)
I20260709 12:17:46.056290 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
W20260709 12:17:46.056574 22433 leader_election.cc:343] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:46.057019 22469 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.057018 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:46.057907 22320 raft_consensus.cc:3060] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.060000 22360 leader_election.cc:304] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:46.061178 22553 raft_consensus.cc:2804] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.047343 22513 ts_tablet_manager.cc:1434] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 12:17:46.022500 22522 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.047257 22511 raft_consensus.cc:359] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.048134 22510 consensus_queue.cc:260] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.025965 22518 leader_election.cc:290] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.063467 22470 raft_consensus.cc:2468] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:46.063700 22513 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:46.068084 22522 leader_election.cc:290] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Requested vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.073516 22320 raft_consensus.cc:2468] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 1.
I20260709 12:17:46.075037 22434 leader_election.cc:304] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:46.075062 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:46.077944 22513 tablet_bootstrap.cc:654] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:46.078421 22516 raft_consensus.cc:2804] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.064539 22553 raft_consensus.cc:697] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:46.079753 22553 consensus_queue.cc:237] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.066280 22511 raft_consensus.cc:385] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.081687 22511 raft_consensus.cc:740] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.078076 22396 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.082760 22511 consensus_queue.cc:260] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.086299 22511 ts_tablet_manager.cc:1434] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.002s
I20260709 12:17:46.087883 22511 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:46.080760 22516 raft_consensus.cc:697] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:46.088613 22510 ts_tablet_manager.cc:1434] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.111s	user 0.016s	sys 0.012s
I20260709 12:17:46.091418 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:46.095098 22470 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.082856 22518 raft_consensus.cc:493] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.098505 22516 consensus_queue.cc:237] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:46.099036 22518 raft_consensus.cc:515] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.101204 22518 leader_election.cc:290] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.101897 22469 raft_consensus.cc:2468] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:46.105213 22510 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
W20260709 12:17:46.106956 22360 leader_election.cc:343] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:46.111541 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
I20260709 12:17:46.112370 22320 raft_consensus.cc:2468] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 0.
I20260709 12:17:46.118448 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.119287 22469 raft_consensus.cc:2468] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 0.
I20260709 12:17:46.121121 22360 leader_election.cc:304] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:46.122820 22360 leader_election.cc:304] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:46.123720 22556 raft_consensus.cc:2804] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.115295 22396 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:46.124233 22532 raft_consensus.cc:2804] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:46.124457 22556 raft_consensus.cc:697] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:46.125288 22532 raft_consensus.cc:493] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.125706 22532 raft_consensus.cc:3060] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.125943 22556 consensus_queue.cc:237] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.126669 22284 leader_election.cc:304] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:46.126103 22216 catalog_manager.cc:5697] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.128293 22522 raft_consensus.cc:2804] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.128952 22522 raft_consensus.cc:697] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Becoming Leader. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Running, Role: LEADER
I20260709 12:17:46.129833 22522 consensus_queue.cc:237] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.131793 22513 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:46.132411 22513 ts_tablet_manager.cc:1403] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.069s	user 0.023s	sys 0.018s
I20260709 12:17:46.131428 22217 catalog_manager.cc:5697] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.139662 22510 tablet_bootstrap.cc:654] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:46.141495 22513 raft_consensus.cc:359] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.142282 22511 tablet_bootstrap.cc:654] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:46.142511 22513 raft_consensus.cc:385] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.143152 22513 raft_consensus.cc:740] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.144330 22513 consensus_queue.cc:260] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.147192 22513 ts_tablet_manager.cc:1434] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 12:17:46.148340 22513 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71: Bootstrap starting.
I20260709 12:17:46.130672 22470 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:46.162706 22532 raft_consensus.cc:515] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.166913 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:46.167594 22320 raft_consensus.cc:3060] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.167996 22217 catalog_manager.cc:5697] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193). New cstate: current_term: 1 leader_uuid: "18929f381fe84136b29bfbcc3c9aad3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.176198 22320 raft_consensus.cc:2468] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:46.178679 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:46.180063 22470 raft_consensus.cc:3060] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.183425 22513 tablet_bootstrap.cc:654] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:46.188934 22510 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:46.189558 22510 ts_tablet_manager.cc:1403] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.013s
I20260709 12:17:46.179355 22532 leader_election.cc:290] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.193529 22470 raft_consensus.cc:2468] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:46.197333 22360 leader_election.cc:304] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:46.198417 22532 raft_consensus.cc:2804] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.198549 22511 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:46.199345 22511 ts_tablet_manager.cc:1403] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.112s	user 0.017s	sys 0.034s
I20260709 12:17:46.199411 22532 raft_consensus.cc:697] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:46.203176 22511 raft_consensus.cc:359] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.204241 22511 raft_consensus.cc:385] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.205464 22510 raft_consensus.cc:359] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.206879 22510 raft_consensus.cc:385] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.207139 22510 raft_consensus.cc:740] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.207489 22522 raft_consensus.cc:493] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.208236 22522 raft_consensus.cc:515] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.208230 22532 consensus_queue.cc:237] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.212590 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:46.211591 22510 consensus_queue.cc:260] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.213802 22396 raft_consensus.cc:2393] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 18929f381fe84136b29bfbcc3c9aad3f in current term 1: Already voted for candidate d2c2eb9733334ac587cd465847499c71 in this term.
I20260709 12:17:46.215425 22510 ts_tablet_manager.cc:1434] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:17:46.216603 22510 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102: Bootstrap starting.
I20260709 12:17:46.215680 22217 catalog_manager.cc:5697] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.217989 22522 leader_election.cc:290] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.217736 22217 catalog_manager.cc:5697] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.204660 22511 raft_consensus.cc:740] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.220137 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.222548 22285 leader_election.cc:304] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f; no voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71
I20260709 12:17:46.224412 22522 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.231698 22513 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71: No bootstrap required, opened a new log
I20260709 12:17:46.229722 22511 consensus_queue.cc:260] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.232416 22513 ts_tablet_manager.cc:1403] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71: Time spent bootstrapping tablet: real 0.084s	user 0.038s	sys 0.003s
I20260709 12:17:46.233596 22510 tablet_bootstrap.cc:654] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:46.235635 22513 raft_consensus.cc:359] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.236449 22513 raft_consensus.cc:385] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.236717 22513 raft_consensus.cc:740] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.236625 22522 raft_consensus.cc:2749] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:46.237401 22511 ts_tablet_manager.cc:1434] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.002s
I20260709 12:17:46.237949 22513 consensus_queue.cc:260] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.238901 22511 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:46.240026 22513 ts_tablet_manager.cc:1434] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:17:46.287175 22532 raft_consensus.cc:493] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.288515 22532 raft_consensus.cc:515] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.292855 22532 leader_election.cc:290] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.293923 22522 raft_consensus.cc:493] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.294678 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.294952 22522 raft_consensus.cc:515] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.296375 22470 raft_consensus.cc:2393] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5823e004c86c4257b8b0e25da0fb8102 in current term 1: Already voted for candidate 18929f381fe84136b29bfbcc3c9aad3f in this term.
I20260709 12:17:46.307727 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:46.307791 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
I20260709 12:17:46.306851 22522 leader_election.cc:290] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.312207 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.313548 22470 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 0.
I20260709 12:17:46.314692 22510 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102: No bootstrap required, opened a new log
I20260709 12:17:46.315945 22510 ts_tablet_manager.cc:1403] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102: Time spent bootstrapping tablet: real 0.100s	user 0.009s	sys 0.011s
I20260709 12:17:46.316483 22511 tablet_bootstrap.cc:654] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:46.318450 22285 leader_election.cc:304] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:46.310545 22396 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 0.
I20260709 12:17:46.319844 22522 raft_consensus.cc:2804] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:46.321326 22522 raft_consensus.cc:493] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.321844 22522 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.322453 22510 raft_consensus.cc:359] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:46.323166 22360 leader_election.cc:304] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102; no voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71
I20260709 12:17:46.323750 22510 raft_consensus.cc:385] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.324249 22510 raft_consensus.cc:740] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.324472 22532 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.325569 22510 consensus_queue.cc:260] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:46.330534 22510 ts_tablet_manager.cc:1434] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.007s
I20260709 12:17:46.337549 22216 catalog_manager.cc:5697] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.343670 22511 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:46.344477 22511 ts_tablet_manager.cc:1403] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.106s	user 0.025s	sys 0.009s
I20260709 12:17:46.348600 22511 raft_consensus.cc:359] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.349680 22511 raft_consensus.cc:385] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.350090 22511 raft_consensus.cc:740] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.351096 22511 consensus_queue.cc:260] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.352538 22522 raft_consensus.cc:515] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.353147 22511 ts_tablet_manager.cc:1434] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:46.355249 22511 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:46.357782 22396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:46.358686 22396 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.361758 22532 raft_consensus.cc:2749] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:46.363919 22511 tablet_bootstrap.cc:654] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:46.367614 22536 consensus_queue.cc:1048] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.369736 22536 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.370627 22516 consensus_queue.cc:1048] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.377494 22522 leader_election.cc:290] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Requested vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.379058 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:46.380072 22469 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.381315 22396 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:46.383947 22284 leader_election.cc:304] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:46.386792 22570 raft_consensus.cc:2804] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.370397 22536 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.392771 22511 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: No bootstrap required, opened a new log
I20260709 12:17:46.387591 22570 raft_consensus.cc:697] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Becoming Leader. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Running, Role: LEADER
I20260709 12:17:46.393646 22511 ts_tablet_manager.cc:1403] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.014s
I20260709 12:17:46.394472 22570 consensus_queue.cc:237] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.395980 22516 consensus_queue.cc:1048] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.397557 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:46.398396 22395 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 0.
I20260709 12:17:46.397596 22511 raft_consensus.cc:359] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.400046 22511 raft_consensus.cc:385] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:46.400066 22284 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:46.400393 22511 raft_consensus.cc:740] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:46.401373 22522 raft_consensus.cc:2804] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:46.401346 22511 consensus_queue.cc:260] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.401940 22522 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.402320 22522 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.403672 22468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.404521 22468 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 0.
I20260709 12:17:46.396773 22469 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:46.412999 22536 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.429461 22532 raft_consensus.cc:493] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.430135 22532 raft_consensus.cc:515] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:46.440183 22532 leader_election.cc:290] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033), 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209)
I20260709 12:17:46.442023 22522 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.443252 22320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f" is_pre_election: true
I20260709 12:17:46.443487 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.444434 22320 raft_consensus.cc:2393] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5823e004c86c4257b8b0e25da0fb8102 in current term 1: Already voted for candidate d2c2eb9733334ac587cd465847499c71 in this term.
I20260709 12:17:46.446264 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:46.446705 22522 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Requested vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.446965 22394 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.447896 22360 leader_election.cc:304] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102; no voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71
I20260709 12:17:46.449463 22556 raft_consensus.cc:3060] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.454222 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:46.454608 22470 raft_consensus.cc:3060] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.455084 22469 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.468060 22511 ts_tablet_manager.cc:1434] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.074s	user 0.026s	sys 0.043s
I20260709 12:17:46.495637 22217 catalog_manager.cc:5697] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193). New cstate: current_term: 1 leader_uuid: "18929f381fe84136b29bfbcc3c9aad3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.505223 22556 raft_consensus.cc:2749] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:46.505789 22570 consensus_queue.cc:1048] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.506541 22394 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:46.508013 22284 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:46.509295 22570 raft_consensus.cc:2804] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:46.509822 22570 raft_consensus.cc:697] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Becoming Leader. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Running, Role: LEADER
I20260709 12:17:46.510753 22570 consensus_queue.cc:237] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.513976 22532 consensus_queue.cc:1048] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.515231 22469 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:46.525147 22532 consensus_queue.cc:1048] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.552495 22217 catalog_manager.cc:5697] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193). New cstate: current_term: 1 leader_uuid: "18929f381fe84136b29bfbcc3c9aad3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.596261 22321 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:46.603159 22570 raft_consensus.cc:493] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.603735 22570 raft_consensus.cc:515] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.604374 22532 consensus_queue.cc:1048] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.607038 22532 consensus_queue.cc:1048] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.609032 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:46.609333 22570 leader_election.cc:290] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.616886 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:46.617714 22469 raft_consensus.cc:2468] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 1.
I20260709 12:17:46.618901 22285 leader_election.cc:304] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:46.620165 22522 raft_consensus.cc:2804] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:46.620573 22522 raft_consensus.cc:493] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5823e004c86c4257b8b0e25da0fb8102)
I20260709 12:17:46.621032 22522 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.632383 22532 consensus_queue.cc:1048] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.640606 22515 consensus_queue.cc:1048] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.641776 22522 raft_consensus.cc:515] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.643007 22320 raft_consensus.cc:1240] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Rejecting Update request from peer 5823e004c86c4257b8b0e25da0fb8102 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:46.645993 22522 leader_election.cc:290] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 2 election: Requested vote from peers 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:46.648017 22556 consensus_queue.cc:1059] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.649024 22556 raft_consensus.cc:3055] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.649421 22556 raft_consensus.cc:740] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:46.650169 22532 consensus_queue.cc:1048] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.650687 22556 consensus_queue.cc:260] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.651335 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:46.650446 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "18929f381fe84136b29bfbcc3c9aad3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:46.652201 22556 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.675019 22516 consensus_queue.cc:1048] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.688135 22395 raft_consensus.cc:2468] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18929f381fe84136b29bfbcc3c9aad3f in term 2.
I20260709 12:17:46.692764 10011 ts_manager.cc:295] Set tserver state for 18929f381fe84136b29bfbcc3c9aad3f to MAINTENANCE_MODE
I20260709 12:17:46.693892 22284 leader_election.cc:304] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: d2c2eb9733334ac587cd465847499c71
I20260709 12:17:46.695299 22522 raft_consensus.cc:2804] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.695781 22522 raft_consensus.cc:697] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 LEADER]: Becoming Leader. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Running, Role: LEADER
I20260709 12:17:46.696482 22574 consensus_queue.cc:1048] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.696673 22522 consensus_queue.cc:237] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:46.694171 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
I20260709 12:17:46.720899 22570 consensus_queue.cc:1048] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:46.744851 22219 catalog_manager.cc:5697] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f reported cstate change: term changed from 1 to 2, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193). New cstate: current_term: 2 leader_uuid: "18929f381fe84136b29bfbcc3c9aad3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.746763 22556 consensus_queue.cc:1048] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:17:46.778358 22360 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 407 similar messages]
W20260709 12:17:46.787773 22360 consensus_peers.cc:597] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:46.798076 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:46.799125 10011 tablet_replica.cc:333] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.799782 10011 raft_consensus.cc:2243] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:46.800460 10011 pending_rounds.cc:70] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Trying to abort 2 pending ops.
I20260709 12:17:46.800678 10011 pending_rounds.cc:77] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305623415600529408 op_type: NO_OP noop_request { }
I20260709 12:17:46.800997 10011 raft_consensus.cc:2889] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:46.801249 10011 pending_rounds.cc:77] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623413339779072 op_type: NO_OP noop_request { }
I20260709 12:17:46.801508 10011 raft_consensus.cc:2889] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:46.801856 10011 raft_consensus.cc:2272] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.804627 10011 tablet_replica.cc:333] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.805243 10011 raft_consensus.cc:2243] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:46.805889 10011 pending_rounds.cc:70] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Trying to abort 1 pending ops.
I20260709 12:17:46.806084 10011 pending_rounds.cc:77] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623414856871936 op_type: NO_OP noop_request { }
I20260709 12:17:46.806371 10011 raft_consensus.cc:2889] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:46.806639 10011 raft_consensus.cc:2272] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.809181 10011 tablet_replica.cc:333] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.809862 10011 raft_consensus.cc:2243] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:46.810310 10011 raft_consensus.cc:2272] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.812752 10011 tablet_replica.cc:333] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.813369 10011 raft_consensus.cc:2243] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:46.814043 10011 pending_rounds.cc:70] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Trying to abort 1 pending ops.
I20260709 12:17:46.814266 10011 pending_rounds.cc:77] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623414461861888 op_type: NO_OP noop_request { }
I20260709 12:17:46.814544 10011 raft_consensus.cc:2889] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:46.814826 10011 raft_consensus.cc:2272] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.817220 10011 tablet_replica.cc:333] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.817919 10011 raft_consensus.cc:2243] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:46.818785 10011 raft_consensus.cc:2272] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.821715 10011 tablet_replica.cc:333] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.822377 10011 raft_consensus.cc:2243] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:46.822846 10011 raft_consensus.cc:2272] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.825615 10011 tablet_replica.cc:333] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.826354 10011 raft_consensus.cc:2243] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:46.826831 10011 raft_consensus.cc:2272] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.829151 10011 tablet_replica.cc:333] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.829807 10011 raft_consensus.cc:2243] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:46.830266 10011 raft_consensus.cc:2272] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.833058 10011 tablet_replica.cc:333] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.833585 10011 raft_consensus.cc:2243] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:46.834144 10011 raft_consensus.cc:2272] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.836681 10011 tablet_replica.cc:333] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: stopping tablet replica
I20260709 12:17:46.837217 10011 raft_consensus.cc:2243] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:46.838122 10011 raft_consensus.cc:2272] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:46.875217 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
W20260709 12:17:46.926617 22434 consensus_peers.cc:597] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:46.986167 22360 consensus_peers.cc:597] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:47.127257 22434 consensus_peers.cc:597] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:47.223646 22360 consensus_peers.cc:597] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:47.645007 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:47.926234 22516 raft_consensus.cc:493] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 18929f381fe84136b29bfbcc3c9aad3f)
I20260709 12:17:47.926990 22516 raft_consensus.cc:515] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:47.929210 22516 leader_election.cc:290] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:47.930569 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
W20260709 12:17:47.933501 22434 leader_election.cc:336] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:47.933916 22434 leader_election.cc:304] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:47.934517 22516 raft_consensus.cc:2749] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:48.014066 22553 raft_consensus.cc:493] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 18929f381fe84136b29bfbcc3c9aad3f)
I20260709 12:17:48.014724 22553 raft_consensus.cc:515] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.016685 22553 leader_election.cc:290] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.018445 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:48.019164 22469 raft_consensus.cc:2468] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:48.020459 22360 leader_election.cc:304] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:48.021165 22553 raft_consensus.cc:2804] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:48.021260 22360 leader_election.cc:336] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.021581 22553 raft_consensus.cc:493] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 18929f381fe84136b29bfbcc3c9aad3f)
I20260709 12:17:48.022022 22553 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.028142 22553 raft_consensus.cc:515] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.030019 22553 leader_election.cc:290] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.031747 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:48.032429 22469 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:48.033655 22360 leader_election.cc:336] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.034791 22556 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.035317 22556 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.037395 22556 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.039445 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:48.039980 22469 raft_consensus.cc:2468] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:48.040169 22470 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:48.041208 22360 leader_election.cc:304] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:48.041953 22556 raft_consensus.cc:2804] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:48.042126 22360 leader_election.cc:336] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.042488 22556 raft_consensus.cc:697] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:48.043099 22360 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:48.043795 22553 raft_consensus.cc:2804] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:48.043443 22556 consensus_queue.cc:237] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.044241 22553 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.044575 22553 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.051910 22553 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.051661 22218 catalog_manager.cc:5697] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 1 to 2, leader changed from 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:48.054152 22553 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.054891 22553 raft_consensus.cc:493] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.055328 22553 raft_consensus.cc:515] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.057188 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:48.057878 22470 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.058303 22553 leader_election.cc:290] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
W20260709 12:17:48.059298 22360 leader_election.cc:336] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.061683 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:48.062448 22469 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
W20260709 12:17:48.063393 22360 leader_election.cc:336] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.064473 22360 leader_election.cc:304] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:48.065259 22553 raft_consensus.cc:2804] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:48.065701 22553 raft_consensus.cc:493] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.066023 22553 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.067862 22470 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:48.069928 22360 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:48.070847 22556 raft_consensus.cc:2804] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:48.071352 22556 raft_consensus.cc:697] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:48.072482 22556 consensus_queue.cc:237] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.073024 22553 raft_consensus.cc:515] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.075438 22553 leader_election.cc:290] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.078167 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:48.078883 22470 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:48.080870 22360 leader_election.cc:336] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.081950 22218 catalog_manager.cc:5697] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 1 to 2, leader changed from 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:48.085088 22470 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:48.086371 22360 leader_election.cc:304] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:48.087258 22556 raft_consensus.cc:2804] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:48.087750 22556 raft_consensus.cc:697] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:48.088593 22556 consensus_queue.cc:237] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.097011 22218 catalog_manager.cc:5697] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 1 to 2, leader changed from 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:48.138389 22516 raft_consensus.cc:493] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5823e004c86c4257b8b0e25da0fb8102)
I20260709 12:17:48.138870 22516 raft_consensus.cc:515] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.140589 22516 leader_election.cc:290] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:48.141847 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:48.142683 22394 raft_consensus.cc:2393] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d2c2eb9733334ac587cd465847499c71 in current term 2: Already voted for candidate 18929f381fe84136b29bfbcc3c9aad3f in this term.
W20260709 12:17:48.144487 22434 leader_election.cc:336] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.144848 22434 leader_election.cc:304] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:48.145449 22516 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.151367 22516 raft_consensus.cc:2749] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:48.220889 22556 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 18929f381fe84136b29bfbcc3c9aad3f)
I20260709 12:17:48.221369 22556 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.223294 22556 leader_election.cc:290] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.224933 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:48.225708 22516 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 18929f381fe84136b29bfbcc3c9aad3f)
I20260709 12:17:48.226302 22516 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
W20260709 12:17:48.227274 22360 leader_election.cc:336] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.227380 22470 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 1.
I20260709 12:17:48.228562 22516 leader_election.cc:290] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:48.229710 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102" is_pre_election: true
I20260709 12:17:48.230014 22360 leader_election.cc:304] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:48.230445 22394 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 1.
I20260709 12:17:48.231407 22556 raft_consensus.cc:2804] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:48.231832 22556 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 18929f381fe84136b29bfbcc3c9aad3f)
I20260709 12:17:48.231969 22433 leader_election.cc:304] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:48.232352 22556 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.232908 22516 raft_consensus.cc:2804] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:48.232940 22434 leader_election.cc:336] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.233546 22516 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 18929f381fe84136b29bfbcc3c9aad3f)
I20260709 12:17:48.234032 22516 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:48.239706 22556 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.240358 22516 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.241566 22556 leader_election.cc:290] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.242318 22516 leader_election.cc:290] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:48.243844 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:48.243851 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:48.244740 22470 raft_consensus.cc:2393] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5823e004c86c4257b8b0e25da0fb8102 in current term 2: Already voted for candidate d2c2eb9733334ac587cd465847499c71 in this term.
I20260709 12:17:48.244845 22394 raft_consensus.cc:2393] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d2c2eb9733334ac587cd465847499c71 in current term 2: Already voted for candidate 5823e004c86c4257b8b0e25da0fb8102 in this term.
W20260709 12:17:48.246577 22434 leader_election.cc:336] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
W20260709 12:17:48.246647 22360 leader_election.cc:336] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.247000 22434 leader_election.cc:304] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:48.247162 22360 leader_election.cc:304] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102; no voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71
I20260709 12:17:48.247756 22516 raft_consensus.cc:2749] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:48.247931 22556 raft_consensus.cc:2749] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:48.425722 22470 raft_consensus.cc:1275] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:48.427189 22556 consensus_queue.cc:1048] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:48.442286 22360 consensus_peers.cc:597] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:48.571982 22470 raft_consensus.cc:1275] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:48.573410 22553 consensus_queue.cc:1048] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:48.588495 22360 consensus_peers.cc:597] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:48.646286 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:48.704088 22469 raft_consensus.cc:1275] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:48.705387 22611 consensus_queue.cc:1048] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:48.713934 22360 consensus_peers.cc:597] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:48.891390 22532 raft_consensus.cc:493] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.891871 22532 raft_consensus.cc:515] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.893679 22532 leader_election.cc:290] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.895524 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:48.896282 22469 raft_consensus.cc:2468] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:48.897680 22360 leader_election.cc:304] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:48.898533 22532 raft_consensus.cc:2804] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:48.898628 22360 leader_election.cc:336] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.898970 22532 raft_consensus.cc:493] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.899367 22532 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:48.905436 22532 raft_consensus.cc:515] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.907353 22532 leader_election.cc:290] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:48.908942 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fc7aca87fcd41549f8f2e90a42cb925" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:48.909600 22469 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:48.910820 22360 leader_election.cc:336] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:48.915055 22469 raft_consensus.cc:2468] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 3.
I20260709 12:17:48.916198 22360 leader_election.cc:304] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:48.916922 22532 raft_consensus.cc:2804] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:48.917387 22532 raft_consensus.cc:697] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:48.918186 22532 consensus_queue.cc:237] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:48.925679 22218 catalog_manager.cc:5697] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 2 to 3, leader changed from 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:49.140739 22434 consensus_peers.cc:597] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:49.358318 22360 consensus_peers.cc:597] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:49.360325 22360 consensus_peers.cc:597] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:49.382458 22470 raft_consensus.cc:1275] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:49.383780 22553 consensus_queue.cc:1048] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:49.525985 22360 consensus_peers.cc:597] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:49.647501 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:49.787403 22360 consensus_peers.cc:597] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:49.799479 22532 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.799949 22532 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:49.801702 22532 leader_election.cc:290] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:49.803112 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71" is_pre_election: true
I20260709 12:17:49.803848 22470 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:49.804920 22360 leader_election.cc:304] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:49.805693 22532 raft_consensus.cc:2804] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:49.805728 22360 leader_election.cc:336] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:49.806049 22532 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.806443 22532 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.811746 22532 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:49.813445 22532 leader_election.cc:290] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:49.814905 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:49.815655 22470 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:49.817073 22360 leader_election.cc:336] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111)
I20260709 12:17:49.821131 22470 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 3.
I20260709 12:17:49.822372 22360 leader_election.cc:304] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:49.823089 22532 raft_consensus.cc:2804] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:49.823519 22532 raft_consensus.cc:697] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:49.824318 22532 consensus_queue.cc:237] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:49.831862 22219 catalog_manager.cc:5697] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 1 to 3, leader changed from 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:49.896237 22434 consensus_peers.cc:597] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:50.256964 22360 consensus_peers.cc:597] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:50.446646 22470 raft_consensus.cc:1275] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:50.447924 22532 consensus_queue.cc:1048] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:50.648672 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:51.649832 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:51.782431 22360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111) [suppressed 76 similar messages]
W20260709 12:17:51.786338 22360 consensus_peers.cc:597] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:51.786772 22360 consensus_peers.cc:597] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:51.787055 22360 consensus_peers.cc:597] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:51.901021 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:51.909173 22632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:51.912443 22633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:51.915496 10011 server_base.cc:1061] running on GCE node
W20260709 12:17:51.915650 22635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:51.916980 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:51.917207 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:51.917455 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599471917435 us; error 0 us; skew 500 ppm
I20260709 12:17:51.918093 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:51.921126 10011 webserver.cc:533] Webserver started at http://127.9.198.193:43013/ using document root <none> and password file <none>
I20260709 12:17:51.921804 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:51.922029 10011 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:17:51.926287 10011 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:17:51.931071 22640 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:51.931972 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:17:51.932318 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "18929f381fe84136b29bfbcc3c9aad3f"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-glnm"
I20260709 12:17:51.932715 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:51.950600 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:51.951866 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:51.953439 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:51.957449 22647 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:17:52.014834 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:17:52.015170 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.059s	user 0.003s	sys 0.000s
I20260709 12:17:52.015691 10011 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:17:52.022003 22647 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.036580 22647 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:52.037580 22647 tablet_bootstrap.cc:492] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.038401 22647 ts_tablet_manager.cc:1403] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:17:52.041244 22647 raft_consensus.cc:359] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.041999 22647 raft_consensus.cc:740] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.042716 22647 consensus_queue.cc:260] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.044974 22647 ts_tablet_manager.cc:1434] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:52.045750 22647 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.060417 22647 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:52.061309 22647 tablet_bootstrap.cc:492] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.061887 10011 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:17:52.062011 22647 ts_tablet_manager.cc:1403] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.004s
I20260709 12:17:52.062264 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.047s	user 0.047s	sys 0.000s
I20260709 12:17:52.064589 22647 raft_consensus.cc:359] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.065179 22647 raft_consensus.cc:740] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.066085 22647 consensus_queue.cc:260] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.068054 22647 ts_tablet_manager.cc:1434] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:52.069056 22647 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.084983 22647 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:52.086005 22647 tablet_bootstrap.cc:492] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.086781 22647 ts_tablet_manager.cc:1403] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:17:52.089083 22647 raft_consensus.cc:359] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.089748 22647 raft_consensus.cc:740] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.090534 22647 consensus_queue.cc:260] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.092988 22647 ts_tablet_manager.cc:1434] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:52.094009 22647 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
W20260709 12:17:52.096375 22434 consensus_peers.cc:597] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:17:52.115896 22647 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:52.116884 22647 tablet_bootstrap.cc:492] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.117600 22647 ts_tablet_manager.cc:1403] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
I20260709 12:17:52.119913 22647 raft_consensus.cc:359] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.120976 22647 raft_consensus.cc:740] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.121941 22647 consensus_queue.cc:260] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.124596 22647 ts_tablet_manager.cc:1434] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:17:52.125540 22647 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.134019 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:42209
I20260709 12:17:52.134230 22708 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:42209 every 8 connection(s)
I20260709 12:17:52.139756 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:52.141693 22647 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:52.142953 22647 tablet_bootstrap.cc:492] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.143705 22647 ts_tablet_manager.cc:1403] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 12:17:52.146502 22647 raft_consensus.cc:359] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.147207 22647 raft_consensus.cc:740] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.148138 22647 consensus_queue.cc:260] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.150310 22647 ts_tablet_manager.cc:1434] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:17:52.151150 22647 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.162068 22710 heartbeater.cc:344] Connected to a master server at 127.9.198.254:44497
I20260709 12:17:52.162616 22710 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:52.163609 22710 heartbeater.cc:507] Master 127.9.198.254:44497 requested a full tablet report, sending...
I20260709 12:17:52.170642 22647 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:17:52.170805 22219 ts_manager.cc:194] Re-registered known tserver with Master: 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209)
I20260709 12:17:52.170919 22394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "050a9774d2d84e8686f55f7198286712"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:52.171854 22394 raft_consensus.cc:606] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:52.171945 22647 tablet_bootstrap.cc:492] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.172863 22647 ts_tablet_manager.cc:1403] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.002s
I20260709 12:17:52.175243 22394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "899030d87cdd4814a01f394782e836b6"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:52.175989 22394 raft_consensus.cc:606] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:52.176041 22647 raft_consensus.cc:359] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.177186 22647 raft_consensus.cc:740] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.178179 22647 consensus_queue.cc:260] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.179766 22394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b939523530634117855680eb8e3713ea"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:52.180537 22394 raft_consensus.cc:606] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:52.182297 22647 ts_tablet_manager.cc:1434] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:17:52.183719 22647 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.184561 22395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:52.185254 22395 raft_consensus.cc:606] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:52.186985 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:17:52.188141 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:52.189379 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:52.193470 22717 raft_consensus.cc:993] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.195336 22532 raft_consensus.cc:1081] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.196314 22219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44928
I20260709 12:17:52.197395 22470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "050a9774d2d84e8686f55f7198286712"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:52.198084 22470 raft_consensus.cc:493] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.198452 22470 raft_consensus.cc:3060] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.208169 22710 heartbeater.cc:499] Master 127.9.198.254:44497 was elected leader, sending a full tablet report...
I20260709 12:17:52.211354 22470 raft_consensus.cc:515] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.218807 22683 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.221189 22470 leader_election.cc:290] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:52.223400 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:52.224164 22395 raft_consensus.cc:3055] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:52.224531 22395 raft_consensus.cc:740] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:52.225435 22395 consensus_queue.cc:260] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.226827 22395 raft_consensus.cc:3060] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.240010 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050a9774d2d84e8686f55f7198286712" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:52.241173 22395 raft_consensus.cc:2468] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 2.
W20260709 12:17:52.242000 22434 leader_election.cc:343] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:52.243544 22433 leader_election.cc:304] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5823e004c86c4257b8b0e25da0fb8102, d2c2eb9733334ac587cd465847499c71; no voters: 18929f381fe84136b29bfbcc3c9aad3f
I20260709 12:17:52.244690 22629 raft_consensus.cc:2804] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:52.246030 22647 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:52.247491 22629 raft_consensus.cc:697] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 2 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:52.247511 22647 tablet_bootstrap.cc:492] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.248399 22629 consensus_queue.cc:237] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.248857 22647 ts_tablet_manager.cc:1403] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.065s	user 0.030s	sys 0.016s
I20260709 12:17:52.251935 22647 raft_consensus.cc:359] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:52.252712 22647 raft_consensus.cc:740] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.253926 22647 consensus_queue.cc:260] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:52.256201 22647 ts_tablet_manager.cc:1434] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:52.257057 22647 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.258780 22218 catalog_manager.cc:5697] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 1 to 2, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:52.269922 22647 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:52.270663 22647 tablet_bootstrap.cc:492] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.271247 22647 ts_tablet_manager.cc:1403] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:17:52.273332 22647 raft_consensus.cc:359] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.273950 22647 raft_consensus.cc:740] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.274588 22647 consensus_queue.cc:260] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.276325 22647 ts_tablet_manager.cc:1434] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:52.277508 22647 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.291376 22647 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260709 12:17:52.292145 22647 tablet_bootstrap.cc:492] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.292687 22647 ts_tablet_manager.cc:1403] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 12:17:52.294528 22647 raft_consensus.cc:359] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.295150 22647 raft_consensus.cc:740] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.295773 22647 consensus_queue.cc:260] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.297595 22647 ts_tablet_manager.cc:1434] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:52.298496 22647 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap starting.
I20260709 12:17:52.312178 22683 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.312927 22647 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:17:52.313803 22647 tablet_bootstrap.cc:492] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Bootstrap complete.
I20260709 12:17:52.314352 22647 ts_tablet_manager.cc:1403] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 12:17:52.316311 22647 raft_consensus.cc:359] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.316931 22647 raft_consensus.cc:740] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18929f381fe84136b29bfbcc3c9aad3f, State: Initialized, Role: FOLLOWER
I20260709 12:17:52.317516 22647 consensus_queue.cc:260] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.317865 22683 pending_rounds.cc:85] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting all ops after (but not including) 0
I20260709 12:17:52.318290 22683 pending_rounds.cc:107] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:17:52.318583 22683 raft_consensus.cc:2889] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.319502 22647 ts_tablet_manager.cc:1434] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:52.448854 22717 raft_consensus.cc:993] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.449365 22532 raft_consensus.cc:1081] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.451148 22470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b939523530634117855680eb8e3713ea"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:52.451638 22470 raft_consensus.cc:493] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.451879 22470 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.456923 22470 raft_consensus.cc:515] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.459466 22470 leader_election.cc:290] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 4 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:52.459775 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:52.460033 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b939523530634117855680eb8e3713ea" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:52.460522 22682 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.460884 22395 raft_consensus.cc:3055] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:52.461270 22395 raft_consensus.cc:740] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:52.462260 22395 consensus_queue.cc:260] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.463605 22395 raft_consensus.cc:3060] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.468236 22682 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 4.
I20260709 12:17:52.469667 22434 leader_election.cc:304] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:52.470562 22629 raft_consensus.cc:2804] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:52.471076 22629 raft_consensus.cc:697] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 4 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:52.471378 22395 raft_consensus.cc:2468] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 4.
I20260709 12:17:52.471947 22629 consensus_queue.cc:237] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.478948 22218 catalog_manager.cc:5697] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 3 to 4, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 4 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:52.485591 22682 raft_consensus.cc:3060] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.491513 22682 pending_rounds.cc:85] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting all ops after (but not including) 1
I20260709 12:17:52.491787 22682 pending_rounds.cc:107] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:52.492033 22682 raft_consensus.cc:2889] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.518399 22717 raft_consensus.cc:993] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.518851 22532 raft_consensus.cc:1081] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.520393 22470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:52.520879 22470 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.521160 22470 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.526120 22470 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.527618 22470 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:52.528666 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:52.528672 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:52.529359 22682 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.529500 22395 raft_consensus.cc:3055] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:52.530030 22395 raft_consensus.cc:740] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:52.530838 22395 consensus_queue.cc:260] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.532073 22683 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.532109 22395 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.536142 22682 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:52.537457 22434 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:52.538309 22629 raft_consensus.cc:2804] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:52.538816 22629 raft_consensus.cc:697] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:52.539637 22629 consensus_queue.cc:237] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.539908 22683 pending_rounds.cc:85] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting all ops after (but not including) 0
I20260709 12:17:52.539839 22395 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:52.540480 22683 pending_rounds.cc:107] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:17:52.540786 22683 raft_consensus.cc:2889] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.548213 22218 catalog_manager.cc:5697] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 2 to 3, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:52.601594 22683 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:52.650887 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:52.683988 22717 raft_consensus.cc:993] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.684525 22532 raft_consensus.cc:1081] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:52.686110 22470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "899030d87cdd4814a01f394782e836b6"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:52.686638 22470 raft_consensus.cc:493] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:52.686952 22470 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.692097 22470 raft_consensus.cc:515] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.693751 22470 leader_election.cc:290] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:52.694810 22683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:52.694975 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899030d87cdd4814a01f394782e836b6" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:52.695503 22683 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.695644 22395 raft_consensus.cc:3055] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:52.696069 22395 raft_consensus.cc:740] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:52.696975 22395 consensus_queue.cc:260] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.698616 22395 raft_consensus.cc:3060] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.698844 22682 raft_consensus.cc:3060] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:52.703650 22683 raft_consensus.cc:2468] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:52.704909 22434 leader_election.cc:304] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:52.705821 22722 raft_consensus.cc:2804] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:52.706338 22722 raft_consensus.cc:697] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:52.707093 22722 consensus_queue.cc:237] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:52.708009 22395 raft_consensus.cc:2468] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:52.708954 22682 raft_consensus.cc:1275] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:52.710850 22722 consensus_queue.cc:1048] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:52.715617 22218 catalog_manager.cc:5697] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 2 to 3, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:52.728960 22395 raft_consensus.cc:1275] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:52.730959 22629 consensus_queue.cc:1048] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:52.998090 22395 raft_consensus.cc:1275] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:52.999483 22629 consensus_queue.cc:1048] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:53.013207 22682 raft_consensus.cc:1275] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:53.013827 22395 raft_consensus.cc:1275] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:53.015439 22722 consensus_queue.cc:1048] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:53.015654 22629 consensus_queue.cc:1048] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:53.029156 22682 raft_consensus.cc:1275] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:53.031020 22629 consensus_queue.cc:1048] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:53.089195 22682 raft_consensus.cc:1275] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:53.090803 22722 consensus_queue.cc:1048] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:53.099830 22395 raft_consensus.cc:1275] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:53.101868 22629 consensus_queue.cc:1048] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:53.206058 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:53.223853 22470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
mode: GRACEFUL
new_leader_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:47238
I20260709 12:17:53.224339 22470 raft_consensus.cc:606] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Received request to transfer leadership to TS 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:53.226763 22470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
mode: GRACEFUL
new_leader_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:47238
I20260709 12:17:53.227155 22470 raft_consensus.cc:606] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Received request to transfer leadership to TS 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:53.228202 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:53.228811 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:53.229475 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:53.324925 22629 raft_consensus.cc:993] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71: : Instructing follower 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:53.325460 22743 raft_consensus.cc:1081] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Signalling peer 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:53.327067 22395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:56840
I20260709 12:17:53.327593 22395 raft_consensus.cc:493] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:53.327917 22395 raft_consensus.cc:3060] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:53.333323 22395 raft_consensus.cc:515] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:53.335304 22395 leader_election.cc:290] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Requested vote from peers d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033), 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209)
I20260709 12:17:53.336054 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:53.336354 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf4075e9de7430bbbf1d3d362a5b585" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:53.336763 22470 raft_consensus.cc:3055] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:53.336966 22682 raft_consensus.cc:3060] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:53.337095 22470 raft_consensus.cc:740] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:53.337924 22470 consensus_queue.cc:260] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:53.338865 22470 raft_consensus.cc:3060] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:53.343005 22682 raft_consensus.cc:2468] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:53.344040 22470 raft_consensus.cc:2468] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:53.344208 22360 leader_election.cc:304] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:53.345081 22742 raft_consensus.cc:2804] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:53.345489 22742 raft_consensus.cc:697] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:53.346203 22742 consensus_queue.cc:237] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } }
I20260709 12:17:53.353811 22219 catalog_manager.cc:5697] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 1 to 2, leader changed from d2c2eb9733334ac587cd465847499c71 (127.9.198.195) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:53.558648 22629 raft_consensus.cc:993] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: : Instructing follower 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:53.559240 22743 raft_consensus.cc:1081] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Signalling peer 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:53.560998 22395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:56840
I20260709 12:17:53.561512 22395 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:53.561908 22395 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.567341 22395 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:53.569185 22395 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 4 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:53.570212 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:53.570372 22470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:53.570984 22682 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.571111 22470 raft_consensus.cc:3055] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:53.571585 22470 raft_consensus.cc:740] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:53.572283 22470 consensus_queue.cc:260] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:53.573211 22470 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.578145 22682 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 4.
I20260709 12:17:53.578454 22470 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 4.
I20260709 12:17:53.579463 22360 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:53.580282 22742 raft_consensus.cc:2804] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:53.580744 22742 raft_consensus.cc:697] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:53.581543 22742 consensus_queue.cc:237] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:53.588354 22219 catalog_manager.cc:5697] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 3 to 4, leader changed from d2c2eb9733334ac587cd465847499c71 (127.9.198.195) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 4 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:53.651981 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:53.770385 22682 raft_consensus.cc:1275] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:53.771193 22470 raft_consensus.cc:1275] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:53.771941 22742 consensus_queue.cc:1048] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:53.772771 22532 consensus_queue.cc:1048] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:54.013159 22682 raft_consensus.cc:1275] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:54.014644 22532 consensus_queue.cc:1048] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.023240 22470 raft_consensus.cc:1275] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:54.025552 22760 consensus_queue.cc:1048] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.237700 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:54.245101 22395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:54.245784 22395 raft_consensus.cc:606] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:54.248463 22395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:54.248998 22395 raft_consensus.cc:606] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:54.250144 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:54.250660 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:54.251231 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:54.492626 22532 raft_consensus.cc:993] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:54.493180 22760 raft_consensus.cc:1081] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:54.494983 22470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:54.495435 22470 raft_consensus.cc:493] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:54.495671 22470 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:54.500488 22470 raft_consensus.cc:515] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:54.502027 22470 leader_election.cc:290] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:54.502959 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:54.503023 22395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:54.503872 22682 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:54.503966 22395 raft_consensus.cc:3055] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:54.504480 22395 raft_consensus.cc:740] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:54.505199 22395 consensus_queue.cc:260] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:54.506291 22395 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:54.510120 22682 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:54.511569 22434 leader_election.cc:304] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:54.512554 22743 raft_consensus.cc:2804] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:54.513130 22395 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:54.513783 22743 raft_consensus.cc:697] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:54.514928 22743 consensus_queue.cc:237] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:54.522789 22218 catalog_manager.cc:5697] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 2 to 3, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:54.653138 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:54.771385 22532 raft_consensus.cc:993] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:54.771984 22768 raft_consensus.cc:1081] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:54.773749 22470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:54.774272 22470 raft_consensus.cc:493] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:54.774570 22470 raft_consensus.cc:3060] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:54.779371 22470 raft_consensus.cc:515] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:54.780855 22470 leader_election.cc:290] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:54.781834 22683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:54.781909 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50f9c35d2acf470c9b81f6d5e71b16e5" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:54.782637 22683 raft_consensus.cc:3060] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:54.782709 22394 raft_consensus.cc:3055] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:54.783150 22394 raft_consensus.cc:740] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:54.783942 22394 consensus_queue.cc:260] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:54.784956 22394 raft_consensus.cc:3060] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:54.788304 22683 raft_consensus.cc:2468] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 2.
I20260709 12:17:54.789517 22434 leader_election.cc:304] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:54.790360 22763 raft_consensus.cc:2804] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:54.790838 22763 raft_consensus.cc:697] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 2 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:54.791469 22394 raft_consensus.cc:2468] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 2.
I20260709 12:17:54.791807 22763 consensus_queue.cc:237] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:54.800180 22218 catalog_manager.cc:5697] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 1 to 2, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:55.044870 22683 raft_consensus.cc:1275] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:55.046262 22763 consensus_queue.cc:1048] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:55.058302 22394 raft_consensus.cc:1275] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:55.060168 22763 consensus_queue.cc:1048] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:55.258271 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:55.266294 22469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
mode: GRACEFUL
new_leader_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:47238
I20260709 12:17:55.266866 22469 raft_consensus.cc:606] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Received request to transfer leadership to TS 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:55.270057 22469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
mode: GRACEFUL
new_leader_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:47238
I20260709 12:17:55.270694 22469 raft_consensus.cc:606] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Received request to transfer leadership to TS 5823e004c86c4257b8b0e25da0fb8102
I20260709 12:17:55.271878 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:55.272365 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:55.272892 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:55.334604 22394 raft_consensus.cc:1275] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:55.335947 22773 consensus_queue.cc:1048] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:55.344688 22682 raft_consensus.cc:1275] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:55.346496 22743 consensus_queue.cc:1048] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:55.453420 22773 raft_consensus.cc:993] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71: : Instructing follower 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:55.454566 22743 raft_consensus.cc:1081] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Signalling peer 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:55.456283 22394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:56840
I20260709 12:17:55.456982 22394 raft_consensus.cc:493] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:55.457382 22394 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.465271 22394 raft_consensus.cc:515] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:55.467180 22394 leader_election.cc:290] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 4 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:55.467965 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:55.468216 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd63024784244f99b9dd7cb8d9fd67c9" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:55.468734 22682 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.468950 22469 raft_consensus.cc:3055] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:55.469422 22469 raft_consensus.cc:740] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:55.470295 22469 consensus_queue.cc:260] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:55.471477 22469 raft_consensus.cc:3060] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.475317 22682 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 4.
I20260709 12:17:55.476567 22360 leader_election.cc:304] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:55.477311 22768 raft_consensus.cc:2804] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:55.477926 22768 raft_consensus.cc:697] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:55.478379 22469 raft_consensus.cc:2468] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 4.
I20260709 12:17:55.478947 22768 consensus_queue.cc:237] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:55.487795 22218 catalog_manager.cc:5697] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 3 to 4, leader changed from d2c2eb9733334ac587cd465847499c71 (127.9.198.195) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 4 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:55.654214 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:55.676337 22773 raft_consensus.cc:993] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71: : Instructing follower 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:55.677032 22743 raft_consensus.cc:1081] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Signalling peer 5823e004c86c4257b8b0e25da0fb8102 to start an election
I20260709 12:17:55.678774 22394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
 from {username='slave'} at 127.0.0.1:56840
I20260709 12:17:55.679342 22394 raft_consensus.cc:493] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:55.679675 22394 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.685016 22394 raft_consensus.cc:515] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:55.686975 22394 leader_election.cc:290] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), d2c2eb9733334ac587cd465847499c71 (127.9.198.195:34033)
I20260709 12:17:55.687801 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:55.688089 22469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "5823e004c86c4257b8b0e25da0fb8102" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d2c2eb9733334ac587cd465847499c71"
I20260709 12:17:55.688544 22682 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.688759 22469 raft_consensus.cc:3055] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:55.689189 22469 raft_consensus.cc:740] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:55.690021 22469 consensus_queue.cc:260] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:55.691064 22469 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.695149 22682 raft_consensus.cc:2468] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:55.696406 22360 leader_election.cc:304] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, 5823e004c86c4257b8b0e25da0fb8102; no voters: 
I20260709 12:17:55.696894 22469 raft_consensus.cc:2468] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5823e004c86c4257b8b0e25da0fb8102 in term 2.
I20260709 12:17:55.697078 22768 raft_consensus.cc:2804] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:55.697652 22768 raft_consensus.cc:697] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Leader. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:55.698509 22768 consensus_queue.cc:237] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:55.705379 22218 catalog_manager.cc:5697] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 reported cstate change: term changed from 1 to 2, leader changed from d2c2eb9733334ac587cd465847499c71 (127.9.198.195) to 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "5823e004c86c4257b8b0e25da0fb8102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:55.898947 22469 raft_consensus.cc:1275] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:55.900215 22768 consensus_queue.cc:1048] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:55.908381 22682 raft_consensus.cc:1275] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:55.909979 22784 consensus_queue.cc:1048] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:56.129307 22469 raft_consensus.cc:1275] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:56.130928 22784 consensus_queue.cc:1048] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:56.140403 22682 raft_consensus.cc:1275] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Refusing update from remote peer 5823e004c86c4257b8b0e25da0fb8102: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:56.141932 22784 consensus_queue.cc:1048] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:56.279575 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:56.287514 22394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:56.288089 22394 raft_consensus.cc:606] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:56.290858 22394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17"
dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
mode: GRACEFUL
new_leader_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:41542
I20260709 12:17:56.291412 22394 raft_consensus.cc:606] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 LEADER]: Received request to transfer leadership to TS d2c2eb9733334ac587cd465847499c71
I20260709 12:17:56.292579 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:56.293143 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:56.293795 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:56.473603 22768 raft_consensus.cc:993] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:56.474838 22784 raft_consensus.cc:1081] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:56.477299 22469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:56.478277 22469 raft_consensus.cc:493] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:56.478757 22469 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.488171 22469 raft_consensus.cc:515] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:56.490316 22469 leader_election.cc:290] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 5 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:56.491843 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:56.491761 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41a3923298149d5bb746f9af9916a17" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:56.492966 22394 raft_consensus.cc:3055] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:56.492969 22682 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.493316 22394 raft_consensus.cc:740] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:56.494562 22394 consensus_queue.cc:260] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:56.496176 22394 raft_consensus.cc:3060] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.500459 22682 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 5.
I20260709 12:17:56.502082 22434 leader_election.cc:304] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:56.503114 22743 raft_consensus.cc:2804] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.504120 22394 raft_consensus.cc:2468] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 5.
I20260709 12:17:56.506212 22743 raft_consensus.cc:697] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 5 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:56.508028 22743 consensus_queue.cc:237] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:56.516453 22219 catalog_manager.cc:5697] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 4 to 5, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 5 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:56.655426 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:56.684062 22768 raft_consensus.cc:993] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102: : Instructing follower d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:56.684690 22784 raft_consensus.cc:1081] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Signalling peer d2c2eb9733334ac587cd465847499c71 to start an election
I20260709 12:17:56.686247 22469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de"
dest_uuid: "d2c2eb9733334ac587cd465847499c71"
 from {username='slave'} at 127.0.0.1:45988
I20260709 12:17:56.686826 22469 raft_consensus.cc:493] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:56.687203 22469 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.695591 22469 raft_consensus.cc:515] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:56.697717 22469 leader_election.cc:290] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Requested vote from peers 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209), 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705)
I20260709 12:17:56.698447 22682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "18929f381fe84136b29bfbcc3c9aad3f"
I20260709 12:17:56.698853 22394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a0dae4b76494f108367efbd4676b0de" candidate_uuid: "d2c2eb9733334ac587cd465847499c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5823e004c86c4257b8b0e25da0fb8102"
I20260709 12:17:56.699229 22682 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.699579 22394 raft_consensus.cc:3055] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:56.699950 22394 raft_consensus.cc:740] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5823e004c86c4257b8b0e25da0fb8102, State: Running, Role: LEADER
I20260709 12:17:56.700997 22394 consensus_queue.cc:260] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:56.702549 22394 raft_consensus.cc:3060] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.704970 22682 raft_consensus.cc:2468] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:56.706390 22434 leader_election.cc:304] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18929f381fe84136b29bfbcc3c9aad3f, d2c2eb9733334ac587cd465847499c71; no voters: 
I20260709 12:17:56.707553 22792 raft_consensus.cc:2804] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:56.708062 22792 raft_consensus.cc:697] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Becoming Leader. State: Replica: d2c2eb9733334ac587cd465847499c71, State: Running, Role: LEADER
I20260709 12:17:56.709523 22394 raft_consensus.cc:2468] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2c2eb9733334ac587cd465847499c71 in term 3.
I20260709 12:17:56.709287 22792 consensus_queue.cc:237] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } }
I20260709 12:17:56.716682 22219 catalog_manager.cc:5697] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 reported cstate change: term changed from 2 to 3, leader changed from 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194) to d2c2eb9733334ac587cd465847499c71 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "d2c2eb9733334ac587cd465847499c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2c2eb9733334ac587cd465847499c71" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 34033 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:57.026747 22682 raft_consensus.cc:1275] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 5 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:57.028088 22743 consensus_queue.cc:1048] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:57.046697 22394 raft_consensus.cc:1275] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 5 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:57.049153 22800 consensus_queue.cc:1048] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:57.141741 22683 raft_consensus.cc:1275] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:57.143074 22800 consensus_queue.cc:1048] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18929f381fe84136b29bfbcc3c9aad3f" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 42209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.158679 22394 raft_consensus.cc:1275] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Refusing update from remote peer d2c2eb9733334ac587cd465847499c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:57.160454 22792 consensus_queue.cc:1048] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5823e004c86c4257b8b0e25da0fb8102" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.301222 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:57.328918 10011 tablet_server.cc:179] TabletServer@127.9.198.193:42209 shutting down...
W20260709 12:17:57.343125 22434 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 14 similar messages]
W20260709 12:17:57.348944 22434 consensus_peers.cc:597] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.361435 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:57.362361 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.363116 10011 raft_consensus.cc:2243] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.363642 10011 raft_consensus.cc:2272] T 0fc7aca87fcd41549f8f2e90a42cb925 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.366966 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.367646 10011 raft_consensus.cc:2243] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.368165 10011 raft_consensus.cc:2272] T 050a9774d2d84e8686f55f7198286712 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.371480 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.372504 10011 raft_consensus.cc:2243] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.373185 10011 raft_consensus.cc:2272] T e41a3923298149d5bb746f9af9916a17 P 18929f381fe84136b29bfbcc3c9aad3f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.377581 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.378455 10011 raft_consensus.cc:2243] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.378871 10011 raft_consensus.cc:2272] T f12134c8100e45b8b2321b8c6b1e342e P 18929f381fe84136b29bfbcc3c9aad3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.381388 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.382001 10011 raft_consensus.cc:2243] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.382527 10011 raft_consensus.cc:2272] T dd63024784244f99b9dd7cb8d9fd67c9 P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.384804 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.385383 10011 raft_consensus.cc:2243] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.385919 10011 raft_consensus.cc:2272] T 899030d87cdd4814a01f394782e836b6 P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.388408 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.388927 10011 raft_consensus.cc:2243] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.389377 10011 raft_consensus.cc:2272] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.391759 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.392223 10011 raft_consensus.cc:2243] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.392639 10011 raft_consensus.cc:2272] T 5a0dae4b76494f108367efbd4676b0de P 18929f381fe84136b29bfbcc3c9aad3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.396394 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.397104 10011 raft_consensus.cc:2243] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.397825 10011 raft_consensus.cc:2272] T 6bf4075e9de7430bbbf1d3d362a5b585 P 18929f381fe84136b29bfbcc3c9aad3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.400714 10011 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:57.401175 10011 raft_consensus.cc:2243] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.401649 10011 raft_consensus.cc:2272] T b939523530634117855680eb8e3713ea P 18929f381fe84136b29bfbcc3c9aad3f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.437777 10011 tablet_server.cc:196] TabletServer@127.9.198.193:42209 shutdown complete.
I20260709 12:17:57.469341 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
I20260709 12:17:57.496258 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:57.497090 10011 tablet_replica.cc:333] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.498111 10011 raft_consensus.cc:2243] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.498857 10011 raft_consensus.cc:2272] T 050a9774d2d84e8686f55f7198286712 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.501349 10011 tablet_replica.cc:333] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.502048 10011 raft_consensus.cc:2243] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.502506 10011 raft_consensus.cc:2272] T e41a3923298149d5bb746f9af9916a17 P 5823e004c86c4257b8b0e25da0fb8102 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.506052 10011 tablet_replica.cc:333] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.506682 10011 raft_consensus.cc:2243] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.507694 10011 raft_consensus.cc:2272] T f12134c8100e45b8b2321b8c6b1e342e P 5823e004c86c4257b8b0e25da0fb8102 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.510252 10011 tablet_replica.cc:333] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.510931 10011 raft_consensus.cc:2243] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.511415 10011 raft_consensus.cc:2272] T 5a0dae4b76494f108367efbd4676b0de P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.515348 10011 tablet_replica.cc:333] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.516223 10011 raft_consensus.cc:2243] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.517396 10011 raft_consensus.cc:2272] T 0fc7aca87fcd41549f8f2e90a42cb925 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.521701 10011 tablet_replica.cc:333] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.522516 10011 raft_consensus.cc:2243] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.523309 10011 raft_consensus.cc:2272] T 899030d87cdd4814a01f394782e836b6 P 5823e004c86c4257b8b0e25da0fb8102 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:57.523988 22360 consensus_peers.cc:597] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.526216 10011 tablet_replica.cc:333] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.527100 10011 raft_consensus.cc:2243] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.528334 10011 raft_consensus.cc:2272] T dd63024784244f99b9dd7cb8d9fd67c9 P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.532315 10011 tablet_replica.cc:333] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
W20260709 12:17:57.532924 22434 consensus_peers.cc:597] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.533228 10011 raft_consensus.cc:2243] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.533974 10011 raft_consensus.cc:2272] T 50f9c35d2acf470c9b81f6d5e71b16e5 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.536520 10011 tablet_replica.cc:333] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.537063 10011 raft_consensus.cc:2243] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.538105 10011 raft_consensus.cc:2272] T 6bf4075e9de7430bbbf1d3d362a5b585 P 5823e004c86c4257b8b0e25da0fb8102 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.540428 10011 tablet_replica.cc:333] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102: stopping tablet replica
I20260709 12:17:57.540922 10011 raft_consensus.cc:2243] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.541414 10011 raft_consensus.cc:2272] T b939523530634117855680eb8e3713ea P 5823e004c86c4257b8b0e25da0fb8102 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.577358 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
W20260709 12:17:57.577935 22434 consensus_peers.cc:597] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.605852 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
W20260709 12:17:57.615710 22433 consensus_peers.cc:597] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 -> Peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Couldn't send request to peer 5823e004c86c4257b8b0e25da0fb8102. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:33705: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:57.638722 22434 consensus_peers.cc:597] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 -> Peer 18929f381fe84136b29bfbcc3c9aad3f (127.9.198.193:42209): Couldn't send request to peer 18929f381fe84136b29bfbcc3c9aad3f. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:42209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.641589 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:57.642427 10011 tablet_replica.cc:333] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.643154 10011 raft_consensus.cc:2243] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.644079 10011 raft_consensus.cc:2272] T 050a9774d2d84e8686f55f7198286712 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.646646 10011 tablet_replica.cc:333] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.647202 10011 raft_consensus.cc:2243] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.647691 10011 raft_consensus.cc:2272] T 0fc7aca87fcd41549f8f2e90a42cb925 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.650061 10011 tablet_replica.cc:333] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.650575 10011 raft_consensus.cc:2243] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.651356 10011 raft_consensus.cc:2272] T 5a0dae4b76494f108367efbd4676b0de P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.654946 10011 tablet_replica.cc:333] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.656144 10011 raft_consensus.cc:2243] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.656759 10011 raft_consensus.cc:2272] T dd63024784244f99b9dd7cb8d9fd67c9 P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:57.656790 22267 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:57.659284 10011 tablet_replica.cc:333] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.659952 10011 raft_consensus.cc:2243] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.661032 10011 raft_consensus.cc:2272] T 50f9c35d2acf470c9b81f6d5e71b16e5 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.664448 10011 tablet_replica.cc:333] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.665130 10011 raft_consensus.cc:2243] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:17:57.665797 22433 consensus_peers.cc:597] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 -> Peer 5823e004c86c4257b8b0e25da0fb8102 (127.9.198.194:33705): Couldn't send request to peer 5823e004c86c4257b8b0e25da0fb8102. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:33705: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:57.666452 10011 raft_consensus.cc:2272] T 899030d87cdd4814a01f394782e836b6 P d2c2eb9733334ac587cd465847499c71 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.668817 10011 tablet_replica.cc:333] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.669361 10011 raft_consensus.cc:2243] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.669955 10011 raft_consensus.cc:2272] T 6bf4075e9de7430bbbf1d3d362a5b585 P d2c2eb9733334ac587cd465847499c71 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.672333 10011 tablet_replica.cc:333] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.672875 10011 raft_consensus.cc:2243] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.673834 10011 raft_consensus.cc:2272] T b939523530634117855680eb8e3713ea P d2c2eb9733334ac587cd465847499c71 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.676349 10011 tablet_replica.cc:333] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.676867 10011 raft_consensus.cc:2243] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:57.677306 10011 raft_consensus.cc:2272] T f12134c8100e45b8b2321b8c6b1e342e P d2c2eb9733334ac587cd465847499c71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.679790 10011 tablet_replica.cc:333] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71: stopping tablet replica
I20260709 12:17:57.680279 10011 raft_consensus.cc:2243] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.681103 10011 raft_consensus.cc:2272] T e41a3923298149d5bb746f9af9916a17 P d2c2eb9733334ac587cd465847499c71 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.715112 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:17:57.744076 10011 master.cc:562] Master@127.9.198.254:44497 shutting down...
I20260709 12:17:57.770534 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:57.771137 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:57.771503 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P b58eaca590ba4825acfb32537e46c13c: stopping tablet replica
I20260709 12:17:57.792846 10011 master.cc:584] Master@127.9.198.254:44497 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13404 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:17:57.839633 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:33063
I20260709 12:17:57.840818 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:57.846959 22813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:57.847577 22816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:57.847995 22814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:57.848562 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:57.849551 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:57.849792 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:57.849910 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599477849898 us; error 0 us; skew 500 ppm
I20260709 12:17:57.850391 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:57.852769 10011 webserver.cc:533] Webserver started at http://127.9.198.254:41497/ using document root <none> and password file <none>
I20260709 12:17:57.853258 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:57.853416 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:57.853981 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:57.855161 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "02e347683fb244debf73cbb6550f1ad8"
format_stamp: "Formatted at 2026-07-09 12:17:57 on dist-test-slave-glnm"
I20260709 12:17:57.859689 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:17:57.863158 22821 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:57.863924 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:17:57.864197 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "02e347683fb244debf73cbb6550f1ad8"
format_stamp: "Formatted at 2026-07-09 12:17:57 on dist-test-slave-glnm"
I20260709 12:17:57.864440 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:57.884830 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:57.886157 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:57.925621 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:33063
I20260709 12:17:57.925817 22872 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:33063 every 8 connection(s)
I20260709 12:17:57.929957 22873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:57.942394 22873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8: Bootstrap starting.
I20260709 12:17:57.947449 22873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:57.951794 22873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8: No bootstrap required, opened a new log
I20260709 12:17:57.954043 22873 raft_consensus.cc:359] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e347683fb244debf73cbb6550f1ad8" member_type: VOTER }
I20260709 12:17:57.954432 22873 raft_consensus.cc:385] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:57.954614 22873 raft_consensus.cc:740] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e347683fb244debf73cbb6550f1ad8, State: Initialized, Role: FOLLOWER
I20260709 12:17:57.955125 22873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e347683fb244debf73cbb6550f1ad8" member_type: VOTER }
I20260709 12:17:57.955528 22873 raft_consensus.cc:399] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:57.955721 22873 raft_consensus.cc:493] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:57.955940 22873 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:57.960951 22873 raft_consensus.cc:515] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e347683fb244debf73cbb6550f1ad8" member_type: VOTER }
I20260709 12:17:57.961442 22873 leader_election.cc:304] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02e347683fb244debf73cbb6550f1ad8; no voters: 
I20260709 12:17:57.962623 22873 leader_election.cc:290] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:57.963153 22876 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:57.964612 22876 raft_consensus.cc:697] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 1 LEADER]: Becoming Leader. State: Replica: 02e347683fb244debf73cbb6550f1ad8, State: Running, Role: LEADER
I20260709 12:17:57.965349 22876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e347683fb244debf73cbb6550f1ad8" member_type: VOTER }
I20260709 12:17:57.965983 22873 sys_catalog.cc:565] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:57.970366 22877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02e347683fb244debf73cbb6550f1ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e347683fb244debf73cbb6550f1ad8" member_type: VOTER } }
I20260709 12:17:57.971491 22878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02e347683fb244debf73cbb6550f1ad8. Latest consensus state: current_term: 1 leader_uuid: "02e347683fb244debf73cbb6550f1ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02e347683fb244debf73cbb6550f1ad8" member_type: VOTER } }
I20260709 12:17:57.972229 22877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:57.972343 22878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:57.977064 22885 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:57.983562 22885 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:57.984795 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:17:57.993254 22885 catalog_manager.cc:1383] Generated new cluster ID: 746e4d0d0a734df29e98d2c4d17532cc
I20260709 12:17:57.993573 22885 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:58.019759 22885 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:58.021131 22885 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:58.034685 22885 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8: Generated new TSK 0
I20260709 12:17:58.035377 22885 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:58.052112 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:58.058983 22894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:58.059613 22895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:58.060943 22897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:58.062132 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:58.063179 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:58.063426 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:58.063596 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599478063579 us; error 0 us; skew 500 ppm
I20260709 12:17:58.064188 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:58.066843 10011 webserver.cc:533] Webserver started at http://127.9.198.193:45085/ using document root <none> and password file <none>
I20260709 12:17:58.067404 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:58.067611 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:58.067891 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:58.069052 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "4fa9f9b388db484e91b1870d7ff25206"
format_stamp: "Formatted at 2026-07-09 12:17:58 on dist-test-slave-glnm"
I20260709 12:17:58.073884 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:17:58.077590 22902 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.078459 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:17:58.078778 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "4fa9f9b388db484e91b1870d7ff25206"
format_stamp: "Formatted at 2026-07-09 12:17:58 on dist-test-slave-glnm"
I20260709 12:17:58.079090 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:58.111513 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:58.112831 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:58.114441 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:58.116840 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:58.117067 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.117334 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:58.117528 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.163244 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:45151
I20260709 12:17:58.163352 22965 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:45151 every 8 connection(s)
I20260709 12:17:58.169328 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:58.177577 22970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:58.180065 22971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:58.182488 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:58.183401 22966 heartbeater.cc:344] Connected to a master server at 127.9.198.254:33063
W20260709 12:17:58.183566 22973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:58.183873 22966 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:58.184757 22966 heartbeater.cc:507] Master 127.9.198.254:33063 requested a full tablet report, sending...
I20260709 12:17:58.184784 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:58.185159 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:58.185367 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599478185345 us; error 0 us; skew 500 ppm
I20260709 12:17:58.186127 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:58.187213 22838 ts_manager.cc:194] Registered new tserver with Master: 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193:45151)
I20260709 12:17:58.189074 10011 webserver.cc:533] Webserver started at http://127.9.198.194:46603/ using document root <none> and password file <none>
I20260709 12:17:58.189553 22838 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55072
I20260709 12:17:58.189797 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:58.190172 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:58.190482 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:58.191859 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "f42e6a7ca86d4c0cbae013966ee77306"
format_stamp: "Formatted at 2026-07-09 12:17:58 on dist-test-slave-glnm"
I20260709 12:17:58.196897 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:17:58.200851 22978 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.201750 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:17:58.202109 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "f42e6a7ca86d4c0cbae013966ee77306"
format_stamp: "Formatted at 2026-07-09 12:17:58 on dist-test-slave-glnm"
I20260709 12:17:58.202466 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:58.234855 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:58.236261 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:58.237920 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:58.240409 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:58.240671 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.240942 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:58.241142 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.285010 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:34967
I20260709 12:17:58.285112 23041 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:34967 every 8 connection(s)
I20260709 12:17:58.290711 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:58.299530 23045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:58.300362 23046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:58.301860 23048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:58.303989 23042 heartbeater.cc:344] Connected to a master server at 127.9.198.254:33063
I20260709 12:17:58.304733 23042 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:58.305992 23042 heartbeater.cc:507] Master 127.9.198.254:33063 requested a full tablet report, sending...
I20260709 12:17:58.308338 10011 server_base.cc:1061] running on GCE node
I20260709 12:17:58.308622 22838 ts_manager.cc:194] Registered new tserver with Master: f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967)
I20260709 12:17:58.309319 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:58.309602 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:58.309847 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599478309830 us; error 0 us; skew 500 ppm
I20260709 12:17:58.310465 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:58.310519 22838 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55086
I20260709 12:17:58.313505 10011 webserver.cc:533] Webserver started at http://127.9.198.195:46013/ using document root <none> and password file <none>
I20260709 12:17:58.314141 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:58.314368 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:58.314639 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:58.315812 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "e68da64289de492a8004d6926aafa9e5"
format_stamp: "Formatted at 2026-07-09 12:17:58 on dist-test-slave-glnm"
I20260709 12:17:58.320369 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:17:58.323850 23053 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.324682 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:17:58.325035 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "e68da64289de492a8004d6926aafa9e5"
format_stamp: "Formatted at 2026-07-09 12:17:58 on dist-test-slave-glnm"
I20260709 12:17:58.325359 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:58.345901 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:58.347147 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:58.348838 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:58.351178 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:58.351440 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.351696 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:58.351887 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:58.394127 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:35269
I20260709 12:17:58.394261 23115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:35269 every 8 connection(s)
I20260709 12:17:58.408593 23116 heartbeater.cc:344] Connected to a master server at 127.9.198.254:33063
I20260709 12:17:58.409071 23116 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:58.409930 23116 heartbeater.cc:507] Master 127.9.198.254:33063 requested a full tablet report, sending...
I20260709 12:17:58.411978 22838 ts_manager.cc:194] Registered new tserver with Master: e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:58.412159 10011 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014825591s
I20260709 12:17:58.413877 22838 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55098
I20260709 12:17:58.440049 22838 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55102:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:58.442456 22838 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:58.506886 22930 tablet_service.cc:1511] Processing CreateTablet for tablet b892daedebde4bb6858eeb5b3022acc5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=678b4c401a324fa2a0944d9c9c386c11]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:58.508244 22930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b892daedebde4bb6858eeb5b3022acc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.509824 22931 tablet_service.cc:1511] Processing CreateTablet for tablet 219ef8b150f14dbcbfceba7664213de6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=678b4c401a324fa2a0944d9c9c386c11]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:58.511678 22931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219ef8b150f14dbcbfceba7664213de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.511325 23005 tablet_service.cc:1511] Processing CreateTablet for tablet b892daedebde4bb6858eeb5b3022acc5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=678b4c401a324fa2a0944d9c9c386c11]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:58.512866 23005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b892daedebde4bb6858eeb5b3022acc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.509990 23006 tablet_service.cc:1511] Processing CreateTablet for tablet 219ef8b150f14dbcbfceba7664213de6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=678b4c401a324fa2a0944d9c9c386c11]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:58.514482 23006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219ef8b150f14dbcbfceba7664213de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.539069 23081 tablet_service.cc:1511] Processing CreateTablet for tablet 219ef8b150f14dbcbfceba7664213de6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=678b4c401a324fa2a0944d9c9c386c11]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:58.541160 23081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219ef8b150f14dbcbfceba7664213de6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.543532 23080 tablet_service.cc:1511] Processing CreateTablet for tablet b892daedebde4bb6858eeb5b3022acc5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=678b4c401a324fa2a0944d9c9c386c11]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:58.545754 23080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b892daedebde4bb6858eeb5b3022acc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.569394 23131 tablet_bootstrap.cc:492] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206: Bootstrap starting.
I20260709 12:17:58.572052 23132 tablet_bootstrap.cc:492] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306: Bootstrap starting.
I20260709 12:17:58.586205 23131 tablet_bootstrap.cc:654] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.588588 23133 tablet_bootstrap.cc:492] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5: Bootstrap starting.
I20260709 12:17:58.589181 23132 tablet_bootstrap.cc:654] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.593801 23131 tablet_bootstrap.cc:492] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206: No bootstrap required, opened a new log
I20260709 12:17:58.594316 23131 ts_tablet_manager.cc:1403] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.003s
I20260709 12:17:58.596191 23132 tablet_bootstrap.cc:492] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306: No bootstrap required, opened a new log
I20260709 12:17:58.596771 23132 ts_tablet_manager.cc:1403] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306: Time spent bootstrapping tablet: real 0.025s	user 0.000s	sys 0.016s
I20260709 12:17:58.596868 23133 tablet_bootstrap.cc:654] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.597487 23131 raft_consensus.cc:359] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.598353 23131 raft_consensus.cc:385] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.598647 23131 raft_consensus.cc:740] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.599429 23131 consensus_queue.cc:260] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.600072 23132 raft_consensus.cc:359] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.600999 23132 raft_consensus.cc:385] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.601400 23132 raft_consensus.cc:740] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42e6a7ca86d4c0cbae013966ee77306, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.602792 23133 tablet_bootstrap.cc:492] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5: No bootstrap required, opened a new log
I20260709 12:17:58.602535 23132 consensus_queue.cc:260] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.603116 23131 ts_tablet_manager.cc:1434] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:17:58.603291 23133 ts_tablet_manager.cc:1403] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.007s
I20260709 12:17:58.604449 23131 tablet_bootstrap.cc:492] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206: Bootstrap starting.
I20260709 12:17:58.605260 23042 heartbeater.cc:499] Master 127.9.198.254:33063 was elected leader, sending a full tablet report...
I20260709 12:17:58.605782 23132 ts_tablet_manager.cc:1434] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.005s
I20260709 12:17:58.606848 23133 raft_consensus.cc:359] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.607637 23133 raft_consensus.cc:385] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.607887 23133 raft_consensus.cc:740] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68da64289de492a8004d6926aafa9e5, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.608846 22966 heartbeater.cc:499] Master 127.9.198.254:33063 was elected leader, sending a full tablet report...
I20260709 12:17:58.609977 23133 consensus_queue.cc:260] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.606839 23132 tablet_bootstrap.cc:492] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306: Bootstrap starting.
I20260709 12:17:58.612982 23131 tablet_bootstrap.cc:654] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.621309 23132 tablet_bootstrap.cc:654] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.636006 23132 tablet_bootstrap.cc:492] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306: No bootstrap required, opened a new log
I20260709 12:17:58.636791 23132 ts_tablet_manager.cc:1403] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.006s
I20260709 12:17:58.639858 23116 heartbeater.cc:499] Master 127.9.198.254:33063 was elected leader, sending a full tablet report...
I20260709 12:17:58.640005 23133 ts_tablet_manager.cc:1434] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5: Time spent starting tablet: real 0.036s	user 0.021s	sys 0.011s
I20260709 12:17:58.641171 23133 tablet_bootstrap.cc:492] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5: Bootstrap starting.
I20260709 12:17:58.642434 23131 tablet_bootstrap.cc:492] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206: No bootstrap required, opened a new log
I20260709 12:17:58.641391 23132 raft_consensus.cc:359] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.643149 23131 ts_tablet_manager.cc:1403] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.015s
I20260709 12:17:58.643162 23132 raft_consensus.cc:385] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.643692 23132 raft_consensus.cc:740] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42e6a7ca86d4c0cbae013966ee77306, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.644728 23132 consensus_queue.cc:260] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.646837 23131 raft_consensus.cc:359] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.647716 23131 raft_consensus.cc:385] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.648072 23131 raft_consensus.cc:740] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.649108 23131 consensus_queue.cc:260] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.649860 23133 tablet_bootstrap.cc:654] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.651768 23131 ts_tablet_manager.cc:1434] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:58.653815 23132 ts_tablet_manager.cc:1434] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 12:17:58.658802 23133 tablet_bootstrap.cc:492] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5: No bootstrap required, opened a new log
I20260709 12:17:58.659420 23133 ts_tablet_manager.cc:1403] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.008s
I20260709 12:17:58.662623 23133 raft_consensus.cc:359] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.663403 23133 raft_consensus.cc:385] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.663856 23133 raft_consensus.cc:740] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68da64289de492a8004d6926aafa9e5, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.664670 23133 consensus_queue.cc:260] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.666378 23133 ts_tablet_manager.cc:1434] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:17:58.729514 23137 raft_consensus.cc:493] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.730199 23137 raft_consensus.cc:515] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.732542 23137 leader_election.cc:290] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:58.742594 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219ef8b150f14dbcbfceba7664213de6" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306" is_pre_election: true
I20260709 12:17:58.743351 23017 raft_consensus.cc:2468] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 0.
I20260709 12:17:58.743902 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219ef8b150f14dbcbfceba7664213de6" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68da64289de492a8004d6926aafa9e5" is_pre_election: true
I20260709 12:17:58.744625 22905 leader_election.cc:304] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:17:58.744746 23091 raft_consensus.cc:2468] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 0.
I20260709 12:17:58.745548 23137 raft_consensus.cc:2804] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:58.745978 23137 raft_consensus.cc:493] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.746346 23137 raft_consensus.cc:3060] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:58.752319 23137 raft_consensus.cc:515] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.754711 23137 leader_election.cc:290] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 election: Requested vote from peers f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:58.755146 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219ef8b150f14dbcbfceba7664213de6" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
I20260709 12:17:58.755364 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219ef8b150f14dbcbfceba7664213de6" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68da64289de492a8004d6926aafa9e5"
I20260709 12:17:58.755952 23017 raft_consensus.cc:3060] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:58.755952 23091 raft_consensus.cc:3060] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:58.764771 23017 raft_consensus.cc:2468] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 1.
I20260709 12:17:58.764771 23091 raft_consensus.cc:2468] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 1.
I20260709 12:17:58.766515 22905 leader_election.cc:304] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:17:58.767455 23137 raft_consensus.cc:2804] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:58.768661 23137 raft_consensus.cc:697] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 LEADER]: Becoming Leader. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Running, Role: LEADER
I20260709 12:17:58.769411 23137 consensus_queue.cc:237] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.777683 22837 catalog_manager.cc:5697] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "4fa9f9b388db484e91b1870d7ff25206" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:58.796878 23137 raft_consensus.cc:493] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.797503 23137 raft_consensus.cc:515] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.799367 23137 leader_election.cc:290] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:58.800401 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306" is_pre_election: true
I20260709 12:17:58.800462 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68da64289de492a8004d6926aafa9e5" is_pre_election: true
I20260709 12:17:58.801059 23017 raft_consensus.cc:2468] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 0.
I20260709 12:17:58.801302 23091 raft_consensus.cc:2468] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 0.
I20260709 12:17:58.802259 22905 leader_election.cc:304] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:17:58.803093 23137 raft_consensus.cc:2804] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:58.803495 23137 raft_consensus.cc:493] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.803755 23137 raft_consensus.cc:3060] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:58.809051 23137 raft_consensus.cc:515] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.810781 23137 leader_election.cc:290] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 election: Requested vote from peers f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:58.811697 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
I20260709 12:17:58.811858 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68da64289de492a8004d6926aafa9e5"
I20260709 12:17:58.812283 23017 raft_consensus.cc:3060] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:58.812460 23091 raft_consensus.cc:3060] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:58.817706 23017 raft_consensus.cc:2468] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 1.
I20260709 12:17:58.817803 23091 raft_consensus.cc:2468] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 1.
I20260709 12:17:58.818921 22905 leader_election.cc:304] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:17:58.819630 23137 raft_consensus.cc:2804] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:58.820138 23137 raft_consensus.cc:697] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 LEADER]: Becoming Leader. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Running, Role: LEADER
I20260709 12:17:58.820977 23137 consensus_queue.cc:237] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.828035 22837 catalog_manager.cc:5697] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "4fa9f9b388db484e91b1870d7ff25206" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:58.864009 22836 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55114:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:58.866107 22836 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:58.891723 22930 tablet_service.cc:1511] Processing CreateTablet for tablet 7eaf5e5e7f6a48e58dabee5426dde638 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d65b6f88039e4ff0a93c5f937e4a484f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:58.893117 22930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eaf5e5e7f6a48e58dabee5426dde638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.892916 23005 tablet_service.cc:1511] Processing CreateTablet for tablet 7eaf5e5e7f6a48e58dabee5426dde638 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d65b6f88039e4ff0a93c5f937e4a484f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:58.894367 23005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eaf5e5e7f6a48e58dabee5426dde638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.895452 23081 tablet_service.cc:1511] Processing CreateTablet for tablet 7eaf5e5e7f6a48e58dabee5426dde638 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d65b6f88039e4ff0a93c5f937e4a484f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:58.896916 23081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eaf5e5e7f6a48e58dabee5426dde638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.897235 22931 tablet_service.cc:1511] Processing CreateTablet for tablet c864b2d857c144e0970c64b039a4765b (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d65b6f88039e4ff0a93c5f937e4a484f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:58.898694 22931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c864b2d857c144e0970c64b039a4765b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.904516 23006 tablet_service.cc:1511] Processing CreateTablet for tablet c864b2d857c144e0970c64b039a4765b (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d65b6f88039e4ff0a93c5f937e4a484f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:58.906123 23006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c864b2d857c144e0970c64b039a4765b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.911083 23080 tablet_service.cc:1511] Processing CreateTablet for tablet c864b2d857c144e0970c64b039a4765b (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d65b6f88039e4ff0a93c5f937e4a484f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:58.912421 23131 tablet_bootstrap.cc:492] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206: Bootstrap starting.
I20260709 12:17:58.912562 23080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c864b2d857c144e0970c64b039a4765b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:58.919318 23131 tablet_bootstrap.cc:654] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.927429 23133 tablet_bootstrap.cc:492] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5: Bootstrap starting.
I20260709 12:17:58.930639 23132 tablet_bootstrap.cc:492] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306: Bootstrap starting.
I20260709 12:17:58.934944 23133 tablet_bootstrap.cc:654] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.937809 23132 tablet_bootstrap.cc:654] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.946568 23131 tablet_bootstrap.cc:492] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206: No bootstrap required, opened a new log
I20260709 12:17:58.946630 23133 tablet_bootstrap.cc:492] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5: No bootstrap required, opened a new log
I20260709 12:17:58.947233 23131 ts_tablet_manager.cc:1403] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.000s
I20260709 12:17:58.947544 23133 ts_tablet_manager.cc:1403] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.008s
I20260709 12:17:58.949024 23132 tablet_bootstrap.cc:492] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306: No bootstrap required, opened a new log
I20260709 12:17:58.949697 23132 ts_tablet_manager.cc:1403] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.007s
I20260709 12:17:58.950798 23133 raft_consensus.cc:359] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.950764 23131 raft_consensus.cc:359] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.951692 23133 raft_consensus.cc:385] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.951778 23131 raft_consensus.cc:385] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.952033 23133 raft_consensus.cc:740] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68da64289de492a8004d6926aafa9e5, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.952257 23131 raft_consensus.cc:740] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.952867 23132 raft_consensus.cc:359] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.952903 23133 consensus_queue.cc:260] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.953346 23131 consensus_queue.cc:260] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.953923 23132 raft_consensus.cc:385] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:58.954267 23132 raft_consensus.cc:740] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42e6a7ca86d4c0cbae013966ee77306, State: Initialized, Role: FOLLOWER
I20260709 12:17:58.955718 23131 ts_tablet_manager.cc:1434] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:17:58.955718 23132 consensus_queue.cc:260] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.964197 23131 tablet_bootstrap.cc:492] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206: Bootstrap starting.
I20260709 12:17:58.968922 23133 ts_tablet_manager.cc:1434] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:17:58.971899 23138 raft_consensus.cc:493] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.972834 23133 tablet_bootstrap.cc:492] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5: Bootstrap starting.
I20260709 12:17:58.972525 23138 raft_consensus.cc:515] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:58.974028 23131 tablet_bootstrap.cc:654] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.975306 23138 leader_election.cc:290] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193:45151), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
W20260709 12:17:58.978384 22886 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:58.979290 23132 ts_tablet_manager.cc:1434] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.010s
I20260709 12:17:58.980567 23132 tablet_bootstrap.cc:492] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: Bootstrap starting.
I20260709 12:17:58.995086 23133 tablet_bootstrap.cc:654] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.997243 23131 tablet_bootstrap.cc:492] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206: No bootstrap required, opened a new log
I20260709 12:17:58.997505 23132 tablet_bootstrap.cc:654] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:58.998540 23131 ts_tablet_manager.cc:1403] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.009s
I20260709 12:17:59.002514 23131 raft_consensus.cc:359] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.003970 23131 raft_consensus.cc:385] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:59.004384 23131 raft_consensus.cc:740] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Initialized, Role: FOLLOWER
I20260709 12:17:59.007145 23131 consensus_queue.cc:260] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.010322 23131 ts_tablet_manager.cc:1434] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:17:59.018404 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638" candidate_uuid: "f42e6a7ca86d4c0cbae013966ee77306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68da64289de492a8004d6926aafa9e5" is_pre_election: true
I20260709 12:17:59.019320 23091 raft_consensus.cc:2468] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f42e6a7ca86d4c0cbae013966ee77306 in term 0.
I20260709 12:17:59.020864 22980 leader_election.cc:304] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e68da64289de492a8004d6926aafa9e5, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:17:59.021559 23133 tablet_bootstrap.cc:492] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5: No bootstrap required, opened a new log
I20260709 12:17:59.021716 23138 raft_consensus.cc:2804] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:59.021951 23132 tablet_bootstrap.cc:492] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: No bootstrap required, opened a new log
I20260709 12:17:59.022363 23133 ts_tablet_manager.cc:1403] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.008s
I20260709 12:17:59.022639 23132 ts_tablet_manager.cc:1403] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: Time spent bootstrapping tablet: real 0.042s	user 0.025s	sys 0.006s
I20260709 12:17:59.022333 23138 raft_consensus.cc:493] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.023249 23138 raft_consensus.cc:3060] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:59.026044 23133 raft_consensus.cc:359] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.026225 23132 raft_consensus.cc:359] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.027098 23132 raft_consensus.cc:385] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:59.027158 23133 raft_consensus.cc:385] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:59.027518 23132 raft_consensus.cc:740] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42e6a7ca86d4c0cbae013966ee77306, State: Initialized, Role: FOLLOWER
I20260709 12:17:59.027568 23133 raft_consensus.cc:740] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68da64289de492a8004d6926aafa9e5, State: Initialized, Role: FOLLOWER
I20260709 12:17:59.028474 23132 consensus_queue.cc:260] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.028611 23133 consensus_queue.cc:260] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.031446 23133 ts_tablet_manager.cc:1434] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:17:59.035953 22941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638" candidate_uuid: "f42e6a7ca86d4c0cbae013966ee77306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa9f9b388db484e91b1870d7ff25206" is_pre_election: true
I20260709 12:17:59.036841 23132 ts_tablet_manager.cc:1434] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 12:17:59.036804 22941 raft_consensus.cc:2468] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f42e6a7ca86d4c0cbae013966ee77306 in term 0.
I20260709 12:17:59.035912 23138 raft_consensus.cc:515] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.041999 22941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638" candidate_uuid: "f42e6a7ca86d4c0cbae013966ee77306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
I20260709 12:17:59.042610 22941 raft_consensus.cc:3060] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:59.049281 22941 raft_consensus.cc:2468] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f42e6a7ca86d4c0cbae013966ee77306 in term 1.
I20260709 12:17:59.049429 23138 leader_election.cc:290] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [CANDIDATE]: Term 1 election: Requested vote from peers 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193:45151), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:59.050487 22979 leader_election.cc:304] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:17:59.050684 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638" candidate_uuid: "f42e6a7ca86d4c0cbae013966ee77306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e68da64289de492a8004d6926aafa9e5"
I20260709 12:17:59.051401 23091 raft_consensus.cc:3060] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:59.051484 23138 raft_consensus.cc:2804] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:59.052050 23138 raft_consensus.cc:697] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 LEADER]: Becoming Leader. State: Replica: f42e6a7ca86d4c0cbae013966ee77306, State: Running, Role: LEADER
I20260709 12:17:59.052732 23138 consensus_queue.cc:237] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.057943 23091 raft_consensus.cc:2468] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f42e6a7ca86d4c0cbae013966ee77306 in term 1.
I20260709 12:17:59.060962 22836 catalog_manager.cc:5697] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 reported cstate change: term changed from 0 to 1, leader changed from <none> to f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "f42e6a7ca86d4c0cbae013966ee77306" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:59.160120 23139 raft_consensus.cc:493] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.160780 23139 raft_consensus.cc:515] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.163442 23139 leader_election.cc:290] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193:45151), f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967)
I20260709 12:17:59.174500 22941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "e68da64289de492a8004d6926aafa9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa9f9b388db484e91b1870d7ff25206" is_pre_election: true
I20260709 12:17:59.175347 22941 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e68da64289de492a8004d6926aafa9e5 in term 0.
I20260709 12:17:59.176061 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "e68da64289de492a8004d6926aafa9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306" is_pre_election: true
I20260709 12:17:59.176563 23054 leader_election.cc:304] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, e68da64289de492a8004d6926aafa9e5; no voters: 
I20260709 12:17:59.176925 23017 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e68da64289de492a8004d6926aafa9e5 in term 0.
I20260709 12:17:59.177520 23139 raft_consensus.cc:2804] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:59.177955 23139 raft_consensus.cc:493] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.178305 23139 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:59.184612 23139 raft_consensus.cc:515] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.186412 23139 leader_election.cc:290] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [CANDIDATE]: Term 1 election: Requested vote from peers 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193:45151), f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967)
I20260709 12:17:59.187430 22941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "e68da64289de492a8004d6926aafa9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
I20260709 12:17:59.187556 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "e68da64289de492a8004d6926aafa9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
I20260709 12:17:59.188068 22941 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:59.188177 23017 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:59.193967 23017 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e68da64289de492a8004d6926aafa9e5 in term 1.
I20260709 12:17:59.194015 22941 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e68da64289de492a8004d6926aafa9e5 in term 1.
I20260709 12:17:59.195320 23054 leader_election.cc:304] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, e68da64289de492a8004d6926aafa9e5; no voters: 
I20260709 12:17:59.196105 23139 raft_consensus.cc:2804] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:59.197099 23139 raft_consensus.cc:697] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 LEADER]: Becoming Leader. State: Replica: e68da64289de492a8004d6926aafa9e5, State: Running, Role: LEADER
I20260709 12:17:59.198017 23139 consensus_queue.cc:237] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.205283 22836 catalog_manager.cc:5697] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e68da64289de492a8004d6926aafa9e5 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "e68da64289de492a8004d6926aafa9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:59.243046 22941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5"
dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
mode: GRACEFUL
new_leader_uuid: "e68da64289de492a8004d6926aafa9e5"
 from {username='slave'} at 127.0.0.1:54332
I20260709 12:17:59.243603 22941 raft_consensus.cc:606] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 LEADER]: Received request to transfer leadership to TS e68da64289de492a8004d6926aafa9e5
I20260709 12:17:59.257223 23017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638"
dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
mode: GRACEFUL
new_leader_uuid: "4fa9f9b388db484e91b1870d7ff25206"
 from {username='slave'} at 127.0.0.1:52994
I20260709 12:17:59.257825 23017 raft_consensus.cc:606] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 LEADER]: Received request to transfer leadership to TS 4fa9f9b388db484e91b1870d7ff25206
I20260709 12:17:59.271929 23137 consensus_queue.cc:1048] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:59.272301 23091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b"
dest_uuid: "e68da64289de492a8004d6926aafa9e5"
mode: GRACEFUL
new_leader_uuid: "4fa9f9b388db484e91b1870d7ff25206"
 from {username='slave'} at 127.0.0.1:41958
I20260709 12:17:59.272974 23091 raft_consensus.cc:606] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 LEADER]: Received request to transfer leadership to TS 4fa9f9b388db484e91b1870d7ff25206
I20260709 12:17:59.283728 23170 consensus_queue.cc:1048] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:59.283768 23137 raft_consensus.cc:993] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206: : Instructing follower e68da64289de492a8004d6926aafa9e5 to start an election
I20260709 12:17:59.284881 23137 raft_consensus.cc:1081] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 LEADER]: Signalling peer e68da64289de492a8004d6926aafa9e5 to start an election
I20260709 12:17:59.286944 23090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5"
dest_uuid: "e68da64289de492a8004d6926aafa9e5"
 from {username='slave'} at 127.0.0.1:41926
I20260709 12:17:59.308738 23137 consensus_queue.cc:1048] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:59.311527 23137 consensus_queue.cc:1048] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:59.332628 23090 raft_consensus.cc:493] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:59.333158 23090 raft_consensus.cc:3060] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.342973 23090 raft_consensus.cc:515] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.350327 22941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5" candidate_uuid: "e68da64289de492a8004d6926aafa9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
I20260709 12:17:59.350764 23017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b892daedebde4bb6858eeb5b3022acc5" candidate_uuid: "e68da64289de492a8004d6926aafa9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
I20260709 12:17:59.352792 23090 leader_election.cc:290] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [CANDIDATE]: Term 2 election: Requested vote from peers 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193:45151), f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967)
I20260709 12:17:59.360543 22941 raft_consensus.cc:3055] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:59.361078 22941 raft_consensus.cc:740] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Running, Role: LEADER
I20260709 12:17:59.362124 22941 consensus_queue.cc:260] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.363775 22941 raft_consensus.cc:3060] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.371815 22941 raft_consensus.cc:2468] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e68da64289de492a8004d6926aafa9e5 in term 2.
I20260709 12:17:59.375746 23054 leader_election.cc:304] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, e68da64289de492a8004d6926aafa9e5; no voters: f42e6a7ca86d4c0cbae013966ee77306
I20260709 12:17:59.376658 23139 raft_consensus.cc:2804] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:59.377209 23139 raft_consensus.cc:697] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 2 LEADER]: Becoming Leader. State: Replica: e68da64289de492a8004d6926aafa9e5, State: Running, Role: LEADER
I20260709 12:17:59.378067 23139 consensus_queue.cc:237] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.386669 22836 catalog_manager.cc:5697] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 reported cstate change: term changed from 1 to 2, leader changed from 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193) to e68da64289de492a8004d6926aafa9e5 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "e68da64289de492a8004d6926aafa9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:59.450928 23138 consensus_queue.cc:1048] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:59.458122 23162 raft_consensus.cc:993] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306: : Instructing follower 4fa9f9b388db484e91b1870d7ff25206 to start an election
I20260709 12:17:59.459545 23138 raft_consensus.cc:1081] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 LEADER]: Signalling peer 4fa9f9b388db484e91b1870d7ff25206 to start an election
I20260709 12:17:59.463145 22940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638"
dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
 from {username='slave'} at 127.0.0.1:54306
I20260709 12:17:59.463824 22940 raft_consensus.cc:493] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:59.464251 22940 raft_consensus.cc:3060] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.466305 23138 consensus_queue.cc:1048] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:59.477185 22940 raft_consensus.cc:515] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.480455 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
I20260709 12:17:59.481133 23016 raft_consensus.cc:3055] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:59.481324 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eaf5e5e7f6a48e58dabee5426dde638" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e68da64289de492a8004d6926aafa9e5"
I20260709 12:17:59.481524 23016 raft_consensus.cc:740] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f42e6a7ca86d4c0cbae013966ee77306, State: Running, Role: LEADER
I20260709 12:17:59.482640 23016 consensus_queue.cc:260] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.483632 23016 raft_consensus.cc:3060] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.485529 22940 leader_election.cc:290] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 2 election: Requested vote from peers f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:59.492883 23016 raft_consensus.cc:2468] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 2.
I20260709 12:17:59.494143 22905 leader_election.cc:304] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: e68da64289de492a8004d6926aafa9e5
I20260709 12:17:59.494988 23170 raft_consensus.cc:2804] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:59.495522 23170 raft_consensus.cc:697] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 LEADER]: Becoming Leader. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Running, Role: LEADER
I20260709 12:17:59.496313 23170 consensus_queue.cc:237] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.502595 22838 catalog_manager.cc:5697] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 reported cstate change: term changed from 1 to 2, leader changed from f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194) to 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4fa9f9b388db484e91b1870d7ff25206" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:59.687220 23139 consensus_queue.cc:1048] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:59.698140 23139 consensus_queue.cc:1048] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:59.708016 23185 raft_consensus.cc:993] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5: : Instructing follower 4fa9f9b388db484e91b1870d7ff25206 to start an election
I20260709 12:17:59.708516 23139 raft_consensus.cc:1081] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 LEADER]: Signalling peer 4fa9f9b388db484e91b1870d7ff25206 to start an election
I20260709 12:17:59.710171 22940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b"
dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
 from {username='slave'} at 127.0.0.1:54318
I20260709 12:17:59.710793 22940 raft_consensus.cc:493] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:59.711220 22940 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.716539 22940 raft_consensus.cc:515] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.718320 22940 leader_election.cc:290] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 2 election: Requested vote from peers f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194:34967), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:17:59.719344 23016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
I20260709 12:17:59.719463 23090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "4fa9f9b388db484e91b1870d7ff25206" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e68da64289de492a8004d6926aafa9e5"
I20260709 12:17:59.720108 23016 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.720247 23090 raft_consensus.cc:3055] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:59.720765 23090 raft_consensus.cc:740] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e68da64289de492a8004d6926aafa9e5, State: Running, Role: LEADER
I20260709 12:17:59.721540 23090 consensus_queue.cc:260] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.722659 23090 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.726774 23016 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 2.
I20260709 12:17:59.728045 22905 leader_election.cc:304] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:17:59.728832 23170 raft_consensus.cc:2804] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:59.728871 23090 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa9f9b388db484e91b1870d7ff25206 in term 2.
I20260709 12:17:59.729363 23170 raft_consensus.cc:697] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 LEADER]: Becoming Leader. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Running, Role: LEADER
I20260709 12:17:59.730378 23170 consensus_queue.cc:237] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:17:59.737128 22837 catalog_manager.cc:5697] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 reported cstate change: term changed from 1 to 2, leader changed from e68da64289de492a8004d6926aafa9e5 (127.9.198.195) to 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4fa9f9b388db484e91b1870d7ff25206" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:59.787699 22940 raft_consensus.cc:1275] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Refusing update from remote peer e68da64289de492a8004d6926aafa9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:59.789177 23139 consensus_queue.cc:1048] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:59.789501 10011 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:17:59.791704 10011 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:17:59.792196 10011 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:17:59.798087 23016 raft_consensus.cc:3060] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.811822 23016 raft_consensus.cc:1275] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Refusing update from remote peer e68da64289de492a8004d6926aafa9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:59.813787 23185 consensus_queue.cc:1048] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:59.822256 22940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b"
dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
mode: GRACEFUL
new_leader_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
 from {username='slave'} at 127.0.0.1:54336
I20260709 12:17:59.822973 22940 raft_consensus.cc:606] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 LEADER]: Received request to transfer leadership to TS f42e6a7ca86d4c0cbae013966ee77306
I20260709 12:17:59.824622 10011 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:17:59.918366 23090 raft_consensus.cc:3060] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:59.924906 23090 raft_consensus.cc:1275] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Refusing update from remote peer 4fa9f9b388db484e91b1870d7ff25206: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:59.926416 23170 consensus_queue.cc:1048] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:59.934521 23016 raft_consensus.cc:1275] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Refusing update from remote peer 4fa9f9b388db484e91b1870d7ff25206: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:59.936301 23137 consensus_queue.cc:1048] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:00.152122 23016 raft_consensus.cc:1275] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Refusing update from remote peer 4fa9f9b388db484e91b1870d7ff25206: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:00.153546 23137 consensus_queue.cc:1048] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:00.159348 23175 raft_consensus.cc:993] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206: : Instructing follower f42e6a7ca86d4c0cbae013966ee77306 to start an election
I20260709 12:18:00.159874 23170 raft_consensus.cc:1081] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 LEADER]: Signalling peer f42e6a7ca86d4c0cbae013966ee77306 to start an election
I20260709 12:18:00.162573 23017 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b"
dest_uuid: "f42e6a7ca86d4c0cbae013966ee77306"
 from {username='slave'} at 127.0.0.1:52974
I20260709 12:18:00.163255 23017 raft_consensus.cc:493] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:00.163625 23017 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.165876 23090 raft_consensus.cc:1275] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Refusing update from remote peer 4fa9f9b388db484e91b1870d7ff25206: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:00.169291 23170 consensus_queue.cc:1048] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:00.173112 23017 raft_consensus.cc:515] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:18:00.176558 22940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "f42e6a7ca86d4c0cbae013966ee77306" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4fa9f9b388db484e91b1870d7ff25206"
I20260709 12:18:00.176743 23017 leader_election.cc:290] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [CANDIDATE]: Term 3 election: Requested vote from peers 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193:45151), e68da64289de492a8004d6926aafa9e5 (127.9.198.195:35269)
I20260709 12:18:00.177307 22940 raft_consensus.cc:3055] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:00.177687 22940 raft_consensus.cc:740] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4fa9f9b388db484e91b1870d7ff25206, State: Running, Role: LEADER
I20260709 12:18:00.178568 22940 consensus_queue.cc:260] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:18:00.179469 22940 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.190864 22940 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f42e6a7ca86d4c0cbae013966ee77306 in term 3.
I20260709 12:18:00.181607 23091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c864b2d857c144e0970c64b039a4765b" candidate_uuid: "f42e6a7ca86d4c0cbae013966ee77306" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e68da64289de492a8004d6926aafa9e5"
I20260709 12:18:00.192174 22979 leader_election.cc:304] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa9f9b388db484e91b1870d7ff25206, f42e6a7ca86d4c0cbae013966ee77306; no voters: 
I20260709 12:18:00.192281 23091 raft_consensus.cc:3060] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:00.193097 23181 raft_consensus.cc:2804] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:00.194340 23181 raft_consensus.cc:697] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 3 LEADER]: Becoming Leader. State: Replica: f42e6a7ca86d4c0cbae013966ee77306, State: Running, Role: LEADER
I20260709 12:18:00.195283 23181 consensus_queue.cc:237] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } }
I20260709 12:18:00.199571 23091 raft_consensus.cc:2468] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f42e6a7ca86d4c0cbae013966ee77306 in term 3.
I20260709 12:18:00.203120 22837 catalog_manager.cc:5697] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 reported cstate change: term changed from 2 to 3, leader changed from 4fa9f9b388db484e91b1870d7ff25206 (127.9.198.193) to f42e6a7ca86d4c0cbae013966ee77306 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "f42e6a7ca86d4c0cbae013966ee77306" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa9f9b388db484e91b1870d7ff25206" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 45151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42e6a7ca86d4c0cbae013966ee77306" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 34967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e68da64289de492a8004d6926aafa9e5" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:00.459061 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
I20260709 12:18:00.479928 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:00.480633 10011 tablet_replica.cc:333] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206: stopping tablet replica
I20260709 12:18:00.481200 10011 raft_consensus.cc:2243] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.481729 10011 raft_consensus.cc:2272] T c864b2d857c144e0970c64b039a4765b P 4fa9f9b388db484e91b1870d7ff25206 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.484705 10011 tablet_replica.cc:333] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206: stopping tablet replica
I20260709 12:18:00.485404 10011 raft_consensus.cc:2243] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:00.486253 10011 raft_consensus.cc:2272] T 7eaf5e5e7f6a48e58dabee5426dde638 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.488952 10011 tablet_replica.cc:333] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206: stopping tablet replica
I20260709 12:18:00.489511 10011 raft_consensus.cc:2243] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:00.490381 10011 raft_consensus.cc:2272] T 219ef8b150f14dbcbfceba7664213de6 P 4fa9f9b388db484e91b1870d7ff25206 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.492379 10011 tablet_replica.cc:333] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206: stopping tablet replica
I20260709 12:18:00.492913 10011 raft_consensus.cc:2243] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.493382 10011 raft_consensus.cc:2272] T b892daedebde4bb6858eeb5b3022acc5 P 4fa9f9b388db484e91b1870d7ff25206 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.508030 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
I20260709 12:18:00.521507 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
I20260709 12:18:00.541059 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:00.541827 10011 tablet_replica.cc:333] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: stopping tablet replica
I20260709 12:18:00.542419 10011 raft_consensus.cc:2243] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:00.543187 10011 pending_rounds.cc:70] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: Trying to abort 1 pending ops.
I20260709 12:18:00.543416 10011 pending_rounds.cc:77] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305623470890061824 op_type: NO_OP noop_request { }
I20260709 12:18:00.543730 10011 raft_consensus.cc:2889] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:18:00.544039 10011 raft_consensus.cc:2272] T c864b2d857c144e0970c64b039a4765b P f42e6a7ca86d4c0cbae013966ee77306 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.546528 10011 tablet_replica.cc:333] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306: stopping tablet replica
I20260709 12:18:00.547078 10011 raft_consensus.cc:2243] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.547524 10011 raft_consensus.cc:2272] T b892daedebde4bb6858eeb5b3022acc5 P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.550168 10011 tablet_replica.cc:333] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306: stopping tablet replica
I20260709 12:18:00.550735 10011 raft_consensus.cc:2243] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.551178 10011 raft_consensus.cc:2272] T 7eaf5e5e7f6a48e58dabee5426dde638 P f42e6a7ca86d4c0cbae013966ee77306 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.553455 10011 tablet_replica.cc:333] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306: stopping tablet replica
I20260709 12:18:00.553997 10011 raft_consensus.cc:2243] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.554418 10011 raft_consensus.cc:2272] T 219ef8b150f14dbcbfceba7664213de6 P f42e6a7ca86d4c0cbae013966ee77306 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.577611 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
I20260709 12:18:00.591370 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
I20260709 12:18:00.611318 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:00.612038 10011 tablet_replica.cc:333] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5: stopping tablet replica
I20260709 12:18:00.612704 10011 raft_consensus.cc:2243] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.613217 10011 raft_consensus.cc:2272] T c864b2d857c144e0970c64b039a4765b P e68da64289de492a8004d6926aafa9e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.615552 10011 tablet_replica.cc:333] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5: stopping tablet replica
I20260709 12:18:00.616112 10011 raft_consensus.cc:2243] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.616560 10011 raft_consensus.cc:2272] T 7eaf5e5e7f6a48e58dabee5426dde638 P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.619220 10011 tablet_replica.cc:333] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5: stopping tablet replica
I20260709 12:18:00.619760 10011 raft_consensus.cc:2243] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:00.620194 10011 raft_consensus.cc:2272] T 219ef8b150f14dbcbfceba7664213de6 P e68da64289de492a8004d6926aafa9e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.622126 10011 tablet_replica.cc:333] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5: stopping tablet replica
I20260709 12:18:00.622599 10011 raft_consensus.cc:2243] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:00.623413 10011 raft_consensus.cc:2272] T b892daedebde4bb6858eeb5b3022acc5 P e68da64289de492a8004d6926aafa9e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.647042 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:18:00.660458 10011 master.cc:562] Master@127.9.198.254:33063 shutting down...
I20260709 12:18:00.677376 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:00.677987 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:00.678284 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P 02e347683fb244debf73cbb6550f1ad8: stopping tablet replica
I20260709 12:18:00.698316 10011 master.cc:584] Master@127.9.198.254:33063 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2888 ms)
[----------] 6 tests from LeaderRebalancerTest (430584 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:18:00.728196 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:43235
I20260709 12:18:00.729364 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:00.735265 23192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:00.735265 23193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:00.736778 23195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:00.736871 10011 server_base.cc:1061] running on GCE node
I20260709 12:18:00.738024 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:00.738207 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:00.738341 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599480738329 us; error 0 us; skew 500 ppm
I20260709 12:18:00.738832 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:00.741047 10011 webserver.cc:533] Webserver started at http://127.9.198.254:37457/ using document root <none> and password file <none>
I20260709 12:18:00.741508 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:00.741735 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:00.742056 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:00.743113 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "0ea09887abd84248b1cf41dd28c939cf"
format_stamp: "Formatted at 2026-07-09 12:18:00 on dist-test-slave-glnm"
I20260709 12:18:00.747684 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:18:00.751210 23200 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:00.752110 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:18:00.752435 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "0ea09887abd84248b1cf41dd28c939cf"
format_stamp: "Formatted at 2026-07-09 12:18:00 on dist-test-slave-glnm"
I20260709 12:18:00.752707 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:00.779577 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:00.780704 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:00.817942 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:43235
I20260709 12:18:00.818096 23251 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:43235 every 8 connection(s)
I20260709 12:18:00.822225 23252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:00.834342 23252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf: Bootstrap starting.
I20260709 12:18:00.839287 23252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:00.843654 23252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf: No bootstrap required, opened a new log
I20260709 12:18:00.846056 23252 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea09887abd84248b1cf41dd28c939cf" member_type: VOTER }
I20260709 12:18:00.846499 23252 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:00.846710 23252 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ea09887abd84248b1cf41dd28c939cf, State: Initialized, Role: FOLLOWER
I20260709 12:18:00.847268 23252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea09887abd84248b1cf41dd28c939cf" member_type: VOTER }
I20260709 12:18:00.847744 23252 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:18:00.847954 23252 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:18:00.848198 23252 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:00.853569 23252 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea09887abd84248b1cf41dd28c939cf" member_type: VOTER }
I20260709 12:18:00.854318 23252 leader_election.cc:304] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ea09887abd84248b1cf41dd28c939cf; no voters: 
I20260709 12:18:00.855625 23252 leader_election.cc:290] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:18:00.856092 23255 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:00.857753 23255 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 1 LEADER]: Becoming Leader. State: Replica: 0ea09887abd84248b1cf41dd28c939cf, State: Running, Role: LEADER
I20260709 12:18:00.858587 23255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea09887abd84248b1cf41dd28c939cf" member_type: VOTER }
I20260709 12:18:00.858897 23252 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:18:00.863430 23256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ea09887abd84248b1cf41dd28c939cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea09887abd84248b1cf41dd28c939cf" member_type: VOTER } }
I20260709 12:18:00.863546 23257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ea09887abd84248b1cf41dd28c939cf. Latest consensus state: current_term: 1 leader_uuid: "0ea09887abd84248b1cf41dd28c939cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea09887abd84248b1cf41dd28c939cf" member_type: VOTER } }
I20260709 12:18:00.864197 23256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [sys.catalog]: This master's current role is: LEADER
I20260709 12:18:00.864226 23257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [sys.catalog]: This master's current role is: LEADER
I20260709 12:18:00.873312 23262 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:18:00.878803 23262 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:18:00.879937 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:18:00.888844 23262 catalog_manager.cc:1383] Generated new cluster ID: 27e0f08bcba042658cbbc74a87814e65
I20260709 12:18:00.889166 23262 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:18:00.902136 23262 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:18:00.903534 23262 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:18:00.916219 23262 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf: Generated new TSK 0
I20260709 12:18:00.916929 23262 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:18:00.947230 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:00.953405 23273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:00.954411 23274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:00.959043 10011 server_base.cc:1061] running on GCE node
W20260709 12:18:00.959285 23276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:00.960421 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:00.960646 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:00.960821 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599480960802 us; error 0 us; skew 500 ppm
I20260709 12:18:00.961479 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:00.963861 10011 webserver.cc:533] Webserver started at http://127.9.198.193:38617/ using document root <none> and password file <none>
I20260709 12:18:00.964342 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:00.964496 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:00.964709 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:00.965945 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "4b6e2a23c11644ec92cc018d8be502eb"
format_stamp: "Formatted at 2026-07-09 12:18:00 on dist-test-slave-glnm"
I20260709 12:18:00.970758 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:18:00.974397 23282 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:00.975361 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:18:00.975693 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "4b6e2a23c11644ec92cc018d8be502eb"
format_stamp: "Formatted at 2026-07-09 12:18:00 on dist-test-slave-glnm"
I20260709 12:18:00.975984 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:01.023289 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:01.024533 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:01.026155 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:01.028602 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:01.028797 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.029121 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:01.029273 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.070052 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:37807
I20260709 12:18:01.070168 23344 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:37807 every 8 connection(s)
I20260709 12:18:01.075831 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:01.082969 23349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:01.084033 23350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:01.086572 23352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:01.086897 10011 server_base.cc:1061] running on GCE node
I20260709 12:18:01.088016 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:01.088214 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:01.088444 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599481088420 us; error 0 us; skew 500 ppm
I20260709 12:18:01.088973 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:01.089175 23345 heartbeater.cc:344] Connected to a master server at 127.9.198.254:43235
I20260709 12:18:01.089665 23345 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:01.090602 23345 heartbeater.cc:507] Master 127.9.198.254:43235 requested a full tablet report, sending...
I20260709 12:18:01.091665 10011 webserver.cc:533] Webserver started at http://127.9.198.194:40875/ using document root <none> and password file <none>
I20260709 12:18:01.092298 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:01.092537 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:01.092870 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:01.092993 23217 ts_manager.cc:194] Registered new tserver with Master: 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807)
I20260709 12:18:01.094427 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
format_stamp: "Formatted at 2026-07-09 12:18:01 on dist-test-slave-glnm"
I20260709 12:18:01.095513 23217 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50864
I20260709 12:18:01.099745 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:18:01.103358 23357 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.104130 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:18:01.104411 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
format_stamp: "Formatted at 2026-07-09 12:18:01 on dist-test-slave-glnm"
I20260709 12:18:01.104660 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:01.123454 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:01.124558 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:01.126075 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:01.128383 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:01.128572 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.128916 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:01.129066 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.169237 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:36153
I20260709 12:18:01.169375 23419 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:36153 every 8 connection(s)
I20260709 12:18:01.174983 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:01.183666 23424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:01.185098 23423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:01.188424 23426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:01.189752 23420 heartbeater.cc:344] Connected to a master server at 127.9.198.254:43235
I20260709 12:18:01.190091 10011 server_base.cc:1061] running on GCE node
I20260709 12:18:01.190256 23420 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:01.191203 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:18:01.191309 23420 heartbeater.cc:507] Master 127.9.198.254:43235 requested a full tablet report, sending...
W20260709 12:18:01.191448 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:01.191758 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599481191736 us; error 0 us; skew 500 ppm
I20260709 12:18:01.192512 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:01.193670 23217 ts_manager.cc:194] Registered new tserver with Master: 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:01.195245 10011 webserver.cc:533] Webserver started at http://127.9.198.195:36105/ using document root <none> and password file <none>
I20260709 12:18:01.195402 23217 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50876
I20260709 12:18:01.195885 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:01.196130 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:01.196460 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:01.197675 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "51a03c362557435daf969e2f7d5ba778"
format_stamp: "Formatted at 2026-07-09 12:18:01 on dist-test-slave-glnm"
I20260709 12:18:01.202318 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:18:01.205698 23431 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.206516 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:18:01.206837 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "51a03c362557435daf969e2f7d5ba778"
format_stamp: "Formatted at 2026-07-09 12:18:01 on dist-test-slave-glnm"
I20260709 12:18:01.207136 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:01.224146 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:01.225307 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:01.226924 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:01.229264 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:01.229507 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.229812 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:01.230001 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:01.270812 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:46541
I20260709 12:18:01.270905 23495 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:46541 every 8 connection(s)
I20260709 12:18:01.285429 23496 heartbeater.cc:344] Connected to a master server at 127.9.198.254:43235
I20260709 12:18:01.285851 23496 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:01.286566 23496 heartbeater.cc:507] Master 127.9.198.254:43235 requested a full tablet report, sending...
I20260709 12:18:01.288432 23217 ts_manager.cc:194] Registered new tserver with Master: 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.289381 10011 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014906168s
I20260709 12:18:01.289788 23217 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50878
I20260709 12:18:01.314630 23217 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50892:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:18:01.317240 23217 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:18:01.394500 23310 tablet_service.cc:1511] Processing CreateTablet for tablet 77d530be1ee246a3aa70928555d7b233 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:01.396140 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d530be1ee246a3aa70928555d7b233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.399116 23307 tablet_service.cc:1511] Processing CreateTablet for tablet 8ebc3148c6444a9191346aaa181695bf (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:01.400743 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ebc3148c6444a9191346aaa181695bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.400846 23383 tablet_service.cc:1511] Processing CreateTablet for tablet 86eb26a3d46147719da1edd9dbd4270b (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:01.401126 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 86eb26a3d46147719da1edd9dbd4270b (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:01.402330 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86eb26a3d46147719da1edd9dbd4270b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.402369 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86eb26a3d46147719da1edd9dbd4270b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.404989 23384 tablet_service.cc:1511] Processing CreateTablet for tablet 703a57dd01894388a102b7bfed51641e (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:01.406471 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 703a57dd01894388a102b7bfed51641e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.406872 23460 tablet_service.cc:1511] Processing CreateTablet for tablet 703a57dd01894388a102b7bfed51641e (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:01.408562 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 703a57dd01894388a102b7bfed51641e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.410708 23385 tablet_service.cc:1511] Processing CreateTablet for tablet 77d530be1ee246a3aa70928555d7b233 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:01.412096 23385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d530be1ee246a3aa70928555d7b233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.412375 23461 tablet_service.cc:1511] Processing CreateTablet for tablet 77d530be1ee246a3aa70928555d7b233 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:01.410519 23459 tablet_service.cc:1511] Processing CreateTablet for tablet 86eb26a3d46147719da1edd9dbd4270b (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:01.398942 23309 tablet_service.cc:1511] Processing CreateTablet for tablet 703a57dd01894388a102b7bfed51641e (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:01.418536 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 703a57dd01894388a102b7bfed51641e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.421991 23461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d530be1ee246a3aa70928555d7b233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.425565 23382 tablet_service.cc:1511] Processing CreateTablet for tablet 8ebc3148c6444a9191346aaa181695bf (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:01.427241 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ebc3148c6444a9191346aaa181695bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.430609 23306 tablet_service.cc:1511] Processing CreateTablet for tablet 92b3b3a4266944f79320f1f2a4f1a0b4 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:01.431962 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86eb26a3d46147719da1edd9dbd4270b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.432349 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92b3b3a4266944f79320f1f2a4f1a0b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.434751 23381 tablet_service.cc:1511] Processing CreateTablet for tablet 92b3b3a4266944f79320f1f2a4f1a0b4 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:01.436064 23511 tablet_bootstrap.cc:492] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:01.436429 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92b3b3a4266944f79320f1f2a4f1a0b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.443426 23458 tablet_service.cc:1511] Processing CreateTablet for tablet 8ebc3148c6444a9191346aaa181695bf (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:01.445070 23458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ebc3148c6444a9191346aaa181695bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.447510 23457 tablet_service.cc:1511] Processing CreateTablet for tablet 92b3b3a4266944f79320f1f2a4f1a0b4 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:01.448552 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 210c6a9c1c5d486f8ace5a042a52bcca (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:01.449205 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92b3b3a4266944f79320f1f2a4f1a0b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.450139 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210c6a9c1c5d486f8ace5a042a52bcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.450876 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 210c6a9c1c5d486f8ace5a042a52bcca (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:01.452441 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210c6a9c1c5d486f8ace5a042a52bcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.454523 23307 tablet_service.cc:1511] Processing CreateTablet for tablet be0f2e3c754b43ed89096070a9d076a0 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:01.455492 23455 tablet_service.cc:1511] Processing CreateTablet for tablet 210c6a9c1c5d486f8ace5a042a52bcca (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:01.455993 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0f2e3c754b43ed89096070a9d076a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.457216 23511 tablet_bootstrap.cc:654] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.464241 23379 tablet_service.cc:1511] Processing CreateTablet for tablet be0f2e3c754b43ed89096070a9d076a0 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:01.465529 23455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210c6a9c1c5d486f8ace5a042a52bcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.465821 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0f2e3c754b43ed89096070a9d076a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.482951 23454 tablet_service.cc:1511] Processing CreateTablet for tablet be0f2e3c754b43ed89096070a9d076a0 (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:01.485172 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0f2e3c754b43ed89096070a9d076a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.498158 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 4eaaadad8ef849f5b67928865a4f97ec (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:01.500310 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eaaadad8ef849f5b67928865a4f97ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.506893 23512 tablet_bootstrap.cc:492] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:01.513530 23512 tablet_bootstrap.cc:654] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.513706 23383 tablet_service.cc:1511] Processing CreateTablet for tablet 4eaaadad8ef849f5b67928865a4f97ec (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:01.515161 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eaaadad8ef849f5b67928865a4f97ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.522126 23512 tablet_bootstrap.cc:492] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:01.522773 23512 ts_tablet_manager.cc:1403] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.009s
I20260709 12:18:01.525925 23512 raft_consensus.cc:359] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.526749 23512 raft_consensus.cc:385] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.527069 23512 raft_consensus.cc:740] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.529349 23453 tablet_service.cc:1511] Processing CreateTablet for tablet 4eaaadad8ef849f5b67928865a4f97ec (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:01.530845 23453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eaaadad8ef849f5b67928865a4f97ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.530704 23309 tablet_service.cc:1511] Processing CreateTablet for tablet a7194c62978a4c2bba240ca41873449a (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:01.532315 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7194c62978a4c2bba240ca41873449a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.546454 23511 tablet_bootstrap.cc:492] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:01.547083 23511 ts_tablet_manager.cc:1403] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.111s	user 0.007s	sys 0.004s
I20260709 12:18:01.550299 23511 raft_consensus.cc:359] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.551039 23511 raft_consensus.cc:385] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.551348 23511 raft_consensus.cc:740] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.552099 23384 tablet_service.cc:1511] Processing CreateTablet for tablet a7194c62978a4c2bba240ca41873449a (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:01.553520 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7194c62978a4c2bba240ca41873449a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.552145 23511 consensus_queue.cc:260] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.602074 23511 ts_tablet_manager.cc:1434] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.055s	user 0.017s	sys 0.001s
I20260709 12:18:01.602926 23345 heartbeater.cc:499] Master 127.9.198.254:43235 was elected leader, sending a full tablet report...
I20260709 12:18:01.604774 23512 consensus_queue.cc:260] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.608161 23452 tablet_service.cc:1511] Processing CreateTablet for tablet a7194c62978a4c2bba240ca41873449a (DEFAULT_TABLE table=test-workload [id=5645df3f9af84d75aff5d471a4c1f275]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:01.605048 23511 tablet_bootstrap.cc:492] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:01.609723 23516 raft_consensus.cc:493] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.633244 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7194c62978a4c2bba240ca41873449a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:01.634502 23516 raft_consensus.cc:515] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.636698 23511 tablet_bootstrap.cc:654] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.647728 23518 tablet_bootstrap.cc:492] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:01.663687 23420 heartbeater.cc:499] Master 127.9.198.254:43235 was elected leader, sending a full tablet report...
I20260709 12:18:01.679054 23516 leader_election.cc:290] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.684130 23512 ts_tablet_manager.cc:1434] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.161s	user 0.027s	sys 0.007s
I20260709 12:18:01.684794 23511 tablet_bootstrap.cc:492] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:01.686108 23512 tablet_bootstrap.cc:492] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:01.687268 23511 ts_tablet_manager.cc:1403] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.017s
I20260709 12:18:01.695715 23512 tablet_bootstrap.cc:654] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.699020 23518 tablet_bootstrap.cc:654] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.695449 23511 raft_consensus.cc:359] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.701745 23511 raft_consensus.cc:385] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.702332 23511 raft_consensus.cc:740] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.703740 23511 consensus_queue.cc:260] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.708695 23511 ts_tablet_manager.cc:1434] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:18:01.710587 23511 tablet_bootstrap.cc:492] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:01.717190 23512 tablet_bootstrap.cc:492] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:01.717856 23512 ts_tablet_manager.cc:1403] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.014s
I20260709 12:18:01.720039 23511 tablet_bootstrap.cc:654] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.720220 23512 raft_consensus.cc:359] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.721037 23512 raft_consensus.cc:385] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.721326 23512 raft_consensus.cc:740] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.722117 23512 consensus_queue.cc:260] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.722502 23517 raft_consensus.cc:493] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.723286 23517 raft_consensus.cc:515] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.727008 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
W20260709 12:18:01.729210 23283 leader_election.cc:343] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.734028 23517 leader_election.cc:290] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.753468 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:01.760277 23512 ts_tablet_manager.cc:1434] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.019s
I20260709 12:18:01.762152 23512 tablet_bootstrap.cc:492] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
W20260709 12:18:01.755674 23284 leader_election.cc:343] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.766111 23284 leader_election.cc:304] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb; no voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:01.767545 23516 raft_consensus.cc:2749] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:01.778136 23516 raft_consensus.cc:493] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.778744 23512 tablet_bootstrap.cc:654] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.778661 23516 raft_consensus.cc:515] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.781044 23516 leader_election.cc:290] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.783692 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86eb26a3d46147719da1edd9dbd4270b" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:01.783887 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86eb26a3d46147719da1edd9dbd4270b" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
W20260709 12:18:01.797240 23283 leader_election.cc:343] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.793231 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86eb26a3d46147719da1edd9dbd4270b" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
W20260709 12:18:01.811890 23358 leader_election.cc:343] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.794399 23395 raft_consensus.cc:2468] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:01.813987 23284 leader_election.cc:304] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:01.815263 23516 raft_consensus.cc:2804] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:01.815680 23516 raft_consensus.cc:493] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.816039 23516 raft_consensus.cc:3060] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:01.818585 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86eb26a3d46147719da1edd9dbd4270b" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:01.820686 23512 tablet_bootstrap.cc:492] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:01.820708 23511 tablet_bootstrap.cc:492] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:01.821532 23512 ts_tablet_manager.cc:1403] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.060s	user 0.023s	sys 0.019s
I20260709 12:18:01.821532 23511 ts_tablet_manager.cc:1403] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.111s	user 0.019s	sys 0.049s
I20260709 12:18:01.823215 23518 tablet_bootstrap.cc:492] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:01.823814 23518 ts_tablet_manager.cc:1403] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.177s	user 0.028s	sys 0.041s
I20260709 12:18:01.825153 23512 raft_consensus.cc:359] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.825448 23511 raft_consensus.cc:359] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.824402 23516 raft_consensus.cc:515] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.826085 23512 raft_consensus.cc:385] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.826295 23511 raft_consensus.cc:385] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.826658 23512 raft_consensus.cc:740] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.826704 23511 raft_consensus.cc:740] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.827018 23320 raft_consensus.cc:2393] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in current term 1: Already voted for candidate 4b6e2a23c11644ec92cc018d8be502eb in this term.
I20260709 12:18:01.827297 23518 raft_consensus.cc:359] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.828186 23518 raft_consensus.cc:385] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.828608 23518 raft_consensus.cc:740] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.827986 23512 consensus_queue.cc:260] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.829610 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86eb26a3d46147719da1edd9dbd4270b" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:01.829497 23518 consensus_queue.cc:260] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.830385 23395 raft_consensus.cc:3060] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:01.830978 23512 ts_tablet_manager.cc:1434] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:18:01.832221 23512 tablet_bootstrap.cc:492] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:01.839138 23512 tablet_bootstrap.cc:654] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.829416 23358 leader_election.cc:304] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:01.843904 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86eb26a3d46147719da1edd9dbd4270b" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
W20260709 12:18:01.845997 23283 leader_election.cc:343] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.846827 23516 leader_election.cc:290] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.847766 23395 raft_consensus.cc:2468] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:01.848757 23523 raft_consensus.cc:2749] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:01.849589 23511 consensus_queue.cc:260] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.853193 23511 ts_tablet_manager.cc:1434] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:18:01.855489 23496 heartbeater.cc:499] Master 127.9.198.254:43235 was elected leader, sending a full tablet report...
I20260709 12:18:01.855546 23529 raft_consensus.cc:493] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.856418 23529 raft_consensus.cc:515] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.858566 23512 tablet_bootstrap.cc:492] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:01.859200 23512 ts_tablet_manager.cc:1403] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.000s
I20260709 12:18:01.860596 23518 ts_tablet_manager.cc:1434] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.036s	user 0.030s	sys 0.000s
I20260709 12:18:01.862483 23518 tablet_bootstrap.cc:492] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:01.867308 23529 leader_election.cc:290] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:01.868995 23518 tablet_bootstrap.cc:654] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.870437 23511 tablet_bootstrap.cc:492] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:01.870932 23512 raft_consensus.cc:359] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.871852 23512 raft_consensus.cc:385] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.872202 23512 raft_consensus.cc:740] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
W20260709 12:18:01.875938 23265 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:01.879293 23284 leader_election.cc:304] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:01.880718 23516 raft_consensus.cc:2804] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:01.881209 23516 raft_consensus.cc:697] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:01.882135 23516 consensus_queue.cc:237] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.884320 23511 tablet_bootstrap.cc:654] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.887982 23512 consensus_queue.cc:260] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.890818 23518 tablet_bootstrap.cc:492] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:01.891418 23518 ts_tablet_manager.cc:1403] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.001s
I20260709 12:18:01.896801 23512 ts_tablet_manager.cc:1434] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.003s
I20260709 12:18:01.898038 23512 tablet_bootstrap.cc:492] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:01.902549 23518 raft_consensus.cc:359] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.903359 23518 raft_consensus.cc:385] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.903663 23518 raft_consensus.cc:740] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.905005 23512 tablet_bootstrap.cc:654] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.904537 23518 consensus_queue.cc:260] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.906688 23523 raft_consensus.cc:493] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.907503 23518 ts_tablet_manager.cc:1434] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 12:18:01.907253 23523 raft_consensus.cc:515] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.910063 23523 leader_election.cc:290] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.911538 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:01.912710 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:01.913153 23518 tablet_bootstrap.cc:492] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
W20260709 12:18:01.913340 23432 leader_election.cc:343] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:18:01.914448 23358 leader_election.cc:343] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.918053 23512 tablet_bootstrap.cc:492] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:01.918659 23512 ts_tablet_manager.cc:1403] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 12:18:01.920246 23518 tablet_bootstrap.cc:654] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.921833 23512 raft_consensus.cc:359] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.922338 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:01.922644 23512 raft_consensus.cc:385] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.923063 23512 raft_consensus.cc:740] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.923164 23471 raft_consensus.cc:2468] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:01.924679 23358 leader_election.cc:304] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:01.924345 23512 consensus_queue.cc:260] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.926193 23523 raft_consensus.cc:2804] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:01.926671 23523 raft_consensus.cc:493] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.927049 23523 raft_consensus.cc:3060] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:01.927611 23512 ts_tablet_manager.cc:1434] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:18:01.930719 23511 tablet_bootstrap.cc:492] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:01.931303 23511 ts_tablet_manager.cc:1403] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.003s
I20260709 12:18:01.933431 23512 tablet_bootstrap.cc:492] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:01.934584 23511 raft_consensus.cc:359] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.935411 23511 raft_consensus.cc:385] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.935746 23511 raft_consensus.cc:740] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.934969 23215 catalog_manager.cc:5697] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 1 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:01.936486 23511 consensus_queue.cc:260] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.936839 23517 raft_consensus.cc:493] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.937426 23517 raft_consensus.cc:515] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.939646 23517 leader_election.cc:290] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.940886 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:01.941855 23523 raft_consensus.cc:515] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
W20260709 12:18:01.943032 23358 leader_election.cc:343] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.944410 23512 tablet_bootstrap.cc:654] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.944604 23523 leader_election.cc:290] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:01.946396 23511 ts_tablet_manager.cc:1434] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:18:01.946540 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:01.947595 23511 tablet_bootstrap.cc:492] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:01.948005 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:01.948478 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:01.948760 23470 raft_consensus.cc:3060] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:01.947589 23395 raft_consensus.cc:2393] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51a03c362557435daf969e2f7d5ba778 in current term 1: Already voted for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in this term.
I20260709 12:18:01.954622 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
W20260709 12:18:01.956627 23358 leader_election.cc:343] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.956997 23518 tablet_bootstrap.cc:492] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:01.957907 23518 ts_tablet_manager.cc:1403] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.000s
I20260709 12:18:01.958948 23470 raft_consensus.cc:2468] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
W20260709 12:18:01.960214 23358 leader_election.cc:343] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:01.960294 23433 leader_election.cc:304] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:01.961095 23358 leader_election.cc:304] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:01.961561 23529 raft_consensus.cc:2749] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:01.962396 23523 raft_consensus.cc:2749] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:01.962625 23518 raft_consensus.cc:359] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.963680 23358 leader_election.cc:304] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:01.965257 23523 raft_consensus.cc:2804] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:01.963706 23518 raft_consensus.cc:385] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:01.966632 23511 tablet_bootstrap.cc:654] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:01.966420 23523 raft_consensus.cc:697] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:01.967772 23523 consensus_queue.cc:237] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.966540 23518 raft_consensus.cc:740] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:01.971441 23518 consensus_queue.cc:260] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.978753 23518 ts_tablet_manager.cc:1434] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.001s
I20260709 12:18:01.980423 23518 tablet_bootstrap.cc:492] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:01.986318 23529 raft_consensus.cc:493] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.987277 23529 raft_consensus.cc:515] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:01.991350 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:01.991559 23214 catalog_manager.cc:5697] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:01.994161 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:01.994412 23319 raft_consensus.cc:2468] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:01.994961 23395 raft_consensus.cc:2468] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:01.996416 23432 leader_election.cc:304] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:01.997083 23529 leader_election.cc:290] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:01.998087 23530 raft_consensus.cc:2804] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:01.998719 23530 raft_consensus.cc:493] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.999076 23530 raft_consensus.cc:3060] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.001040 23518 tablet_bootstrap.cc:654] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.010577 23512 tablet_bootstrap.cc:492] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:02.011117 23512 ts_tablet_manager.cc:1403] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.078s	user 0.034s	sys 0.023s
I20260709 12:18:02.014771 23512 raft_consensus.cc:359] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.015799 23512 raft_consensus.cc:385] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.016180 23512 raft_consensus.cc:740] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.017109 23530 raft_consensus.cc:515] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.020547 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:02.020788 23518 tablet_bootstrap.cc:492] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:02.021296 23319 raft_consensus.cc:3060] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.021782 23511 tablet_bootstrap.cc:492] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:02.022125 23518 ts_tablet_manager.cc:1403] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.013s
I20260709 12:18:02.022363 23511 ts_tablet_manager.cc:1403] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.075s	user 0.017s	sys 0.006s
I20260709 12:18:02.026051 23511 raft_consensus.cc:359] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.027401 23518 raft_consensus.cc:359] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.026839 23511 raft_consensus.cc:385] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.028802 23518 raft_consensus.cc:385] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.028829 23511 raft_consensus.cc:740] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.029757 23518 raft_consensus.cc:740] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.030411 23511 consensus_queue.cc:260] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.031075 23518 consensus_queue.cc:260] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.032446 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:02.033174 23511 ts_tablet_manager.cc:1434] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:18:02.033591 23395 raft_consensus.cc:3060] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.034407 23511 tablet_bootstrap.cc:492] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:02.035887 23518 ts_tablet_manager.cc:1434] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:18:02.037307 23518 tablet_bootstrap.cc:492] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:02.033206 23530 leader_election.cc:290] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:02.042047 23511 tablet_bootstrap.cc:654] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.043088 23395 raft_consensus.cc:2468] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:02.044595 23433 leader_election.cc:304] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:02.045722 23518 tablet_bootstrap.cc:654] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.046981 23530 raft_consensus.cc:2804] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:02.047775 23530 raft_consensus.cc:697] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:02.049010 23530 consensus_queue.cc:237] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.030465 23319 raft_consensus.cc:2468] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:02.056500 23511 tablet_bootstrap.cc:492] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:02.055949 23512 consensus_queue.cc:260] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.057291 23511 ts_tablet_manager.cc:1403] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 12:18:02.060299 23512 ts_tablet_manager.cc:1434] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.001s
I20260709 12:18:02.063223 23511 raft_consensus.cc:359] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.064018 23511 raft_consensus.cc:385] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.064332 23511 raft_consensus.cc:740] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.064428 23518 tablet_bootstrap.cc:492] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:02.065048 23518 ts_tablet_manager.cc:1403] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 12:18:02.065156 23511 consensus_queue.cc:260] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.066311 23512 tablet_bootstrap.cc:492] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:02.068288 23518 raft_consensus.cc:359] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.069092 23518 raft_consensus.cc:385] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.069393 23518 raft_consensus.cc:740] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.070073 23511 ts_tablet_manager.cc:1434] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:18:02.070235 23518 consensus_queue.cc:260] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.071374 23511 tablet_bootstrap.cc:492] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:02.072904 23518 ts_tablet_manager.cc:1434] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:18:02.074059 23518 tablet_bootstrap.cc:492] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:02.076777 23512 tablet_bootstrap.cc:654] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.078742 23511 tablet_bootstrap.cc:654] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.084493 23518 tablet_bootstrap.cc:654] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.087033 23217 catalog_manager.cc:5697] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:02.091262 23512 tablet_bootstrap.cc:492] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:02.091741 23511 tablet_bootstrap.cc:492] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:02.091850 23512 ts_tablet_manager.cc:1403] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.004s
I20260709 12:18:02.092625 23511 ts_tablet_manager.cc:1403] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 12:18:02.095489 23512 raft_consensus.cc:359] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.096513 23512 raft_consensus.cc:385] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.096956 23512 raft_consensus.cc:740] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.096946 23511 raft_consensus.cc:359] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.098294 23511 raft_consensus.cc:385] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.098713 23511 raft_consensus.cc:740] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.098425 23512 consensus_queue.cc:260] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.099807 23511 consensus_queue.cc:260] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.103344 23511 ts_tablet_manager.cc:1434] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:18:02.104803 23511 tablet_bootstrap.cc:492] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:02.106990 23516 raft_consensus.cc:493] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.107767 23516 raft_consensus.cc:515] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.112676 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d530be1ee246a3aa70928555d7b233" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:02.113742 23470 raft_consensus.cc:2468] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:02.116691 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d530be1ee246a3aa70928555d7b233" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:02.114096 23512 ts_tablet_manager.cc:1434] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:18:02.118937 23512 tablet_bootstrap.cc:492] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:02.117777 23395 raft_consensus.cc:2468] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:02.118815 23283 leader_election.cc:304] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.119791 23516 leader_election.cc:290] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.122514 23516 raft_consensus.cc:2804] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:02.122970 23516 raft_consensus.cc:493] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.123337 23516 raft_consensus.cc:3060] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.128454 23511 tablet_bootstrap.cc:654] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.128508 23512 tablet_bootstrap.cc:654] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.131516 23516 raft_consensus.cc:515] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.136008 23518 tablet_bootstrap.cc:492] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:02.136837 23518 ts_tablet_manager.cc:1403] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.028s
I20260709 12:18:02.141074 23518 raft_consensus.cc:359] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.142118 23518 raft_consensus.cc:385] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.142503 23518 raft_consensus.cc:740] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.145186 23516 leader_election.cc:290] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.145308 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d530be1ee246a3aa70928555d7b233" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:02.146028 23395 raft_consensus.cc:3060] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.146258 23518 consensus_queue.cc:260] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.149142 23518 ts_tablet_manager.cc:1434] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.001s
I20260709 12:18:02.150276 23518 tablet_bootstrap.cc:492] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:02.151012 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d530be1ee246a3aa70928555d7b233" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:02.151696 23470 raft_consensus.cc:3060] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.157740 23518 tablet_bootstrap.cc:654] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.182910 23395 raft_consensus.cc:2468] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:02.184653 23284 leader_election.cc:304] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:02.181314 23470 raft_consensus.cc:2468] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:02.187482 23516 raft_consensus.cc:2804] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:02.188252 23516 raft_consensus.cc:697] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:02.189594 23516 consensus_queue.cc:237] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.193405 23518 tablet_bootstrap.cc:492] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:02.194128 23518 ts_tablet_manager.cc:1403] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.020s
I20260709 12:18:02.196841 23511 tablet_bootstrap.cc:492] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:02.197438 23511 ts_tablet_manager.cc:1403] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.015s
I20260709 12:18:02.198244 23518 raft_consensus.cc:359] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.199222 23518 raft_consensus.cc:385] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.199610 23518 raft_consensus.cc:740] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.200140 23512 tablet_bootstrap.cc:492] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:02.200737 23512 ts_tablet_manager.cc:1403] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.057s
I20260709 12:18:02.200537 23511 raft_consensus.cc:359] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.200654 23518 consensus_queue.cc:260] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.201593 23511 raft_consensus.cc:385] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.202045 23511 raft_consensus.cc:740] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.201763 23214 catalog_manager.cc:5697] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 1 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:02.202862 23511 consensus_queue.cc:260] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.203881 23518 ts_tablet_manager.cc:1434] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:18:02.205097 23511 ts_tablet_manager.cc:1434] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:18:02.205127 23518 tablet_bootstrap.cc:492] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:02.206569 23511 tablet_bootstrap.cc:492] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:02.207052 23512 raft_consensus.cc:359] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.207851 23512 raft_consensus.cc:385] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.208168 23512 raft_consensus.cc:740] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.209054 23512 consensus_queue.cc:260] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.211627 23512 ts_tablet_manager.cc:1434] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:18:02.213165 23512 tablet_bootstrap.cc:492] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:02.213322 23511 tablet_bootstrap.cc:654] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.214078 23518 tablet_bootstrap.cc:654] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.217943 23530 raft_consensus.cc:493] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.218685 23530 raft_consensus.cc:515] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.222715 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:02.223675 23319 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:02.225473 23432 leader_election.cc:304] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.226794 23529 raft_consensus.cc:2804] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:02.227300 23529 raft_consensus.cc:493] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.227741 23529 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.237450 23530 leader_election.cc:290] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:02.239979 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:02.240770 23395 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:02.244109 23529 raft_consensus.cc:515] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.252651 23512 tablet_bootstrap.cc:654] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.256178 23516 raft_consensus.cc:493] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.256887 23516 raft_consensus.cc:515] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.259990 23516 leader_election.cc:290] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.260023 23529 leader_election.cc:290] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:02.260978 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:02.261297 23511 tablet_bootstrap.cc:492] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:02.261766 23470 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:02.262090 23511 ts_tablet_manager.cc:1403] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.056s	user 0.028s	sys 0.012s
I20260709 12:18:02.263648 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:02.263875 23512 tablet_bootstrap.cc:492] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:02.263773 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:02.264289 23395 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.264524 23512 ts_tablet_manager.cc:1403] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.003s
I20260709 12:18:02.264662 23319 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.265244 23283 leader_election.cc:304] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.266183 23516 raft_consensus.cc:2804] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:02.266731 23516 raft_consensus.cc:493] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.266177 23511 raft_consensus.cc:359] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.267144 23516 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.267515 23511 raft_consensus.cc:385] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.267318 23512 raft_consensus.cc:359] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.268000 23511 raft_consensus.cc:740] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.268131 23512 raft_consensus.cc:385] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.268729 23512 raft_consensus.cc:740] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.270282 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:02.269532 23512 consensus_queue.cc:260] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.269343 23511 consensus_queue.cc:260] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.271101 23394 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:02.273379 23515 raft_consensus.cc:493] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.273898 23515 raft_consensus.cc:515] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.275020 23518 tablet_bootstrap.cc:492] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:02.275568 23518 ts_tablet_manager.cc:1403] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.036s
I20260709 12:18:02.278386 23518 raft_consensus.cc:359] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.279079 23518 raft_consensus.cc:385] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.279340 23518 raft_consensus.cc:740] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.280081 23518 consensus_queue.cc:260] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.281998 23511 ts_tablet_manager.cc:1434] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:18:02.283977 23518 ts_tablet_manager.cc:1434] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:18:02.283710 23512 ts_tablet_manager.cc:1434] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:18:02.283694 23319 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:02.285478 23518 tablet_bootstrap.cc:492] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:02.289142 23432 leader_election.cc:304] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.292294 23518 tablet_bootstrap.cc:654] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:02.292902 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:02.293763 23393 raft_consensus.cc:2468] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:02.294667 23529 raft_consensus.cc:2804] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:02.295246 23529 raft_consensus.cc:697] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:02.296447 23529 consensus_queue.cc:237] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.304872 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:02.305876 23470 raft_consensus.cc:2468] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:02.307960 23283 leader_election.cc:304] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.285563 23515 leader_election.cc:290] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.312554 23515 raft_consensus.cc:2804] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:02.312925 23515 raft_consensus.cc:493] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.313242 23515 raft_consensus.cc:3060] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.328722 23560 consensus_queue.cc:1048] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:02.331377 23395 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:02.342356 23214 catalog_manager.cc:5697] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:02.314927 23516 raft_consensus.cc:515] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.351524 23515 raft_consensus.cc:515] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.355225 23515 leader_election.cc:290] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
W20260709 12:18:02.358098 23283 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.198.195:46541, user_credentials={real_user=slave}} blocked reactor thread for 50300.4us
I20260709 12:18:02.360270 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:02.360592 23516 leader_election.cc:290] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.361089 23470 raft_consensus.cc:3060] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.363566 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:02.364374 23471 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.365871 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:02.366653 23395 raft_consensus.cc:3060] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.369165 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:02.369918 23393 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.375998 23470 raft_consensus.cc:2468] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:02.378881 23393 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:02.380273 23284 leader_election.cc:304] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:02.382546 23563 raft_consensus.cc:2804] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:02.383594 23518 tablet_bootstrap.cc:492] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:02.383522 23563 raft_consensus.cc:697] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:02.384229 23518 ts_tablet_manager.cc:1403] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.099s	user 0.035s	sys 0.052s
I20260709 12:18:02.384874 23563 consensus_queue.cc:237] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.386937 23471 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:02.387517 23518 raft_consensus.cc:359] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.388301 23518 raft_consensus.cc:385] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:02.388545 23518 raft_consensus.cc:740] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:02.389298 23518 consensus_queue.cc:260] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.391685 23518 ts_tablet_manager.cc:1434] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:18:02.392282 23283 leader_election.cc:304] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.393456 23515 raft_consensus.cc:2804] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:02.394510 23395 raft_consensus.cc:2468] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:02.395155 23515 raft_consensus.cc:697] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:02.397418 23515 consensus_queue.cc:237] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.418890 23471 raft_consensus.cc:3060] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.420710 23214 catalog_manager.cc:5697] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 1 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:02.433290 23529 consensus_queue.cc:1048] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.435497 23529 consensus_queue.cc:1048] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.435818 23563 consensus_queue.cc:1048] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.445757 23523 raft_consensus.cc:493] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.446749 23523 raft_consensus.cc:515] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.450685 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:02.451634 23320 raft_consensus.cc:2468] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:02.453522 23358 leader_election.cc:304] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:02.455667 23517 raft_consensus.cc:2804] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:02.456321 23517 raft_consensus.cc:493] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.456357 23523 leader_election.cc:290] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.456954 23517 raft_consensus.cc:3060] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.459046 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:02.460099 23470 raft_consensus.cc:2468] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:02.460228 23215 catalog_manager.cc:5697] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 1 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:02.454892 23530 raft_consensus.cc:493] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.466205 23530 raft_consensus.cc:515] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.469667 23530 leader_election.cc:290] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:02.472452 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:02.473459 23320 raft_consensus.cc:2468] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:02.475301 23432 leader_election.cc:304] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.477917 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:02.480806 23530 raft_consensus.cc:2804] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:02.481760 23530 raft_consensus.cc:493] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.482234 23530 raft_consensus.cc:3060] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.479444 23394 raft_consensus.cc:2468] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:02.516577 23523 consensus_queue.cc:1048] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:02.524147 23517 raft_consensus.cc:515] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.527562 23517 leader_election.cc:290] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.528807 23530 raft_consensus.cc:515] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.529186 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:02.530015 23319 raft_consensus.cc:3060] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.532284 23530 leader_election.cc:290] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:02.532183 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:02.533093 23320 raft_consensus.cc:3060] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.535508 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:02.536257 23394 raft_consensus.cc:3060] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.537497 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:02.538430 23471 raft_consensus.cc:3060] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.542026 23320 raft_consensus.cc:2468] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:02.543609 23432 leader_election.cc:304] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:02.544764 23530 raft_consensus.cc:2804] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:02.545511 23530 raft_consensus.cc:697] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:02.546495 23530 consensus_queue.cc:237] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.547075 23471 raft_consensus.cc:2468] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:02.549422 23319 raft_consensus.cc:2468] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:02.550369 23358 leader_election.cc:304] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:02.553346 23517 raft_consensus.cc:2804] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:02.551487 23394 raft_consensus.cc:2468] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:02.558385 23517 raft_consensus.cc:697] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:02.560236 23563 raft_consensus.cc:493] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.560830 23563 raft_consensus.cc:515] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.562336 23215 catalog_manager.cc:5697] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:02.563215 23563 leader_election.cc:290] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.564931 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:02.567531 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:02.568485 23319 raft_consensus.cc:3060] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:02.569705 23284 leader_election.cc:304] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb; no voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:02.559684 23517 consensus_queue.cc:237] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.580917 23563 raft_consensus.cc:2749] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:02.582183 23578 consensus_queue.cc:1048] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:02.592360 23216 catalog_manager.cc:5697] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:02.638995 23563 consensus_queue.cc:1048] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.646549 23560 consensus_queue.cc:1048] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.784324 23560 consensus_queue.cc:1048] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.796963 23515 consensus_queue.cc:1048] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:02.811120 23530 consensus_queue.cc:1048] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.823714 23574 consensus_queue.cc:1048] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.857287 23563 consensus_queue.cc:1048] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.869760 23560 consensus_queue.cc:1048] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.873268 23395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "703a57dd01894388a102b7bfed51641e"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
mode: GRACEFUL
new_leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:42182
I20260709 12:18:02.873932 23395 raft_consensus.cc:606] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Received request to transfer leadership to TS 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:02.895925 23470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
mode: GRACEFUL
new_leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:49664
I20260709 12:18:02.896577 23470 raft_consensus.cc:606] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Received request to transfer leadership to TS 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:02.910286 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
mode: GRACEFUL
new_leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:53086
I20260709 12:18:02.910841 23320 raft_consensus.cc:606] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Received request to transfer leadership to TS 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:02.915987 23470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
mode: GRACEFUL
new_leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:49664
I20260709 12:18:02.916553 23470 raft_consensus.cc:606] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Received request to transfer leadership to TS 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:02.921769 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
mode: GRACEFUL
new_leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:53086
I20260709 12:18:02.922299 23320 raft_consensus.cc:606] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Received request to transfer leadership to TS 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:02.927714 23470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
mode: GRACEFUL
new_leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:49664
I20260709 12:18:02.928609 23470 raft_consensus.cc:606] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Received request to transfer leadership to TS 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:02.931392 23529 consensus_queue.cc:1048] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:02.934633 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
mode: GRACEFUL
new_leader_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:42182
I20260709 12:18:02.935228 23394 raft_consensus.cc:606] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Received request to transfer leadership to TS 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:02.943804 23573 consensus_queue.cc:1048] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:02.957059 23574 raft_consensus.cc:993] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778: : Instructing follower 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:02.957715 23573 raft_consensus.cc:1081] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Signalling peer 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:02.959359 23394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:42172
I20260709 12:18:02.959880 23394 raft_consensus.cc:493] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:02.960271 23394 raft_consensus.cc:3060] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:02.965816 23394 raft_consensus.cc:515] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.968027 23394 leader_election.cc:290] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:02.969027 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:02.969564 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eaaadad8ef849f5b67928865a4f97ec" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:02.969929 23320 raft_consensus.cc:3060] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:02.970304 23470 raft_consensus.cc:3055] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:02.970690 23470 raft_consensus.cc:740] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:02.971585 23470 consensus_queue.cc:260] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.973006 23470 raft_consensus.cc:3060] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:02.977370 23320 raft_consensus.cc:2468] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:02.978727 23358 leader_election.cc:304] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:02.979077 23470 raft_consensus.cc:2468] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:02.979544 23517 raft_consensus.cc:2804] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:02.980125 23517 raft_consensus.cc:697] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:02.981004 23517 consensus_queue.cc:237] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:02.987495 23214 catalog_manager.cc:5697] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 1 to 2, leader changed from 51a03c362557435daf969e2f7d5ba778 (127.9.198.195) to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:03.046931 23574 raft_consensus.cc:993] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778: : Instructing follower 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:03.047451 23573 raft_consensus.cc:1081] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Signalling peer 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:03.049075 23394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:42172
I20260709 12:18:03.049744 23394 raft_consensus.cc:493] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:03.050079 23394 raft_consensus.cc:3060] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.055433 23394 raft_consensus.cc:515] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.057054 23394 leader_election.cc:290] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:03.058084 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:03.058251 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c6a9c1c5d486f8ace5a042a52bcca" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:03.058775 23320 raft_consensus.cc:3060] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.058883 23470 raft_consensus.cc:3055] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:03.059368 23470 raft_consensus.cc:740] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:03.060124 23470 consensus_queue.cc:260] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.061213 23470 raft_consensus.cc:3060] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.065155 23320 raft_consensus.cc:2468] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:03.066478 23358 leader_election.cc:304] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:03.066741 23470 raft_consensus.cc:2468] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:03.067373 23517 raft_consensus.cc:2804] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:03.067869 23517 raft_consensus.cc:697] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:03.068707 23517 consensus_queue.cc:237] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.075584 23214 catalog_manager.cc:5697] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 1 to 2, leader changed from 51a03c362557435daf969e2f7d5ba778 (127.9.198.195) to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:03.088697 23580 raft_consensus.cc:993] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93: : Instructing follower 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:03.089160 23517 raft_consensus.cc:1081] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Signalling peer 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:03.090969 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "703a57dd01894388a102b7bfed51641e"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:53066
I20260709 12:18:03.091599 23320 raft_consensus.cc:493] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:03.091989 23320 raft_consensus.cc:3060] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.097430 23320 raft_consensus.cc:515] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.099180 23320 leader_election.cc:290] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:03.100093 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:03.100418 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "703a57dd01894388a102b7bfed51641e" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:03.100811 23394 raft_consensus.cc:3055] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:03.101011 23470 raft_consensus.cc:3060] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.101195 23394 raft_consensus.cc:740] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:03.102175 23394 consensus_queue.cc:260] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.103479 23394 raft_consensus.cc:3060] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.108834 23470 raft_consensus.cc:2468] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:03.110230 23283 leader_election.cc:304] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:03.110658 23394 raft_consensus.cc:2468] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:03.111126 23560 raft_consensus.cc:2804] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:03.111652 23560 raft_consensus.cc:697] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:03.112795 23560 consensus_queue.cc:237] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.116199 23517 consensus_queue.cc:1048] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:03.120505 23214 catalog_manager.cc:5697] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 1 to 2, leader changed from 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194) to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:03.124326 23580 raft_consensus.cc:993] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93: : Instructing follower 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:03.125119 23579 raft_consensus.cc:1081] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Signalling peer 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:03.133760 23517 consensus_queue.cc:1048] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:03.134181 23470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:49656
I20260709 12:18:03.134950 23470 raft_consensus.cc:493] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:03.135346 23470 raft_consensus.cc:3060] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.141973 23470 raft_consensus.cc:515] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.144063 23470 leader_election.cc:290] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:03.145020 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:03.145154 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7194c62978a4c2bba240ca41873449a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:03.145861 23320 raft_consensus.cc:3060] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.145972 23394 raft_consensus.cc:3055] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:03.146562 23394 raft_consensus.cc:740] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:03.147379 23394 consensus_queue.cc:260] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.148372 23394 raft_consensus.cc:3060] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.152212 23320 raft_consensus.cc:2468] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 2.
I20260709 12:18:03.153607 23432 leader_election.cc:304] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:03.154564 23394 raft_consensus.cc:2468] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 2.
I20260709 12:18:03.154654 23573 raft_consensus.cc:2804] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:03.155390 23573 raft_consensus.cc:697] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 2 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:03.156353 23573 consensus_queue.cc:237] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.164124 23214 catalog_manager.cc:5697] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 1 to 2, leader changed from 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194) to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:03.207028 23574 raft_consensus.cc:993] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778: : Instructing follower 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:03.207553 23574 raft_consensus.cc:1081] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Signalling peer 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:03.210078 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:53072
I20260709 12:18:03.210744 23320 raft_consensus.cc:493] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:03.211141 23320 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.217353 23320 raft_consensus.cc:515] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.219185 23320 leader_election.cc:290] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:03.220073 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:03.220382 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:03.220809 23394 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.221033 23470 raft_consensus.cc:3055] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:03.221431 23470 raft_consensus.cc:740] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:03.222229 23470 consensus_queue.cc:260] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.223223 23470 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.226441 23394 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:03.227715 23284 leader_election.cc:304] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:03.228453 23560 raft_consensus.cc:2804] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:03.228890 23470 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:03.229085 23560 raft_consensus.cc:697] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:03.230090 23560 consensus_queue.cc:237] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.237237 23216 catalog_manager.cc:5697] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 1 to 2, leader changed from 51a03c362557435daf969e2f7d5ba778 (127.9.198.195) to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:03.303164 23563 raft_consensus.cc:993] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb: : Instructing follower 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:03.303692 23560 raft_consensus.cc:1081] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Signalling peer 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:03.305320 23394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:42164
I20260709 12:18:03.305879 23394 raft_consensus.cc:493] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:03.306202 23394 raft_consensus.cc:3060] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.311844 23394 raft_consensus.cc:515] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.313724 23394 leader_election.cc:290] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:03.314572 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:03.315282 23320 raft_consensus.cc:3055] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:03.315236 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b3b3a4266944f79320f1f2a4f1a0b4" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:03.315959 23320 raft_consensus.cc:740] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:03.316166 23470 raft_consensus.cc:3060] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.316780 23320 consensus_queue.cc:260] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.317910 23320 raft_consensus.cc:3060] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.321841 23470 raft_consensus.cc:2468] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:03.323016 23358 leader_election.cc:304] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:03.323794 23517 raft_consensus.cc:2804] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:03.323813 23320 raft_consensus.cc:2468] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:03.324417 23517 raft_consensus.cc:697] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:03.325312 23517 consensus_queue.cc:237] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.332868 23216 catalog_manager.cc:5697] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 1 to 2, leader changed from 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193) to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:03.423595 23320 raft_consensus.cc:1275] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.425120 23517 consensus_queue.cc:1048] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.427749 23516 raft_consensus.cc:993] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb: : Instructing follower 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:03.428339 23516 raft_consensus.cc:1081] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Signalling peer 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:03.430316 23394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:42164
I20260709 12:18:03.431021 23394 raft_consensus.cc:493] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:03.431461 23394 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.436784 23470 raft_consensus.cc:1275] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.438647 23579 consensus_queue.cc:1048] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.440049 23394 raft_consensus.cc:515] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.442440 23394 leader_election.cc:290] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:03.446487 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:03.447439 23320 raft_consensus.cc:3055] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:03.447818 23320 raft_consensus.cc:740] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:03.448043 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:03.448771 23320 consensus_queue.cc:260] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.448771 23471 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.450356 23320 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:03.456797 23471 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:03.456799 23320 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:03.458292 23358 leader_election.cc:304] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:03.459156 23580 raft_consensus.cc:2804] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:03.459642 23580 raft_consensus.cc:697] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:03.460431 23580 consensus_queue.cc:237] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:03.467348 23216 catalog_manager.cc:5697] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 1 to 2, leader changed from 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193) to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:03.469835 23471 raft_consensus.cc:1275] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.471333 23580 consensus_queue.cc:1048] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.481971 23320 raft_consensus.cc:1275] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.483565 23600 consensus_queue.cc:1048] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.522356 23471 raft_consensus.cc:1275] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.523819 23516 consensus_queue.cc:1048] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.531840 23394 raft_consensus.cc:1275] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.533617 23560 consensus_queue.cc:1048] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:03.618806 23394 raft_consensus.cc:1275] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.620229 23574 consensus_queue.cc:1048] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.628851 23320 raft_consensus.cc:1275] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.631101 23601 consensus_queue.cc:1048] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:03.782160 23471 raft_consensus.cc:1275] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.783629 23560 consensus_queue.cc:1048] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.791860 23394 raft_consensus.cc:1275] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.793445 23563 consensus_queue.cc:1048] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.886467 23319 raft_consensus.cc:1275] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.887873 23580 consensus_queue.cc:1048] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.897063 23471 raft_consensus.cc:1275] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.898464 23470 raft_consensus.cc:1275] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.899835 23579 consensus_queue.cc:1048] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.901436 23579 consensus_queue.cc:1048] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:03.923516 23319 raft_consensus.cc:1275] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:03.925860 23580 consensus_queue.cc:1048] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:05.968622 23216 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50904:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:18:05.971019 23216 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:18:06.033190 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 78eaeb31f661466e854e3eef1bea725a (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:06.033885 23383 tablet_service.cc:1511] Processing CreateTablet for tablet 78eaeb31f661466e854e3eef1bea725a (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:06.034581 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78eaeb31f661466e854e3eef1bea725a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.035234 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78eaeb31f661466e854e3eef1bea725a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.035540 23452 tablet_service.cc:1511] Processing CreateTablet for tablet 78eaeb31f661466e854e3eef1bea725a (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:06.036907 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78eaeb31f661466e854e3eef1bea725a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.042188 23307 tablet_service.cc:1511] Processing CreateTablet for tablet b9eb27af16f84101a04c28ae7696af3d (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:06.043864 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9eb27af16f84101a04c28ae7696af3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.045565 23457 tablet_service.cc:1511] Processing CreateTablet for tablet b9eb27af16f84101a04c28ae7696af3d (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:06.048233 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9eb27af16f84101a04c28ae7696af3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.046720 23384 tablet_service.cc:1511] Processing CreateTablet for tablet b9eb27af16f84101a04c28ae7696af3d (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:06.051326 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9eb27af16f84101a04c28ae7696af3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.051141 23309 tablet_service.cc:1511] Processing CreateTablet for tablet 9e9a7b21432f4d01b20c9291f0cc9a43 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:06.052528 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e9a7b21432f4d01b20c9291f0cc9a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.058658 23381 tablet_service.cc:1511] Processing CreateTablet for tablet 9e9a7b21432f4d01b20c9291f0cc9a43 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:06.060180 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e9a7b21432f4d01b20c9291f0cc9a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.064199 23460 tablet_service.cc:1511] Processing CreateTablet for tablet 9e9a7b21432f4d01b20c9291f0cc9a43 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:06.065771 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e9a7b21432f4d01b20c9291f0cc9a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.080857 23619 tablet_bootstrap.cc:492] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.084076 23382 tablet_service.cc:1511] Processing CreateTablet for tablet ad1ac780e48248fca8514e58b3abab9a (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:06.085600 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1ac780e48248fca8514e58b3abab9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.091590 23619 tablet_bootstrap.cc:654] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.091177 23310 tablet_service.cc:1511] Processing CreateTablet for tablet ad1ac780e48248fca8514e58b3abab9a (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:06.092747 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1ac780e48248fca8514e58b3abab9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.097672 23452 tablet_service.cc:1511] Processing CreateTablet for tablet ad1ac780e48248fca8514e58b3abab9a (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:06.099181 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1ac780e48248fca8514e58b3abab9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.106760 23307 tablet_service.cc:1511] Processing CreateTablet for tablet b97890a149bd42989ee95dd584d67569 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:06.108386 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97890a149bd42989ee95dd584d67569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.112578 23381 tablet_service.cc:1511] Processing CreateTablet for tablet b97890a149bd42989ee95dd584d67569 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:06.114048 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97890a149bd42989ee95dd584d67569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.147176 23384 tablet_service.cc:1511] Processing CreateTablet for tablet 31fc1459c8b74e219270a982e3118f1d (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:06.148092 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 31fc1459c8b74e219270a982e3118f1d (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:06.148831 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31fc1459c8b74e219270a982e3118f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.149669 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31fc1459c8b74e219270a982e3118f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.155622 23621 tablet_bootstrap.cc:492] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.154321 23460 tablet_service.cc:1511] Processing CreateTablet for tablet b97890a149bd42989ee95dd584d67569 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:06.157213 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97890a149bd42989ee95dd584d67569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.157090 23457 tablet_service.cc:1511] Processing CreateTablet for tablet 31fc1459c8b74e219270a982e3118f1d (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:06.159173 23310 tablet_service.cc:1511] Processing CreateTablet for tablet 08319e09aad5409ab029d052b6c799e9 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:06.160521 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08319e09aad5409ab029d052b6c799e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.165220 23621 tablet_bootstrap.cc:654] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.164903 23459 tablet_service.cc:1511] Processing CreateTablet for tablet 08319e09aad5409ab029d052b6c799e9 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:06.166589 23306 tablet_service.cc:1511] Processing CreateTablet for tablet 763c17e836ff4a64b1caf17acc31bc0c (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:06.171656 23619 tablet_bootstrap.cc:492] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.172410 23619 ts_tablet_manager.cc:1403] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.024s
I20260709 12:18:06.172775 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763c17e836ff4a64b1caf17acc31bc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.168195 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 763c17e836ff4a64b1caf17acc31bc0c (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:06.175352 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763c17e836ff4a64b1caf17acc31bc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.170840 23383 tablet_service.cc:1511] Processing CreateTablet for tablet 08319e09aad5409ab029d052b6c799e9 (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:06.171550 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31fc1459c8b74e219270a982e3118f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.175935 23619 raft_consensus.cc:359] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.176810 23619 raft_consensus.cc:385] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.177130 23619 raft_consensus.cc:740] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.179137 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08319e09aad5409ab029d052b6c799e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.180307 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08319e09aad5409ab029d052b6c799e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.182184 23620 tablet_bootstrap.cc:492] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.190548 23619 consensus_queue.cc:260] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.204309 23620 tablet_bootstrap.cc:654] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.209372 23619 ts_tablet_manager.cc:1434] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.002s
I20260709 12:18:06.222049 23619 tablet_bootstrap.cc:492] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.225061 23454 tablet_service.cc:1511] Processing CreateTablet for tablet 763c17e836ff4a64b1caf17acc31bc0c (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:06.226786 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763c17e836ff4a64b1caf17acc31bc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.232728 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc3d871f84a4e1cbe6543d2864a93fd (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:06.230253 23601 raft_consensus.cc:493] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.234324 23601 raft_consensus.cc:515] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.235476 23381 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc3d871f84a4e1cbe6543d2864a93fd (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:06.237061 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc3d871f84a4e1cbe6543d2864a93fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.236958 23601 leader_election.cc:290] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.238951 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:06.240705 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.234326 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc3d871f84a4e1cbe6543d2864a93fd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:18:06.241303 23433 leader_election.cc:343] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:18:06.242574 23432 leader_election.cc:343] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.243080 23432 leader_election.cc:304] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:06.244349 23601 raft_consensus.cc:2749] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:06.253546 23621 tablet_bootstrap.cc:492] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.254324 23621 ts_tablet_manager.cc:1403] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.099s	user 0.017s	sys 0.012s
I20260709 12:18:06.251462 23619 tablet_bootstrap.cc:654] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.257967 23452 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc3d871f84a4e1cbe6543d2864a93fd (DEFAULT_TABLE table=soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:06.260661 23621 raft_consensus.cc:359] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.267994 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc3d871f84a4e1cbe6543d2864a93fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:06.266243 23621 raft_consensus.cc:385] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.270516 23621 raft_consensus.cc:740] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.271790 23621 consensus_queue.cc:260] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.275089 23621 ts_tablet_manager.cc:1434] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.005s
I20260709 12:18:06.276290 23621 tablet_bootstrap.cc:492] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.284288 23619 tablet_bootstrap.cc:492] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.285014 23619 ts_tablet_manager.cc:1403] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.063s	user 0.020s	sys 0.001s
I20260709 12:18:06.285694 23620 tablet_bootstrap.cc:492] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:06.289404 23620 ts_tablet_manager.cc:1403] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.005s
I20260709 12:18:06.288839 23619 raft_consensus.cc:359] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.290016 23619 raft_consensus.cc:385] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.290436 23619 raft_consensus.cc:740] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.291538 23619 consensus_queue.cc:260] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.293020 23621 tablet_bootstrap.cc:654] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.299018 23620 raft_consensus.cc:359] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.305886 23620 raft_consensus.cc:385] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.312077 23619 ts_tablet_manager.cc:1434] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.001s
I20260709 12:18:06.314378 23620 raft_consensus.cc:740] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.315655 23620 consensus_queue.cc:260] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.314101 23619 tablet_bootstrap.cc:492] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.320045 23620 ts_tablet_manager.cc:1434] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.006s
I20260709 12:18:06.321417 23620 tablet_bootstrap.cc:492] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.330029 23619 tablet_bootstrap.cc:654] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.331394 23621 tablet_bootstrap.cc:492] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.332067 23621 ts_tablet_manager.cc:1403] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.023s
I20260709 12:18:06.335781 23621 raft_consensus.cc:359] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.336707 23621 raft_consensus.cc:385] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.337057 23621 raft_consensus.cc:740] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.330044 23620 tablet_bootstrap.cc:654] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.337996 23621 consensus_queue.cc:260] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.341605 23621 ts_tablet_manager.cc:1434] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 12:18:06.342960 23621 tablet_bootstrap.cc:492] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.350699 23621 tablet_bootstrap.cc:654] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.355479 23619 tablet_bootstrap.cc:492] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.356098 23619 ts_tablet_manager.cc:1403] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.014s
I20260709 12:18:06.359779 23619 raft_consensus.cc:359] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.360482 23621 tablet_bootstrap.cc:492] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.360754 23619 raft_consensus.cc:385] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.361152 23621 ts_tablet_manager.cc:1403] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:18:06.361225 23619 raft_consensus.cc:740] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.362424 23619 consensus_queue.cc:260] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.366353 23619 ts_tablet_manager.cc:1434] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 12:18:06.367966 23619 tablet_bootstrap.cc:492] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.372299 23621 raft_consensus.cc:359] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.373255 23621 raft_consensus.cc:385] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.373589 23621 raft_consensus.cc:740] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.377040 23619 tablet_bootstrap.cc:654] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.377801 23621 consensus_queue.cc:260] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.380776 23621 ts_tablet_manager.cc:1434] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:18:06.386468 23621 tablet_bootstrap.cc:492] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.389978 23601 raft_consensus.cc:493] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.390679 23601 raft_consensus.cc:515] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.394397 23601 leader_election.cc:290] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.398258 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.399402 23319 raft_consensus.cc:2468] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:06.402541 23432 leader_election.cc:304] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:06.403316 23621 tablet_bootstrap.cc:654] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.403591 23601 raft_consensus.cc:2804] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.404186 23601 raft_consensus.cc:493] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.404623 23601 raft_consensus.cc:3060] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.406596 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:06.418924 23621 tablet_bootstrap.cc:492] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.419930 23621 ts_tablet_manager.cc:1403] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.007s
I20260709 12:18:06.417464 23601 raft_consensus.cc:515] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.422824 23632 raft_consensus.cc:493] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:06.423647 23433 leader_election.cc:343] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.425599 23620 tablet_bootstrap.cc:492] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:06.425105 23621 raft_consensus.cc:359] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.426293 23620 ts_tablet_manager.cc:1403] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.022s
I20260709 12:18:06.426379 23621 raft_consensus.cc:385] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.427021 23621 raft_consensus.cc:740] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.424400 23632 raft_consensus.cc:515] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.428236 23621 consensus_queue.cc:260] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.432356 23621 ts_tablet_manager.cc:1434] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.012s	user 0.012s	sys 0.000s
I20260709 12:18:06.433874 23621 tablet_bootstrap.cc:492] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.434295 23632 leader_election.cc:290] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.434141 23601 leader_election.cc:290] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.475298 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:06.477192 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.433799 23620 raft_consensus.cc:359] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.480177 23620 raft_consensus.cc:385] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.480918 23620 raft_consensus.cc:740] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.482575 23620 consensus_queue.cc:260] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.477371 23319 raft_consensus.cc:3060] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.488413 23320 raft_consensus.cc:2468] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:06.492261 23432 leader_election.cc:304] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:06.495328 23601 raft_consensus.cc:2804] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.495980 23601 raft_consensus.cc:493] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.496594 23601 raft_consensus.cc:3060] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.497749 23619 tablet_bootstrap.cc:492] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.502086 23620 ts_tablet_manager.cc:1434] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.075s	user 0.010s	sys 0.005s
I20260709 12:18:06.505055 23620 tablet_bootstrap.cc:492] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.514667 23619 ts_tablet_manager.cc:1403] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.147s	user 0.025s	sys 0.042s
I20260709 12:18:06.532709 23319 raft_consensus.cc:2468] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:06.536103 23432 leader_election.cc:304] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:06.535810 23601 raft_consensus.cc:515] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.537588 23632 raft_consensus.cc:2804] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.545943 23619 raft_consensus.cc:359] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.547153 23619 raft_consensus.cc:385] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.547569 23619 raft_consensus.cc:740] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.548652 23619 consensus_queue.cc:260] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.556569 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:06.550206 23601 leader_election.cc:290] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.550444 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:06.551680 23621 tablet_bootstrap.cc:654] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.558773 23319 raft_consensus.cc:3060] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.554306 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:06.554665 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:06.563671 23395 raft_consensus.cc:3060] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.555346 23632 raft_consensus.cc:697] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
W20260709 12:18:06.569217 23433 leader_election.cc:343] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.574304 23632 consensus_queue.cc:237] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
W20260709 12:18:06.580807 23433 leader_election.cc:343] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.583387 23619 ts_tablet_manager.cc:1434] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.068s	user 0.012s	sys 0.010s
I20260709 12:18:06.585696 23619 tablet_bootstrap.cc:492] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.589556 23620 tablet_bootstrap.cc:654] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.594257 23637 raft_consensus.cc:493] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.594976 23637 raft_consensus.cc:515] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.595552 23619 tablet_bootstrap.cc:654] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.598960 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.599870 23320 raft_consensus.cc:2468] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:06.601495 23432 leader_election.cc:304] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:06.602797 23636 raft_consensus.cc:2804] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.603456 23636 raft_consensus.cc:493] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.603955 23636 raft_consensus.cc:3060] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.605017 23637 leader_election.cc:290] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.608078 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
W20260709 12:18:06.610800 23433 leader_election.cc:343] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.614543 23580 raft_consensus.cc:493] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.615365 23580 raft_consensus.cc:515] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.616286 23395 raft_consensus.cc:2468] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:06.618350 23580 leader_election.cc:290] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.618451 23319 raft_consensus.cc:2468] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:06.619985 23432 leader_election.cc:304] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:06.621515 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:06.621743 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.622573 23471 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:06.625068 23601 raft_consensus.cc:2804] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.622627 23320 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:06.625715 23601 raft_consensus.cc:697] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:06.631695 23636 raft_consensus.cc:515] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.633683 23601 consensus_queue.cc:237] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.634959 23619 tablet_bootstrap.cc:492] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.635620 23619 ts_tablet_manager.cc:1403] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.014s
I20260709 12:18:06.638111 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:06.639271 23318 raft_consensus.cc:3060] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.635540 23636 leader_election.cc:290] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.630296 23358 leader_election.cc:304] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:06.643391 23619 raft_consensus.cc:359] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.644517 23619 raft_consensus.cc:385] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.644937 23619 raft_consensus.cc:740] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.648057 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:06.649712 23621 tablet_bootstrap.cc:492] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.651060 23621 ts_tablet_manager.cc:1403] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.218s	user 0.046s	sys 0.023s
W20260709 12:18:06.652431 23433 leader_election.cc:343] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.652547 23619 consensus_queue.cc:260] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.647305 23580 raft_consensus.cc:2804] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.656540 23621 raft_consensus.cc:359] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.656786 23580 raft_consensus.cc:493] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.659611 23580 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.661779 23318 raft_consensus.cc:2468] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:06.663427 23432 leader_election.cc:304] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:06.663746 23620 tablet_bootstrap.cc:492] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:06.664462 23620 ts_tablet_manager.cc:1403] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.160s	user 0.028s	sys 0.012s
I20260709 12:18:06.664989 23636 raft_consensus.cc:2804] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.665596 23636 raft_consensus.cc:697] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:06.666791 23636 consensus_queue.cc:237] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.658738 23621 raft_consensus.cc:385] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.669186 23620 raft_consensus.cc:359] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.669910 23621 raft_consensus.cc:740] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.670145 23620 raft_consensus.cc:385] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.670555 23620 raft_consensus.cc:740] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.670898 23621 consensus_queue.cc:260] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.671293 23215 catalog_manager.cc:5697] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:06.673482 23621 ts_tablet_manager.cc:1434] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.003s
I20260709 12:18:06.671764 23620 consensus_queue.cc:260] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.675562 23619 ts_tablet_manager.cc:1434] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:18:06.681990 23620 ts_tablet_manager.cc:1434] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.006s
I20260709 12:18:06.683614 23620 tablet_bootstrap.cc:492] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.686559 23619 tablet_bootstrap.cc:492] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.696445 23621 tablet_bootstrap.cc:492] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.698846 23580 raft_consensus.cc:515] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.703970 23620 tablet_bootstrap.cc:654] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.704736 23580 leader_election.cc:290] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.705588 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:06.706389 23318 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.706525 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:06.707470 23471 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.714025 23621 tablet_bootstrap.cc:654] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.718843 23318 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:06.720580 23358 leader_election.cc:304] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:06.722049 23580 raft_consensus.cc:2804] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.722625 23580 raft_consensus.cc:697] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:06.725255 23619 tablet_bootstrap.cc:654] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.726290 23580 consensus_queue.cc:237] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.731501 23620 tablet_bootstrap.cc:492] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:06.732465 23620 ts_tablet_manager.cc:1403] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.006s
I20260709 12:18:06.734774 23621 tablet_bootstrap.cc:492] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.735572 23621 ts_tablet_manager.cc:1403] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.008s
I20260709 12:18:06.737320 23620 raft_consensus.cc:359] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.738507 23620 raft_consensus.cc:385] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.738910 23620 raft_consensus.cc:740] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.739768 23471 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:06.741425 23621 raft_consensus.cc:359] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.742259 23619 tablet_bootstrap.cc:492] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.743150 23619 ts_tablet_manager.cc:1403] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.006s
I20260709 12:18:06.742478 23621 raft_consensus.cc:385] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.746234 23621 raft_consensus.cc:740] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.748167 23621 consensus_queue.cc:260] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.752403 23621 ts_tablet_manager.cc:1434] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 12:18:06.753027 23619 raft_consensus.cc:359] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.750068 23217 catalog_manager.cc:5697] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.753440 23620 consensus_queue.cc:260] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.768458 23621 tablet_bootstrap.cc:492] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.767391 23215 catalog_manager.cc:5697] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:06.767383 23619 raft_consensus.cc:385] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.770035 23619 raft_consensus.cc:740] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.771209 23620 ts_tablet_manager.cc:1434] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.011s
I20260709 12:18:06.771013 23619 consensus_queue.cc:260] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.772686 23620 tablet_bootstrap.cc:492] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.774932 23215 catalog_manager.cc:5697] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:06.774024 23619 ts_tablet_manager.cc:1434] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.031s	user 0.016s	sys 0.000s
I20260709 12:18:06.781025 23619 tablet_bootstrap.cc:492] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.781311 23620 tablet_bootstrap.cc:654] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.783776 23563 raft_consensus.cc:493] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.784358 23563 raft_consensus.cc:515] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.788153 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97890a149bd42989ee95dd584d67569" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
W20260709 12:18:06.789831 23283 leader_election.cc:343] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.790586 23621 tablet_bootstrap.cc:654] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.792933 23619 tablet_bootstrap.cc:654] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.794431 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97890a149bd42989ee95dd584d67569" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
W20260709 12:18:06.796087 23284 leader_election.cc:343] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.796551 23284 leader_election.cc:304] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb; no voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:06.797564 23618 raft_consensus.cc:2749] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:06.802479 23563 leader_election.cc:290] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.804239 23621 tablet_bootstrap.cc:492] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.805116 23621 ts_tablet_manager.cc:1403] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.008s
I20260709 12:18:06.809186 23620 tablet_bootstrap.cc:492] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:06.809784 23580 raft_consensus.cc:493] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.809922 23620 ts_tablet_manager.cc:1403] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.001s
I20260709 12:18:06.810391 23580 raft_consensus.cc:515] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.811051 23621 raft_consensus.cc:359] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.811924 23621 raft_consensus.cc:385] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.812244 23621 raft_consensus.cc:740] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.813151 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.813510 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:06.813184 23621 consensus_queue.cc:260] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.814124 23318 raft_consensus.cc:2393] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in current term 1: Already voted for candidate 51a03c362557435daf969e2f7d5ba778 in this term.
I20260709 12:18:06.813910 23620 raft_consensus.cc:359] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.814814 23620 raft_consensus.cc:385] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.815193 23620 raft_consensus.cc:740] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.815912 23358 leader_election.cc:304] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:06.816437 23621 ts_tablet_manager.cc:1434] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.004s
I20260709 12:18:06.816088 23620 consensus_queue.cc:260] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.817004 23580 leader_election.cc:290] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.817502 23635 raft_consensus.cc:3060] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.819125 23620 ts_tablet_manager.cc:1434] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:18:06.819612 23619 tablet_bootstrap.cc:492] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.820166 23619 ts_tablet_manager.cc:1403] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.014s
I20260709 12:18:06.821043 23620 tablet_bootstrap.cc:492] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.823217 23621 tablet_bootstrap.cc:492] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.823807 23619 raft_consensus.cc:359] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.824626 23619 raft_consensus.cc:385] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.824947 23619 raft_consensus.cc:740] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.825824 23619 consensus_queue.cc:260] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.829241 23619 ts_tablet_manager.cc:1434] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 12:18:06.829947 23635 raft_consensus.cc:2749] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:06.830837 23619 tablet_bootstrap.cc:492] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.832212 23621 tablet_bootstrap.cc:654] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.839711 23619 tablet_bootstrap.cc:654] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.843448 23620 tablet_bootstrap.cc:654] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.855430 23601 raft_consensus.cc:493] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.856047 23601 raft_consensus.cc:515] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.858296 23601 leader_election.cc:290] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.860052 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
W20260709 12:18:06.861826 23433 leader_election.cc:343] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.863111 23619 tablet_bootstrap.cc:492] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.863685 23619 ts_tablet_manager.cc:1403] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.008s
I20260709 12:18:06.864638 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.869735 23318 raft_consensus.cc:2468] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:06.866901 23619 raft_consensus.cc:359] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.871510 23619 raft_consensus.cc:385] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.871845 23619 raft_consensus.cc:740] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.872592 23619 consensus_queue.cc:260] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.870812 23620 tablet_bootstrap.cc:492] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:06.873799 23432 leader_election.cc:304] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:06.874162 23620 ts_tablet_manager.cc:1403] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.029s
I20260709 12:18:06.875332 23619 ts_tablet_manager.cc:1434] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:18:06.875263 23601 raft_consensus.cc:2804] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.875685 23621 tablet_bootstrap.cc:492] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.875877 23601 raft_consensus.cc:493] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.876402 23621 ts_tablet_manager.cc:1403] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.028s
I20260709 12:18:06.876441 23601 raft_consensus.cc:3060] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.876746 23619 tablet_bootstrap.cc:492] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778: Bootstrap starting.
I20260709 12:18:06.878212 23620 raft_consensus.cc:359] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.879115 23620 raft_consensus.cc:385] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.879456 23620 raft_consensus.cc:740] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.880014 23621 raft_consensus.cc:359] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.880358 23620 consensus_queue.cc:260] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.880930 23621 raft_consensus.cc:385] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.881358 23621 raft_consensus.cc:740] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.882349 23621 consensus_queue.cc:260] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.883570 23620 ts_tablet_manager.cc:1434] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:18:06.884904 23621 ts_tablet_manager.cc:1434] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:18:06.885046 23620 tablet_bootstrap.cc:492] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.885788 23619 tablet_bootstrap.cc:654] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.886466 23621 tablet_bootstrap.cc:492] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb: Bootstrap starting.
I20260709 12:18:06.890478 23563 raft_consensus.cc:493] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.891072 23563 raft_consensus.cc:515] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.893397 23563 leader_election.cc:290] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.894419 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
I20260709 12:18:06.895499 23621 tablet_bootstrap.cc:654] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb: Neither blocks nor log segments found. Creating new log.
W20260709 12:18:06.896128 23284 leader_election.cc:343] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.896754 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:06.899950 23471 raft_consensus.cc:2468] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 0.
I20260709 12:18:06.899920 23620 tablet_bootstrap.cc:654] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.901881 23283 leader_election.cc:304] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:06.903867 23563 raft_consensus.cc:2804] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.904341 23563 raft_consensus.cc:493] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.904776 23563 raft_consensus.cc:3060] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.912839 23601 raft_consensus.cc:515] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.922591 23620 tablet_bootstrap.cc:492] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:06.923296 23620 ts_tablet_manager.cc:1403] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.001s
I20260709 12:18:06.923796 23563 raft_consensus.cc:515] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.927173 23620 raft_consensus.cc:359] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.927749 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:06.928119 23620 raft_consensus.cc:385] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.928181 23601 leader_election.cc:290] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:06.928519 23318 raft_consensus.cc:3060] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.928601 23620 raft_consensus.cc:740] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.929564 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:06.930049 23620 consensus_queue.cc:260] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.931128 23635 raft_consensus.cc:493] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.931571 23635 raft_consensus.cc:515] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.932505 23619 tablet_bootstrap.cc:492] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778: No bootstrap required, opened a new log
I20260709 12:18:06.932825 23620 ts_tablet_manager.cc:1434] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:18:06.933111 23619 ts_tablet_manager.cc:1403] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.008s
I20260709 12:18:06.933822 23395 raft_consensus.cc:3060] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.934264 23620 tablet_bootstrap.cc:492] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:06.936486 23619 raft_consensus.cc:359] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.937290 23619 raft_consensus.cc:385] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.937614 23619 raft_consensus.cc:740] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.938756 23621 tablet_bootstrap.cc:492] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb: No bootstrap required, opened a new log
I20260709 12:18:06.938433 23619 consensus_queue.cc:260] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.939536 23621 ts_tablet_manager.cc:1403] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb: Time spent bootstrapping tablet: real 0.053s	user 0.020s	sys 0.019s
I20260709 12:18:06.941936 23563 leader_election.cc:290] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.942682 23620 tablet_bootstrap.cc:654] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.943045 23318 raft_consensus.cc:2468] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:06.944396 23432 leader_election.cc:304] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:06.944727 23619 ts_tablet_manager.cc:1434] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 12:18:06.945379 23601 raft_consensus.cc:2804] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.945919 23601 raft_consensus.cc:697] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:06.946178 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:06.946832 23471 raft_consensus.cc:3060] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.946853 23601 consensus_queue.cc:237] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.941465 23634 raft_consensus.cc:493] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.950222 23634 raft_consensus.cc:515] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.952517 23634 leader_election.cc:290] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.941368 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:06.954819 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.955793 23319 raft_consensus.cc:2393] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in current term 1: Already voted for candidate 51a03c362557435daf969e2f7d5ba778 in this term.
I20260709 12:18:06.956225 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
W20260709 12:18:06.957151 23284 leader_election.cc:343] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.957720 23621 raft_consensus.cc:359] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.958786 23621 raft_consensus.cc:385] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.959129 23621 raft_consensus.cc:740] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.960330 23621 consensus_queue.cc:260] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.963263 23635 leader_election.cc:290] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.965863 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:06.966178 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:06.966799 23319 raft_consensus.cc:2393] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in current term 1: Already voted for candidate 51a03c362557435daf969e2f7d5ba778 in this term.
I20260709 12:18:06.968761 23621 ts_tablet_manager.cc:1434] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 12:18:06.972404 23358 leader_election.cc:304] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778
W20260709 12:18:06.978611 23265 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:06.980216 23471 raft_consensus.cc:2468] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 1.
I20260709 12:18:06.982604 23395 raft_consensus.cc:2468] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:06.984311 23634 raft_consensus.cc:2749] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:06.985805 23283 leader_election.cc:304] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:06.987303 23563 raft_consensus.cc:2804] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.987876 23563 raft_consensus.cc:697] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:06.989226 23563 consensus_queue.cc:237] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.990850 23634 raft_consensus.cc:493] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.991484 23634 raft_consensus.cc:515] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:06.987759 23358 leader_election.cc:304] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:06.994001 23634 leader_election.cc:290] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:06.995558 23634 raft_consensus.cc:3060] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.004784 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:07.005678 23319 raft_consensus.cc:2468] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:07.007076 23636 raft_consensus.cc:493] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.007668 23636 raft_consensus.cc:515] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.008162 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:07.008409 23358 leader_election.cc:304] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.009071 23471 raft_consensus.cc:2468] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:07.010007 23636 leader_election.cc:290] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:07.010897 23635 raft_consensus.cc:2804] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:07.011329 23620 tablet_bootstrap.cc:492] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:07.011454 23635 raft_consensus.cc:493] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.011901 23634 raft_consensus.cc:2749] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:07.012173 23620 ts_tablet_manager.cc:1403] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.078s	user 0.033s	sys 0.016s
I20260709 12:18:07.012135 23635 raft_consensus.cc:3060] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.012562 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc3d871f84a4e1cbe6543d2864a93fd" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:07.013561 23319 raft_consensus.cc:2468] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 0.
I20260709 12:18:07.014792 23432 leader_election.cc:304] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:07.015879 23636 raft_consensus.cc:2804] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:07.016523 23636 raft_consensus.cc:493] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.015864 23620 raft_consensus.cc:359] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.016870 23636 raft_consensus.cc:3060] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.017014 23620 raft_consensus.cc:385] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:07.017359 23620 raft_consensus.cc:740] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:07.018278 23620 consensus_queue.cc:260] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.023774 23632 consensus_queue.cc:1048] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.028594 23620 ts_tablet_manager.cc:1434] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:18:07.031500 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc3d871f84a4e1cbe6543d2864a93fd" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" is_pre_election: true
W20260709 12:18:07.033828 23433 leader_election.cc:343] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:07.038960 23620 tablet_bootstrap.cc:492] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93: Bootstrap starting.
I20260709 12:18:07.047175 23620 tablet_bootstrap.cc:654] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:07.047854 23636 raft_consensus.cc:515] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.050377 23216 catalog_manager.cc:5697] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 1 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.054589 23636 leader_election.cc:290] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:07.061023 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc3d871f84a4e1cbe6543d2864a93fd" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
W20260709 12:18:07.062570 23433 leader_election.cc:343] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:07.062844 23620 tablet_bootstrap.cc:492] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93: No bootstrap required, opened a new log
I20260709 12:18:07.063423 23620 ts_tablet_manager.cc:1403] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent bootstrapping tablet: real 0.025s	user 0.023s	sys 0.001s
I20260709 12:18:07.066493 23620 raft_consensus.cc:359] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.067202 23620 raft_consensus.cc:385] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:07.067461 23620 raft_consensus.cc:740] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Initialized, Role: FOLLOWER
I20260709 12:18:07.067845 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc3d871f84a4e1cbe6543d2864a93fd" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:07.068499 23318 raft_consensus.cc:3060] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.068172 23620 consensus_queue.cc:260] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.070170 23634 raft_consensus.cc:493] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.070430 23620 ts_tablet_manager.cc:1434] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:18:07.070863 23634 raft_consensus.cc:515] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.075238 23634 leader_election.cc:290] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.077869 23635 raft_consensus.cc:515] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.080192 23635 leader_election.cc:290] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.083555 23318 raft_consensus.cc:2468] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 1.
I20260709 12:18:07.084985 23432 leader_election.cc:304] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:07.088498 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97890a149bd42989ee95dd584d67569" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:07.089470 23319 raft_consensus.cc:2468] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:07.091252 23358 leader_election.cc:304] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.091614 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.091904 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:07.088066 23601 raft_consensus.cc:2804] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:07.092641 23318 raft_consensus.cc:3060] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.092774 23601 raft_consensus.cc:697] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:07.091153 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97890a149bd42989ee95dd584d67569" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:07.093873 23471 raft_consensus.cc:2468] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 0.
I20260709 12:18:07.094312 23601 consensus_queue.cc:237] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.096253 23216 catalog_manager.cc:5697] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:07.097596 23635 raft_consensus.cc:2804] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:07.092375 23470 raft_consensus.cc:3060] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.098203 23635 raft_consensus.cc:493] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.099900 23635 raft_consensus.cc:3060] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.101531 23318 raft_consensus.cc:2468] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:07.107956 23358 leader_election.cc:304] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.108783 23634 raft_consensus.cc:2804] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:07.109289 23634 raft_consensus.cc:697] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:07.109453 23635 raft_consensus.cc:515] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.111369 23634 consensus_queue.cc:237] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.112037 23635 leader_election.cc:290] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.115768 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97890a149bd42989ee95dd584d67569" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.116575 23471 raft_consensus.cc:3060] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.113881 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97890a149bd42989ee95dd584d67569" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:07.116273 23636 consensus_queue.cc:1048] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.117239 23319 raft_consensus.cc:3060] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.126188 23319 raft_consensus.cc:2468] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:07.127332 23601 consensus_queue.cc:1048] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.113754 23470 raft_consensus.cc:2468] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:07.129246 23358 leader_election.cc:304] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.129814 23216 catalog_manager.cc:5697] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:07.130834 23635 raft_consensus.cc:2804] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:07.131503 23635 raft_consensus.cc:697] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:07.132768 23635 consensus_queue.cc:237] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.133744 23471 raft_consensus.cc:2468] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 1.
I20260709 12:18:07.141825 23601 consensus_queue.cc:1048] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.159485 23636 consensus_queue.cc:1048] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.160115 23216 catalog_manager.cc:5697] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.179867 23634 raft_consensus.cc:493] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.180430 23634 raft_consensus.cc:515] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.182788 23634 leader_election.cc:290] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.184356 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:07.193526 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:07.199537 23471 raft_consensus.cc:2393] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in current term 1: Already voted for candidate 4b6e2a23c11644ec92cc018d8be502eb in this term.
I20260709 12:18:07.203393 23632 consensus_queue.cc:1048] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.203274 23214 catalog_manager.cc:5697] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.203459 23358 leader_election.cc:304] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:07.207962 23634 raft_consensus.cc:3060] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.217537 23634 raft_consensus.cc:2749] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:07.225364 23634 consensus_queue.cc:1048] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.235971 23634 raft_consensus.cc:493] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.236689 23634 raft_consensus.cc:515] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.239274 23634 leader_election.cc:290] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.240801 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc3d871f84a4e1cbe6543d2864a93fd" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb" is_pre_election: true
I20260709 12:18:07.241781 23318 raft_consensus.cc:2393] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in current term 1: Already voted for candidate 51a03c362557435daf969e2f7d5ba778 in this term.
I20260709 12:18:07.244189 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc3d871f84a4e1cbe6543d2864a93fd" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a03c362557435daf969e2f7d5ba778" is_pre_election: true
I20260709 12:18:07.245911 23358 leader_election.cc:304] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:07.247238 23634 raft_consensus.cc:3060] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:07.250046 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
mode: GRACEFUL
new_leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:42182
I20260709 12:18:07.250911 23394 raft_consensus.cc:606] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Received request to transfer leadership to TS 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:07.255860 23634 raft_consensus.cc:2749] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:07.256765 23634 raft_consensus.cc:993] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: : Instructing follower 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.257261 23634 raft_consensus.cc:1081] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Signalling peer 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.258231 23471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
mode: GRACEFUL
new_leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:49664
I20260709 12:18:07.259027 23471 raft_consensus.cc:606] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Received request to transfer leadership to TS 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:07.265475 23634 consensus_queue.cc:1048] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.269187 23318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:53066
I20260709 12:18:07.270464 23318 raft_consensus.cc:493] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.270906 23318 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.278133 23471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
mode: GRACEFUL
new_leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:49664
I20260709 12:18:07.279028 23471 raft_consensus.cc:606] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Received request to transfer leadership to TS 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:07.288009 23471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
mode: GRACEFUL
new_leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:49664
I20260709 12:18:07.288707 23471 raft_consensus.cc:606] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Received request to transfer leadership to TS 4b6e2a23c11644ec92cc018d8be502eb
I20260709 12:18:07.291306 23318 raft_consensus.cc:515] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.294425 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:07.294678 23320 raft_consensus.cc:1240] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Rejecting Update request from peer 547d1ff7c5b94fbe93a7fa981f24ec93 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.295156 23395 raft_consensus.cc:3055] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.295578 23395 raft_consensus.cc:740] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:07.296475 23318 leader_election.cc:290] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.296475 23395 consensus_queue.cc:260] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.298331 23395 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.303021 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.303725 23471 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.303810 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
mode: GRACEFUL
new_leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:53086
I20260709 12:18:07.304667 23320 raft_consensus.cc:606] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Received request to transfer leadership to TS 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:07.312827 23395 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.312984 23471 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.313022 23470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
mode: GRACEFUL
new_leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:49664
I20260709 12:18:07.314098 23470 raft_consensus.cc:606] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Received request to transfer leadership to TS 547d1ff7c5b94fbe93a7fa981f24ec93
I20260709 12:18:07.314548 23283 leader_election.cc:304] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:07.315572 23563 raft_consensus.cc:2804] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.316176 23563 raft_consensus.cc:697] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:07.317365 23563 consensus_queue.cc:237] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.327199 23217 catalog_manager.cc:5697] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 1 to 2, leader changed from 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194) to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.477932 23636 raft_consensus.cc:993] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: : Instructing follower 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.478432 23632 raft_consensus.cc:1081] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Signalling peer 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.480005 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:53072
I20260709 12:18:07.480581 23320 raft_consensus.cc:493] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.480934 23320 raft_consensus.cc:3060] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.485829 23563 consensus_queue.cc:1048] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.489127 23320 raft_consensus.cc:515] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.492456 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:07.492700 23320 leader_election.cc:290] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.493145 23395 raft_consensus.cc:3060] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.495071 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb27af16f84101a04c28ae7696af3d" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.495800 23470 raft_consensus.cc:3055] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.496130 23470 raft_consensus.cc:740] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:07.496937 23470 consensus_queue.cc:260] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.498232 23470 raft_consensus.cc:3060] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.502241 23632 consensus_queue.cc:1048] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.505537 23470 raft_consensus.cc:2468] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.508293 23283 leader_election.cc:304] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:07.509286 23618 raft_consensus.cc:2804] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.509800 23618 raft_consensus.cc:697] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:07.510661 23618 consensus_queue.cc:237] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.513447 23635 consensus_queue.cc:1048] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.518397 23395 raft_consensus.cc:2468] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.525880 23563 consensus_queue.cc:1048] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.528743 23632 consensus_queue.cc:1048] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.530854 23635 consensus_queue.cc:1048] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.537232 23636 raft_consensus.cc:993] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: : Instructing follower 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:07.537962 23636 raft_consensus.cc:1081] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Signalling peer 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:07.540539 23392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:42172
I20260709 12:18:07.541133 23392 raft_consensus.cc:493] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.541523 23392 raft_consensus.cc:3060] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.544500 23217 catalog_manager.cc:5697] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 1 to 2, leader changed from 51a03c362557435daf969e2f7d5ba778 (127.9.198.195) to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.559383 23635 consensus_queue.cc:1048] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.561321 23618 raft_consensus.cc:993] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: : Instructing follower 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:07.562286 23563 raft_consensus.cc:1081] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Signalling peer 547d1ff7c5b94fbe93a7fa981f24ec93 to start an election
I20260709 12:18:07.566365 23394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
 from {username='slave'} at 127.0.0.1:42164
I20260709 12:18:07.567073 23394 raft_consensus.cc:493] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.566465 23392 raft_consensus.cc:515] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.567528 23394 raft_consensus.cc:3060] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.568322 23395 raft_consensus.cc:1240] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Rejecting Update request from peer 51a03c362557435daf969e2f7d5ba778 for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:07.573941 23392 leader_election.cc:290] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.584339 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:07.585054 23318 raft_consensus.cc:3060] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.588480 23394 raft_consensus.cc:515] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.592813 23318 raft_consensus.cc:2468] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:07.593672 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:07.594326 23319 raft_consensus.cc:3055] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.594647 23319 raft_consensus.cc:740] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:07.595484 23394 leader_election.cc:290] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.596134 23358 leader_election.cc:304] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.595543 23319 consensus_queue.cc:260] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.597008 23634 raft_consensus.cc:2804] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.597481 23634 raft_consensus.cc:697] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:07.597491 23319 raft_consensus.cc:3060] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.598390 23634 consensus_queue.cc:237] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.602844 23637 consensus_queue.cc:1059] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.603950 23632 raft_consensus.cc:3055] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.604308 23632 raft_consensus.cc:740] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:07.604687 23636 raft_consensus.cc:993] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: : Instructing follower 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.605141 23636 raft_consensus.cc:1081] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Signalling peer 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.605945 23319 raft_consensus.cc:2468] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:07.605163 23632 consensus_queue.cc:260] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.608661 23358 leader_election.cc:304] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.609479 23635 raft_consensus.cc:2804] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.609642 23319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:53072
I20260709 12:18:07.609315 23217 catalog_manager.cc:5697] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 1 to 2, leader changed from 51a03c362557435daf969e2f7d5ba778 (127.9.198.195) to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.610196 23635 raft_consensus.cc:697] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Becoming Leader. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:07.610339 23319 raft_consensus.cc:493] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.610855 23319 raft_consensus.cc:3060] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.611194 23635 consensus_queue.cc:237] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.612470 23632 raft_consensus.cc:3060] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.618216 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08319e09aad5409ab029d052b6c799e9" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.618743 23319 raft_consensus.cc:515] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.620237 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31fc1459c8b74e219270a982e3118f1d" candidate_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.620909 23468 raft_consensus.cc:3060] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.622273 23319 leader_election.cc:290] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.622429 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:07.623199 23394 raft_consensus.cc:3060] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.632943 23468 raft_consensus.cc:2468] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:07.635869 23217 catalog_manager.cc:5697] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 reported cstate change: term changed from 1 to 2, leader changed from 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193) to 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.637019 23471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1ac780e48248fca8514e58b3abab9a" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.639283 23471 raft_consensus.cc:3055] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.639725 23471 raft_consensus.cc:740] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:07.642167 23394 raft_consensus.cc:2468] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.643477 23284 leader_election.cc:304] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.644361 23563 raft_consensus.cc:2804] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.644573 23580 consensus_queue.cc:1048] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:07.644949 23563 raft_consensus.cc:697] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:07.645169 23469 raft_consensus.cc:2468] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547d1ff7c5b94fbe93a7fa981f24ec93 in term 2.
I20260709 12:18:07.646045 23563 consensus_queue.cc:237] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.642886 23471 consensus_queue.cc:260] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.650485 23471 raft_consensus.cc:3060] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.657769 23216 catalog_manager.cc:5697] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 1 to 2, leader changed from 51a03c362557435daf969e2f7d5ba778 (127.9.198.195) to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.660687 23471 raft_consensus.cc:2468] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.673794 23632 consensus_queue.cc:1048] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.686379 23601 consensus_queue.cc:1048] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.708910 23601 raft_consensus.cc:993] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: : Instructing follower 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.709450 23636 raft_consensus.cc:1081] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Signalling peer 4b6e2a23c11644ec92cc018d8be502eb to start an election
I20260709 12:18:07.710937 23394 raft_consensus.cc:1275] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.711040 23319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
 from {username='slave'} at 127.0.0.1:53072
I20260709 12:18:07.711715 23319 raft_consensus.cc:493] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.712146 23319 raft_consensus.cc:3060] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.712491 23563 consensus_queue.cc:1048] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.719054 23319 raft_consensus.cc:515] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.721065 23469 raft_consensus.cc:1275] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.722579 23685 consensus_queue.cc:1048] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.725596 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:07.726272 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e9a7b21432f4d01b20c9291f0cc9a43" candidate_uuid: "4b6e2a23c11644ec92cc018d8be502eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51a03c362557435daf969e2f7d5ba778"
I20260709 12:18:07.726292 23394 raft_consensus.cc:3060] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.726881 23469 raft_consensus.cc:3055] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.727233 23469 raft_consensus.cc:740] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:07.728042 23469 consensus_queue.cc:260] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.728977 23469 raft_consensus.cc:3060] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.723568 23319 leader_election.cc:290] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Requested vote from peers 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153), 51a03c362557435daf969e2f7d5ba778 (127.9.198.195:46541)
I20260709 12:18:07.737533 23469 raft_consensus.cc:2468] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.737689 23394 raft_consensus.cc:2468] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b6e2a23c11644ec92cc018d8be502eb in term 2.
I20260709 12:18:07.739118 23284 leader_election.cc:304] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:07.740602 23685 raft_consensus.cc:2804] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.741385 23685 raft_consensus.cc:697] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Leader. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:07.742482 23685 consensus_queue.cc:237] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:07.749353 23216 catalog_manager.cc:5697] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb reported cstate change: term changed from 1 to 2, leader changed from 51a03c362557435daf969e2f7d5ba778 (127.9.198.195) to 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193). New cstate: current_term: 2 leader_uuid: "4b6e2a23c11644ec92cc018d8be502eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.966612 23469 raft_consensus.cc:1275] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.968017 23685 consensus_queue.cc:1048] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.978485 23392 raft_consensus.cc:1275] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.980633 23563 consensus_queue.cc:1048] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.023007 23471 raft_consensus.cc:1275] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.024485 23634 consensus_queue.cc:1048] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.031669 23319 raft_consensus.cc:1275] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.033262 23634 consensus_queue.cc:1048] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.139714 23471 raft_consensus.cc:1275] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.141004 23634 consensus_queue.cc:1048] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.142868 23392 raft_consensus.cc:1275] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.144258 23563 consensus_queue.cc:1048] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.150377 23320 raft_consensus.cc:1275] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 547d1ff7c5b94fbe93a7fa981f24ec93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.156872 23634 consensus_queue.cc:1048] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.157706 23469 raft_consensus.cc:1275] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.158885 23471 raft_consensus.cc:1275] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.161334 23690 consensus_queue.cc:1048] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.163103 23563 consensus_queue.cc:1048] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:08.177278 23392 raft_consensus.cc:1275] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 4b6e2a23c11644ec92cc018d8be502eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:08.178905 23563 consensus_queue.cc:1048] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:10.333256 23216 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50904:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:18:10.353981 23216 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50904:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:18:10.393339 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:10.531240 23731 tablet.cc:1724] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.582391 23729 tablet.cc:1724] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:10.678375 23735 log.cc:927] Time spent T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:18:10.620900 23732 tablet.cc:1724] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:10.684494 23421 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:78eaeb31f661466e854e3eef1bea725a)
W20260709 12:18:10.690493 23421 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:78eaeb31f661466e854e3eef1bea725a)
I20260709 12:18:10.672370 23741 tablet.cc:1724] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:10.698091 23421 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:78eaeb31f661466e854e3eef1bea725a)
W20260709 12:18:10.715216 23737 log.cc:927] Time spent T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: Append to log took a long time: real 0.141s	user 0.003s	sys 0.002s
I20260709 12:18:10.695421 23751 tablet.cc:1724] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.742415 23753 tablet.cc:1724] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.787297 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
mode: GRACEFUL
new_leader_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:37570
I20260709 12:18:10.757416 23757 tablet.cc:1724] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.788177 23393 raft_consensus.cc:606] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Received request to transfer leadership to TS 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:10.753512 23756 tablet.cc:1724] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.816883 23761 tablet.cc:1724] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.822523 23759 tablet.cc:1724] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.826571 23729 tablet.cc:1724] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.831646 23732 tablet.cc:1724] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.856974 23764 tablet.cc:1724] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.854470 23762 tablet.cc:1724] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.874998 23757 tablet.cc:1724] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.889916 23742 raft_consensus.cc:993] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: : Instructing follower 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:10.890697 23749 raft_consensus.cc:1081] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Signalling peer 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:10.892607 23470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:49656
I20260709 12:18:10.893134 23470 raft_consensus.cc:493] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:10.893484 23470 raft_consensus.cc:3060] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:10.896438 23729 tablet.cc:1724] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.900898 23470 raft_consensus.cc:515] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:10.907055 23741 tablet.cc:1724] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.921207 23756 tablet.cc:1724] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.925968 23764 tablet.cc:1724] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.926188 23732 tablet.cc:1724] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.939621 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:10.940336 23317 raft_consensus.cc:3060] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:10.934083 23731 tablet.cc:1724] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.866901 23751 tablet.cc:1724] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.942411 23741 tablet.cc:1724] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.952941 23753 tablet.cc:1724] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.957790 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
mode: GRACEFUL
new_leader_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:50612
I20260709 12:18:10.958541 23320 raft_consensus.cc:606] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Received request to transfer leadership to TS 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:10.959942 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:18:10.960659 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:10.966696 23759 tablet.cc:1724] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.973599 23317 raft_consensus.cc:2468] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 2.
I20260709 12:18:10.975135 23432 leader_election.cc:304] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:10.974375 23470 leader_election.cc:290] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 2 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:10.977402 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c17e836ff4a64b1caf17acc31bc0c" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:10.978148 23393 raft_consensus.cc:3055] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:10.978448 23393 raft_consensus.cc:740] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:10.976262 23693 raft_consensus.cc:2804] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:10.981441 23693 raft_consensus.cc:697] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 2 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:10.981580 23731 tablet.cc:1724] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:10.982537 23693 consensus_queue.cc:237] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:10.984606 23393 consensus_queue.cc:260] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:10.985925 23393 raft_consensus.cc:3060] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:10.987864 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf"
dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
mode: GRACEFUL
new_leader_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:50612
I20260709 12:18:10.988494 23320 raft_consensus.cc:606] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Received request to transfer leadership to TS 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:10.994895 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0"
dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
mode: GRACEFUL
new_leader_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:37570
I20260709 12:18:10.995488 23394 raft_consensus.cc:606] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Received request to transfer leadership to TS 51a03c362557435daf969e2f7d5ba778
I20260709 12:18:10.999225 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:18:10.999784 23729 tablet.cc:1724] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:11.013871 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:11.020401 23393 raft_consensus.cc:2468] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 2.
I20260709 12:18:11.031311 23710 raft_consensus.cc:993] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: : Instructing follower 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:11.031809 23710 raft_consensus.cc:1081] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Signalling peer 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:11.033838 23470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:18:11.034443 23470 raft_consensus.cc:493] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.034768 23470 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.042598 23470 raft_consensus.cc:515] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.042377 23216 catalog_manager.cc:5697] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 1 to 2, leader changed from 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194) to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.047606 23470 leader_election.cc:290] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 3 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:11.048259 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:11.048995 23317 raft_consensus.cc:3055] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:11.047602 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78eaeb31f661466e854e3eef1bea725a" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:11.049360 23317 raft_consensus.cc:740] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:11.049909 23393 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.051250 23317 consensus_queue.cc:260] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.052904 23317 raft_consensus.cc:3060] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.060344 23393 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 3.
I20260709 12:18:11.062397 23433 leader_election.cc:304] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:11.063417 23693 raft_consensus.cc:2804] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:11.064000 23693 raft_consensus.cc:697] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 3 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:11.064920 23693 consensus_queue.cc:237] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.074766 23317 raft_consensus.cc:2468] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 3.
I20260709 12:18:11.085666 23215 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf: soft_deleted_table [id=c50b5d85063647159fe57ce87ef4e021] alter complete (version 1)
I20260709 12:18:11.100893 23215 catalog_manager.cc:5697] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 2 to 3, leader changed from 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193) to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.175680 23725 raft_consensus.cc:993] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb: : Instructing follower 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:11.176169 23725 raft_consensus.cc:1081] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Signalling peer 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:11.177791 23470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:49646
I20260709 12:18:11.178294 23470 raft_consensus.cc:493] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.178604 23470 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.184653 23470 raft_consensus.cc:515] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.186156 23470 leader_election.cc:290] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 3 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:11.187119 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:11.187135 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebc3148c6444a9191346aaa181695bf" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:11.187865 23317 raft_consensus.cc:3055] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:11.188050 23393 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.188177 23317 raft_consensus.cc:740] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b6e2a23c11644ec92cc018d8be502eb, State: Running, Role: LEADER
I20260709 12:18:11.189003 23317 consensus_queue.cc:260] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.190084 23317 raft_consensus.cc:3060] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.194255 23393 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 3.
I20260709 12:18:11.195684 23433 leader_election.cc:304] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a03c362557435daf969e2f7d5ba778, 547d1ff7c5b94fbe93a7fa981f24ec93; no voters: 
I20260709 12:18:11.195952 23317 raft_consensus.cc:2468] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 3.
I20260709 12:18:11.196607 23693 raft_consensus.cc:2804] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:11.197211 23693 raft_consensus.cc:697] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 3 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:11.198122 23693 consensus_queue.cc:237] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.206250 23216 catalog_manager.cc:5697] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 2 to 3, leader changed from 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193) to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.479663 23317 raft_consensus.cc:1275] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:18:11.480973 23693 consensus_queue.cc:1048] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:11.491789 23394 raft_consensus.cc:1275] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:18:11.493742 23773 consensus_queue.cc:1048] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:11.550414 23742 raft_consensus.cc:993] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93: : Instructing follower 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:11.550925 23776 raft_consensus.cc:1081] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Signalling peer 51a03c362557435daf969e2f7d5ba778 to start an election
I20260709 12:18:11.552557 23468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0"
dest_uuid: "51a03c362557435daf969e2f7d5ba778"
 from {username='slave'} at 127.0.0.1:49656
I20260709 12:18:11.553092 23468 raft_consensus.cc:493] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.553417 23468 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.559271 23468 raft_consensus.cc:515] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.561060 23468 leader_election.cc:290] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 3 election: Requested vote from peers 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807), 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153)
I20260709 12:18:11.562026 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b6e2a23c11644ec92cc018d8be502eb"
I20260709 12:18:11.562017 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be0f2e3c754b43ed89096070a9d076a0" candidate_uuid: "51a03c362557435daf969e2f7d5ba778" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93"
I20260709 12:18:11.562803 23317 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.562906 23394 raft_consensus.cc:3055] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:11.563311 23394 raft_consensus.cc:740] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 547d1ff7c5b94fbe93a7fa981f24ec93, State: Running, Role: LEADER
I20260709 12:18:11.564141 23394 consensus_queue.cc:260] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.565177 23394 raft_consensus.cc:3060] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:11.568707 23317 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 3.
I20260709 12:18:11.570124 23432 leader_election.cc:304] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6e2a23c11644ec92cc018d8be502eb, 51a03c362557435daf969e2f7d5ba778; no voters: 
I20260709 12:18:11.570510 23394 raft_consensus.cc:2468] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a03c362557435daf969e2f7d5ba778 in term 3.
I20260709 12:18:11.571110 23773 raft_consensus.cc:2804] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:11.571749 23773 raft_consensus.cc:697] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 3 LEADER]: Becoming Leader. State: Replica: 51a03c362557435daf969e2f7d5ba778, State: Running, Role: LEADER
I20260709 12:18:11.572600 23773 consensus_queue.cc:237] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } }
I20260709 12:18:11.581473 23215 catalog_manager.cc:5697] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 reported cstate change: term changed from 2 to 3, leader changed from 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194) to 51a03c362557435daf969e2f7d5ba778 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "51a03c362557435daf969e2f7d5ba778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a03c362557435daf969e2f7d5ba778" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 46541 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:11.585350 23394 raft_consensus.cc:1275] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:18:11.586779 23773 consensus_queue.cc:1048] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:11.598150 23317 raft_consensus.cc:1275] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:18:11.600598 23773 consensus_queue.cc:1048] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:11.667797 23317 raft_consensus.cc:1275] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:11.669094 23773 consensus_queue.cc:1048] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:11.678738 23394 raft_consensus.cc:1275] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:11.681486 23779 consensus_queue.cc:1048] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:11.973838 23317 raft_consensus.cc:1275] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:11.975167 23733 consensus_queue.cc:1048] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6e2a23c11644ec92cc018d8be502eb" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 37807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:11.986721 23394 raft_consensus.cc:1275] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Refusing update from remote peer 51a03c362557435daf969e2f7d5ba778: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:11.988364 23779 consensus_queue.cc:1048] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547d1ff7c5b94fbe93a7fa981f24ec93" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:12.015196 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:12.021064 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:12.021528 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:12.027084 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:12.027527 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:13.028565 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:13.035977 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:13.036461 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:13.041834 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:13.042259 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:14.043426 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:14.048640 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:14.049039 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:14.055562 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:14.056005 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:15.057163 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:15.063136 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:15.063699 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:15.070151 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:15.070756 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:16.071939 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:16.077896 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:16.078408 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:16.083751 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:16.084414 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:17.086148 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:17.096601 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:17.097048 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:17.104100 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:17.104518 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:18.105743 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:18.110985 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:18.111466 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:18.118368 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:18.118738 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:19.119722 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:19.124573 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:19.124949 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:19.130185 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:19.130661 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:20.131582 10011 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:20.136499 10011 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:20.136919 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:20.141609 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:20.142032 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:21.206987 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
W20260709 12:18:21.221828 23358 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 12:18:21.230835 23358 consensus_peers.cc:597] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.271165 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:21.272475 10011 tablet_replica.cc:333] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.274261 10011 raft_consensus.cc:2243] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.275525 10011 raft_consensus.cc:2272] T b97890a149bd42989ee95dd584d67569 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:21.277547 23358 consensus_peers.cc:597] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.280035 10011 tablet_replica.cc:333] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.281375 10011 raft_consensus.cc:2243] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.282387 10011 raft_consensus.cc:2272] T 08319e09aad5409ab029d052b6c799e9 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:21.285534 23358 consensus_peers.cc:597] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.286955 10011 tablet_replica.cc:333] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.288089 10011 raft_consensus.cc:2243] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.289500 10011 raft_consensus.cc:2272] T ad1ac780e48248fca8514e58b3abab9a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:21.292426 23432 consensus_peers.cc:597] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.293356 10011 tablet_replica.cc:333] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.294275 10011 raft_consensus.cc:2243] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.294961 10011 raft_consensus.cc:2272] T 763c17e836ff4a64b1caf17acc31bc0c P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.297560 10011 tablet_replica.cc:333] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.298372 10011 raft_consensus.cc:2243] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.298902 10011 raft_consensus.cc:2272] T 78eaeb31f661466e854e3eef1bea725a P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.301487 10011 tablet_replica.cc:333] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.302160 10011 raft_consensus.cc:2243] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.302824 10011 raft_consensus.cc:2272] T 4eaaadad8ef849f5b67928865a4f97ec P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.305953 10011 tablet_replica.cc:333] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.306663 10011 raft_consensus.cc:2243] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.307322 10011 raft_consensus.cc:2272] T be0f2e3c754b43ed89096070a9d076a0 P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.310010 10011 tablet_replica.cc:333] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.310675 10011 raft_consensus.cc:2243] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.311811 10011 raft_consensus.cc:2272] T b9eb27af16f84101a04c28ae7696af3d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.315140 10011 tablet_replica.cc:333] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.315869 10011 raft_consensus.cc:2243] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.316489 10011 raft_consensus.cc:2272] T a7194c62978a4c2bba240ca41873449a P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.319640 10011 tablet_replica.cc:333] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.320348 10011 raft_consensus.cc:2243] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.321329 10011 raft_consensus.cc:2272] T 703a57dd01894388a102b7bfed51641e P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.323837 10011 tablet_replica.cc:333] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.324505 10011 raft_consensus.cc:2243] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.325476 10011 raft_consensus.cc:2272] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.328298 10011 tablet_replica.cc:333] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.328914 10011 raft_consensus.cc:2243] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.329926 10011 raft_consensus.cc:2272] T 86eb26a3d46147719da1edd9dbd4270b P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.332618 10011 tablet_replica.cc:333] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.333230 10011 raft_consensus.cc:2243] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.334242 10011 raft_consensus.cc:2272] T 77d530be1ee246a3aa70928555d7b233 P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.336707 10011 tablet_replica.cc:333] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.337293 10011 raft_consensus.cc:2243] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.337924 10011 raft_consensus.cc:2272] T 31fc1459c8b74e219270a982e3118f1d P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.340621 10011 tablet_replica.cc:333] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.341246 10011 raft_consensus.cc:2243] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.341782 10011 raft_consensus.cc:2272] T 8ebc3148c6444a9191346aaa181695bf P 4b6e2a23c11644ec92cc018d8be502eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.344524 10011 tablet_replica.cc:333] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.345108 10011 raft_consensus.cc:2243] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.345697 10011 raft_consensus.cc:2272] T 210c6a9c1c5d486f8ace5a042a52bcca P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.348379 10011 tablet_replica.cc:333] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.348979 10011 raft_consensus.cc:2243] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.349568 10011 raft_consensus.cc:2272] T 5dc3d871f84a4e1cbe6543d2864a93fd P 4b6e2a23c11644ec92cc018d8be502eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.352216 10011 tablet_replica.cc:333] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb: stopping tablet replica
I20260709 12:18:21.352783 10011 raft_consensus.cc:2243] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.353336 10011 raft_consensus.cc:2272] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 4b6e2a23c11644ec92cc018d8be502eb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:21.366093 23432 consensus_peers.cc:597] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:21.374502 23432 consensus_peers.cc:597] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.404726 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
I20260709 12:18:21.458945 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
W20260709 12:18:21.459282 23358 consensus_peers.cc:597] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:37807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:21.487192 23358 consensus_peers.cc:597] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:37807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.491112 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:21.492022 10011 tablet_replica.cc:333] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.492870 10011 raft_consensus.cc:2243] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.493983 10011 raft_consensus.cc:2272] T b97890a149bd42989ee95dd584d67569 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.496498 10011 tablet_replica.cc:333] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.497180 10011 raft_consensus.cc:2243] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.498201 10011 raft_consensus.cc:2272] T 08319e09aad5409ab029d052b6c799e9 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.500532 10011 tablet_replica.cc:333] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.501194 10011 raft_consensus.cc:2243] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.501833 10011 raft_consensus.cc:2272] T ad1ac780e48248fca8514e58b3abab9a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.505019 10011 tablet_replica.cc:333] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.505753 10011 raft_consensus.cc:2243] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.506343 10011 raft_consensus.cc:2272] T b9eb27af16f84101a04c28ae7696af3d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:21.509069 23432 consensus_peers.cc:597] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:37807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.509891 10011 tablet_replica.cc:333] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.510548 10011 raft_consensus.cc:2243] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.511109 10011 raft_consensus.cc:2272] T 763c17e836ff4a64b1caf17acc31bc0c P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.513370 10011 tablet_replica.cc:333] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.514026 10011 raft_consensus.cc:2243] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.514546 10011 raft_consensus.cc:2272] T 78eaeb31f661466e854e3eef1bea725a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.517284 10011 tablet_replica.cc:333] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.518054 10011 raft_consensus.cc:2243] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.519205 10011 raft_consensus.cc:2272] T 4eaaadad8ef849f5b67928865a4f97ec P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:21.520658 23432 consensus_peers.cc:597] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:37807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.522300 10011 tablet_replica.cc:333] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.523016 10011 raft_consensus.cc:2243] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.523597 10011 raft_consensus.cc:2272] T a7194c62978a4c2bba240ca41873449a P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.526031 10011 tablet_replica.cc:333] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.526567 10011 raft_consensus.cc:2243] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.527390 10011 raft_consensus.cc:2272] T 31fc1459c8b74e219270a982e3118f1d P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.529719 10011 tablet_replica.cc:333] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.530339 10011 raft_consensus.cc:2243] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.531275 10011 raft_consensus.cc:2272] T 210c6a9c1c5d486f8ace5a042a52bcca P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.533511 10011 tablet_replica.cc:333] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.534216 10011 raft_consensus.cc:2243] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.534683 10011 raft_consensus.cc:2272] T 8ebc3148c6444a9191346aaa181695bf P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.536769 10011 tablet_replica.cc:333] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.537279 10011 raft_consensus.cc:2243] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.537827 10011 raft_consensus.cc:2272] T 5dc3d871f84a4e1cbe6543d2864a93fd P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.540107 10011 tablet_replica.cc:333] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.540633 10011 raft_consensus.cc:2243] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.541512 10011 raft_consensus.cc:2272] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.543710 10011 tablet_replica.cc:333] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.544190 10011 raft_consensus.cc:2243] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.544627 10011 raft_consensus.cc:2272] T be0f2e3c754b43ed89096070a9d076a0 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.546867 10011 tablet_replica.cc:333] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.547377 10011 raft_consensus.cc:2243] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.547865 10011 raft_consensus.cc:2272] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.550132 10011 tablet_replica.cc:333] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.550608 10011 raft_consensus.cc:2243] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.551043 10011 raft_consensus.cc:2272] T 86eb26a3d46147719da1edd9dbd4270b P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.553313 10011 tablet_replica.cc:333] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.553906 10011 raft_consensus.cc:2243] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.554469 10011 raft_consensus.cc:2272] T 77d530be1ee246a3aa70928555d7b233 P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.556824 10011 tablet_replica.cc:333] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93: stopping tablet replica
I20260709 12:18:21.557375 10011 raft_consensus.cc:2243] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.557971 10011 raft_consensus.cc:2272] T 703a57dd01894388a102b7bfed51641e P 547d1ff7c5b94fbe93a7fa981f24ec93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.602897 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
I20260709 12:18:21.645990 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
W20260709 12:18:21.660400 23433 consensus_peers.cc:597] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 -> Peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Couldn't send request to peer 547d1ff7c5b94fbe93a7fa981f24ec93. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:36153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:21.679436 23433 consensus_peers.cc:597] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 -> Peer 547d1ff7c5b94fbe93a7fa981f24ec93 (127.9.198.194:36153): Couldn't send request to peer 547d1ff7c5b94fbe93a7fa981f24ec93. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.194:36153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.681458 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:21.682288 10011 tablet_replica.cc:333] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.683055 10011 raft_consensus.cc:2243] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.683689 10011 raft_consensus.cc:2272] T b97890a149bd42989ee95dd584d67569 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.686281 10011 tablet_replica.cc:333] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.686901 10011 raft_consensus.cc:2243] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.687438 10011 raft_consensus.cc:2272] T 08319e09aad5409ab029d052b6c799e9 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.689841 10011 tablet_replica.cc:333] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.690418 10011 raft_consensus.cc:2243] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.690944 10011 raft_consensus.cc:2272] T ad1ac780e48248fca8514e58b3abab9a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.693357 10011 tablet_replica.cc:333] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.693998 10011 raft_consensus.cc:2243] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.694541 10011 raft_consensus.cc:2272] T b9eb27af16f84101a04c28ae7696af3d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.696764 10011 tablet_replica.cc:333] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.697399 10011 raft_consensus.cc:2243] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.698328 10011 raft_consensus.cc:2272] T a7194c62978a4c2bba240ca41873449a P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.700649 10011 tablet_replica.cc:333] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.701339 10011 raft_consensus.cc:2243] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.702534 10011 raft_consensus.cc:2272] T 5dc3d871f84a4e1cbe6543d2864a93fd P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.705428 10011 tablet_replica.cc:333] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
W20260709 12:18:21.705938 23432 consensus_peers.cc:597] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 -> Peer 4b6e2a23c11644ec92cc018d8be502eb (127.9.198.193:37807): Couldn't send request to peer 4b6e2a23c11644ec92cc018d8be502eb. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:37807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:21.706244 10011 raft_consensus.cc:2243] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.706895 10011 raft_consensus.cc:2272] T 92b3b3a4266944f79320f1f2a4f1a0b4 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.709188 10011 tablet_replica.cc:333] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.709867 10011 raft_consensus.cc:2243] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.710743 10011 raft_consensus.cc:2272] T be0f2e3c754b43ed89096070a9d076a0 P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.713143 10011 tablet_replica.cc:333] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.713740 10011 raft_consensus.cc:2243] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.714568 10011 raft_consensus.cc:2272] T 763c17e836ff4a64b1caf17acc31bc0c P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.716789 10011 tablet_replica.cc:333] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.717312 10011 raft_consensus.cc:2243] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.718271 10011 raft_consensus.cc:2272] T 78eaeb31f661466e854e3eef1bea725a P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.720590 10011 tablet_replica.cc:333] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.721165 10011 raft_consensus.cc:2243] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.721698 10011 raft_consensus.cc:2272] T 4eaaadad8ef849f5b67928865a4f97ec P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.723953 10011 tablet_replica.cc:333] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.724500 10011 raft_consensus.cc:2243] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.724980 10011 raft_consensus.cc:2272] T 9e9a7b21432f4d01b20c9291f0cc9a43 P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.727404 10011 tablet_replica.cc:333] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.727972 10011 raft_consensus.cc:2243] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.728389 10011 raft_consensus.cc:2272] T 86eb26a3d46147719da1edd9dbd4270b P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.730789 10011 tablet_replica.cc:333] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.731305 10011 raft_consensus.cc:2243] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.731726 10011 raft_consensus.cc:2272] T 77d530be1ee246a3aa70928555d7b233 P 51a03c362557435daf969e2f7d5ba778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.733950 10011 tablet_replica.cc:333] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.734468 10011 raft_consensus.cc:2243] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.734925 10011 raft_consensus.cc:2272] T 31fc1459c8b74e219270a982e3118f1d P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.737069 10011 tablet_replica.cc:333] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.737707 10011 raft_consensus.cc:2243] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.738133 10011 raft_consensus.cc:2272] T 210c6a9c1c5d486f8ace5a042a52bcca P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.740464 10011 tablet_replica.cc:333] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.740952 10011 raft_consensus.cc:2243] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.741935 10011 raft_consensus.cc:2272] T 8ebc3148c6444a9191346aaa181695bf P 51a03c362557435daf969e2f7d5ba778 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.744333 10011 tablet_replica.cc:333] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778: stopping tablet replica
I20260709 12:18:21.744815 10011 raft_consensus.cc:2243] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:21.745313 10011 raft_consensus.cc:2272] T 703a57dd01894388a102b7bfed51641e P 51a03c362557435daf969e2f7d5ba778 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.792575 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:18:21.833019 10011 master.cc:562] Master@127.9.198.254:43235 shutting down...
I20260709 12:18:21.860301 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:21.861022 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:21.861330 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0ea09887abd84248b1cf41dd28c939cf: stopping tablet replica
I20260709 12:18:21.885377 10011 master.cc:584] Master@127.9.198.254:43235 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21213 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:18:21.942137 10011 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.198.254:46803
I20260709 12:18:21.943256 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:21.949085 23808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:21.949549 23809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:21.950577 23811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:21.950976 10011 server_base.cc:1061] running on GCE node
I20260709 12:18:21.952152 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:21.952404 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:21.952576 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599501952544 us; error 0 us; skew 500 ppm
I20260709 12:18:21.953122 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:21.955909 10011 webserver.cc:533] Webserver started at http://127.9.198.254:35079/ using document root <none> and password file <none>
I20260709 12:18:21.956426 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:21.956622 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:21.956914 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:21.958215 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/master-0-root/instance:
uuid: "ee3d396e148244428354d966418022cf"
format_stamp: "Formatted at 2026-07-09 12:18:21 on dist-test-slave-glnm"
I20260709 12:18:21.963179 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:18:21.966857 23816 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:21.967603 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:18:21.967929 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/master-0-root
uuid: "ee3d396e148244428354d966418022cf"
format_stamp: "Formatted at 2026-07-09 12:18:21 on dist-test-slave-glnm"
I20260709 12:18:21.968181 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:21.990062 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:21.991395 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:22.030840 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.254:46803
I20260709 12:18:22.030949 23867 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.254:46803 every 8 connection(s)
I20260709 12:18:22.035060 23868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.047510 23868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf: Bootstrap starting.
I20260709 12:18:22.052711 23868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.057369 23868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf: No bootstrap required, opened a new log
I20260709 12:18:22.059851 23868 raft_consensus.cc:359] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3d396e148244428354d966418022cf" member_type: VOTER }
I20260709 12:18:22.060312 23868 raft_consensus.cc:385] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.060534 23868 raft_consensus.cc:740] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3d396e148244428354d966418022cf, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.061149 23868 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3d396e148244428354d966418022cf" member_type: VOTER }
I20260709 12:18:22.061606 23868 raft_consensus.cc:399] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:18:22.061888 23868 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:18:22.062111 23868 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:22.067332 23868 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3d396e148244428354d966418022cf" member_type: VOTER }
I20260709 12:18:22.067845 23868 leader_election.cc:304] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee3d396e148244428354d966418022cf; no voters: 
I20260709 12:18:22.068913 23868 leader_election.cc:290] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:18:22.069435 23872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:22.070969 23872 raft_consensus.cc:697] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 1 LEADER]: Becoming Leader. State: Replica: ee3d396e148244428354d966418022cf, State: Running, Role: LEADER
I20260709 12:18:22.071743 23872 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3d396e148244428354d966418022cf" member_type: VOTER }
I20260709 12:18:22.072124 23868 sys_catalog.cc:565] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:18:22.077000 23874 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee3d396e148244428354d966418022cf. Latest consensus state: current_term: 1 leader_uuid: "ee3d396e148244428354d966418022cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3d396e148244428354d966418022cf" member_type: VOTER } }
I20260709 12:18:22.077131 23873 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee3d396e148244428354d966418022cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3d396e148244428354d966418022cf" member_type: VOTER } }
I20260709 12:18:22.077594 23874 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [sys.catalog]: This master's current role is: LEADER
I20260709 12:18:22.078008 23873 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [sys.catalog]: This master's current role is: LEADER
I20260709 12:18:22.084075 23880 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:18:22.089118 23880 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:18:22.091493 10011 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:18:22.099030 23880 catalog_manager.cc:1383] Generated new cluster ID: c0f336b3ce634128a2f8d28e8f3f98af
I20260709 12:18:22.099370 23880 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:18:22.127724 23880 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:18:22.129181 23880 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:18:22.143304 23880 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf: Generated new TSK 0
I20260709 12:18:22.143973 23880 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:18:22.158581 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:22.165483 23890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:22.165740 23891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:22.168931 23893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:22.169679 10011 server_base.cc:1061] running on GCE node
I20260709 12:18:22.170789 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:22.170979 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:22.171202 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599502171179 us; error 0 us; skew 500 ppm
I20260709 12:18:22.171779 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:22.174571 10011 webserver.cc:533] Webserver started at http://127.9.198.193:46471/ using document root <none> and password file <none>
I20260709 12:18:22.175104 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:22.175320 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:22.175592 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:22.176720 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-0-root/instance:
uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
format_stamp: "Formatted at 2026-07-09 12:18:22 on dist-test-slave-glnm"
I20260709 12:18:22.181591 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:18:22.185870 23898 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.186890 10011 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 12:18:22.187182 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-0-root
uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
format_stamp: "Formatted at 2026-07-09 12:18:22 on dist-test-slave-glnm"
I20260709 12:18:22.187441 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:22.206260 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:22.207533 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:22.209049 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:22.211581 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:22.211766 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.212077 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:22.212256 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.255601 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.193:41953
I20260709 12:18:22.255790 23960 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.193:41953 every 8 connection(s)
I20260709 12:18:22.261013 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:22.269613 23967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:22.275830 23966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:22.276387 23969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:22.278846 23961 heartbeater.cc:344] Connected to a master server at 127.9.198.254:46803
I20260709 12:18:22.279201 10011 server_base.cc:1061] running on GCE node
I20260709 12:18:22.279413 23961 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:22.280357 23961 heartbeater.cc:507] Master 127.9.198.254:46803 requested a full tablet report, sending...
I20260709 12:18:22.280766 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:22.281061 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:22.281282 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599502281261 us; error 0 us; skew 500 ppm
I20260709 12:18:22.282035 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:22.282845 23833 ts_manager.cc:194] Registered new tserver with Master: e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953)
I20260709 12:18:22.284802 10011 webserver.cc:533] Webserver started at http://127.9.198.194:36229/ using document root <none> and password file <none>
I20260709 12:18:22.285250 23833 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49072
I20260709 12:18:22.285581 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:22.285948 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:22.286253 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:22.287600 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-1-root/instance:
uuid: "a8c21fcb800b438b9df58d40c6853250"
format_stamp: "Formatted at 2026-07-09 12:18:22 on dist-test-slave-glnm"
I20260709 12:18:22.292639 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:18:22.296308 23974 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.297333 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:18:22.297611 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-1-root
uuid: "a8c21fcb800b438b9df58d40c6853250"
format_stamp: "Formatted at 2026-07-09 12:18:22 on dist-test-slave-glnm"
I20260709 12:18:22.297926 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:22.329101 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:22.330401 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:22.331903 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:22.334345 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:22.334599 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.334847 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:22.335027 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.380517 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.194:46607
I20260709 12:18:22.380621 24036 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.194:46607 every 8 connection(s)
I20260709 12:18:22.385792 10011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:22.392828 24040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:22.395094 24041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:22.398012 10011 server_base.cc:1061] running on GCE node
W20260709 12:18:22.398379 24043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:22.399632 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:22.399863 10011 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:22.400068 10011 hybrid_clock.cc:648] HybridClock initialized: now 1783599502400031 us; error 0 us; skew 500 ppm
I20260709 12:18:22.400856 10011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:22.403856 10011 webserver.cc:533] Webserver started at http://127.9.198.195:45669/ using document root <none> and password file <none>
I20260709 12:18:22.404068 24037 heartbeater.cc:344] Connected to a master server at 127.9.198.254:46803
I20260709 12:18:22.404484 10011 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:22.404601 24037 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:22.404819 10011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:22.405191 10011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:22.405616 24037 heartbeater.cc:507] Master 127.9.198.254:46803 requested a full tablet report, sending...
I20260709 12:18:22.406587 10011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-2-root/instance:
uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
format_stamp: "Formatted at 2026-07-09 12:18:22 on dist-test-slave-glnm"
I20260709 12:18:22.407848 23833 ts_manager.cc:194] Registered new tserver with Master: a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:22.409330 23833 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49088
I20260709 12:18:22.411818 10011 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:18:22.415815 24048 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.416656 10011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:18:22.417084 10011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-2-root
uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
format_stamp: "Formatted at 2026-07-09 12:18:22 on dist-test-slave-glnm"
I20260709 12:18:22.417407 10011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskwqrz4y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599049974374-10011-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:22.443802 10011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:22.445089 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:22.446792 10011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:22.449189 10011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:22.449429 10011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.449705 10011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:22.449924 10011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:22.493144 10011 rpc_server.cc:307] RPC server started. Bound to: 127.9.198.195:43695
I20260709 12:18:22.493251 24111 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.198.195:43695 every 8 connection(s)
I20260709 12:18:22.508246 24112 heartbeater.cc:344] Connected to a master server at 127.9.198.254:46803
I20260709 12:18:22.508809 24112 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:22.509861 24112 heartbeater.cc:507] Master 127.9.198.254:46803 requested a full tablet report, sending...
I20260709 12:18:22.512203 23833 ts_manager.cc:194] Registered new tserver with Master: 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:22.512781 10011 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016222671s
I20260709 12:18:22.513844 23833 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49096
I20260709 12:18:22.540453 23833 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49110:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:18:22.543435 23833 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:18:22.625074 24002 tablet_service.cc:1511] Processing CreateTablet for tablet 64f33ba99e41400b8d16dae902c15b22 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:22.627130 24002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64f33ba99e41400b8d16dae902c15b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.629339 24000 tablet_service.cc:1511] Processing CreateTablet for tablet 4bad77401a0647049e6f37a773c719f6 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:22.630916 24000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bad77401a0647049e6f37a773c719f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.630776 24001 tablet_service.cc:1511] Processing CreateTablet for tablet f8dc14c966494b2f930c926041c32bae (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:22.632284 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8dc14c966494b2f930c926041c32bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.636780 23926 tablet_service.cc:1511] Processing CreateTablet for tablet 64f33ba99e41400b8d16dae902c15b22 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:22.638432 23926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64f33ba99e41400b8d16dae902c15b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.639968 23999 tablet_service.cc:1511] Processing CreateTablet for tablet 14b7bfd5aa9844a685f6a3384f7f1d65 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:22.641531 23999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14b7bfd5aa9844a685f6a3384f7f1d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.642874 23925 tablet_service.cc:1511] Processing CreateTablet for tablet f8dc14c966494b2f930c926041c32bae (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:22.644390 23925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8dc14c966494b2f930c926041c32bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.646571 23998 tablet_service.cc:1511] Processing CreateTablet for tablet 15b60d1559cb4f40a2463a3d2ef322d4 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:22.646858 23924 tablet_service.cc:1511] Processing CreateTablet for tablet 4bad77401a0647049e6f37a773c719f6 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:22.648073 23998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b60d1559cb4f40a2463a3d2ef322d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.648388 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bad77401a0647049e6f37a773c719f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.652462 23923 tablet_service.cc:1511] Processing CreateTablet for tablet 14b7bfd5aa9844a685f6a3384f7f1d65 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:22.653151 23997 tablet_service.cc:1511] Processing CreateTablet for tablet d22599f2e4d04980829f7a39547e0117 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:22.653973 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14b7bfd5aa9844a685f6a3384f7f1d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.654621 23997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22599f2e4d04980829f7a39547e0117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.658861 23922 tablet_service.cc:1511] Processing CreateTablet for tablet 15b60d1559cb4f40a2463a3d2ef322d4 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:22.659535 23996 tablet_service.cc:1511] Processing CreateTablet for tablet be9fe9035a814d269d247bde6ab34d50 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:22.660323 23922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b60d1559cb4f40a2463a3d2ef322d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.660982 23996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9fe9035a814d269d247bde6ab34d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.663317 23921 tablet_service.cc:1511] Processing CreateTablet for tablet d22599f2e4d04980829f7a39547e0117 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:22.665318 23921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22599f2e4d04980829f7a39547e0117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.667907 23920 tablet_service.cc:1511] Processing CreateTablet for tablet be9fe9035a814d269d247bde6ab34d50 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:22.671492 23920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9fe9035a814d269d247bde6ab34d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.673043 23994 tablet_service.cc:1511] Processing CreateTablet for tablet 550bb56a01054b30a9262bb704bd1d2b (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:22.674592 23994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550bb56a01054b30a9262bb704bd1d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.669890 23919 tablet_service.cc:1511] Processing CreateTablet for tablet 6d3e13d378d74eab8e016be08fe8832f (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:22.666038 23995 tablet_service.cc:1511] Processing CreateTablet for tablet 6d3e13d378d74eab8e016be08fe8832f (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:22.678620 23995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d3e13d378d74eab8e016be08fe8832f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.678620 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d3e13d378d74eab8e016be08fe8832f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.687093 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 550bb56a01054b30a9262bb704bd1d2b (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:22.688668 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550bb56a01054b30a9262bb704bd1d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.700016 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 64f33ba99e41400b8d16dae902c15b22 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:22.701566 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64f33ba99e41400b8d16dae902c15b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.704116 24075 tablet_service.cc:1511] Processing CreateTablet for tablet f8dc14c966494b2f930c926041c32bae (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:22.705687 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8dc14c966494b2f930c926041c32bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.711509 24074 tablet_service.cc:1511] Processing CreateTablet for tablet 4bad77401a0647049e6f37a773c719f6 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:22.713162 24074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bad77401a0647049e6f37a773c719f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.720264 24073 tablet_service.cc:1511] Processing CreateTablet for tablet 14b7bfd5aa9844a685f6a3384f7f1d65 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:22.721992 24073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14b7bfd5aa9844a685f6a3384f7f1d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.727474 24072 tablet_service.cc:1511] Processing CreateTablet for tablet 15b60d1559cb4f40a2463a3d2ef322d4 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:22.728334 24070 tablet_service.cc:1511] Processing CreateTablet for tablet be9fe9035a814d269d247bde6ab34d50 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:22.729182 24072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b60d1559cb4f40a2463a3d2ef322d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.731091 24070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be9fe9035a814d269d247bde6ab34d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.732522 24071 tablet_service.cc:1511] Processing CreateTablet for tablet d22599f2e4d04980829f7a39547e0117 (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:22.734081 24071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22599f2e4d04980829f7a39547e0117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.735000 24068 tablet_service.cc:1511] Processing CreateTablet for tablet 550bb56a01054b30a9262bb704bd1d2b (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:22.736507 24068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 550bb56a01054b30a9262bb704bd1d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.738953 24069 tablet_service.cc:1511] Processing CreateTablet for tablet 6d3e13d378d74eab8e016be08fe8832f (DEFAULT_TABLE table=test-workload [id=3e28f91a52de43cfad7e134f469e727e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:22.740463 24069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d3e13d378d74eab8e016be08fe8832f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:22.767050 24127 tablet_bootstrap.cc:492] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:22.803169 24129 tablet_bootstrap.cc:492] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:22.809929 24129 tablet_bootstrap.cc:654] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.866428 24128 tablet_bootstrap.cc:492] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:22.873148 24127 tablet_bootstrap.cc:654] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.874114 24128 tablet_bootstrap.cc:654] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.884748 24129 tablet_bootstrap.cc:492] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:22.885185 24127 tablet_bootstrap.cc:492] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:22.885407 24129 ts_tablet_manager.cc:1403] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.083s	user 0.021s	sys 0.040s
I20260709 12:18:22.885792 24127 ts_tablet_manager.cc:1403] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.119s	user 0.016s	sys 0.000s
I20260709 12:18:22.886999 24128 tablet_bootstrap.cc:492] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:22.887502 24128 ts_tablet_manager.cc:1403] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.004s
I20260709 12:18:22.888593 24129 raft_consensus.cc:359] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.888916 24127 raft_consensus.cc:359] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.889744 24129 raft_consensus.cc:385] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.889781 24127 raft_consensus.cc:385] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.890040 24128 raft_consensus.cc:359] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.890275 24129 raft_consensus.cc:740] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.890347 24127 raft_consensus.cc:740] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.890900 24128 raft_consensus.cc:385] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.891281 24128 raft_consensus.cc:740] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.891476 24129 consensus_queue.cc:260] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.891589 24127 consensus_queue.cc:260] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.892052 24128 consensus_queue.cc:260] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.894644 24037 heartbeater.cc:499] Master 127.9.198.254:46803 was elected leader, sending a full tablet report...
I20260709 12:18:22.895246 24127 ts_tablet_manager.cc:1434] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:18:22.896015 23961 heartbeater.cc:499] Master 127.9.198.254:46803 was elected leader, sending a full tablet report...
I20260709 12:18:22.896472 24127 tablet_bootstrap.cc:492] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:22.904417 24127 tablet_bootstrap.cc:654] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.906061 24112 heartbeater.cc:499] Master 127.9.198.254:46803 was elected leader, sending a full tablet report...
I20260709 12:18:22.906590 24128 ts_tablet_manager.cc:1434] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.010s
I20260709 12:18:22.907660 24128 tablet_bootstrap.cc:492] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:22.914712 24128 tablet_bootstrap.cc:654] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.917110 24129 ts_tablet_manager.cc:1434] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 12:18:22.921259 24129 tablet_bootstrap.cc:492] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:22.931002 24129 tablet_bootstrap.cc:654] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.934986 24127 tablet_bootstrap.cc:492] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:22.935814 24127 ts_tablet_manager.cc:1403] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.013s
I20260709 12:18:22.939242 24127 raft_consensus.cc:359] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.940109 24127 raft_consensus.cc:385] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.940456 24127 raft_consensus.cc:740] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.941403 24127 consensus_queue.cc:260] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.944284 24127 ts_tablet_manager.cc:1434] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:18:22.945515 24127 tablet_bootstrap.cc:492] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:22.953227 24127 tablet_bootstrap.cc:654] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.956812 24129 tablet_bootstrap.cc:492] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:22.957485 24129 ts_tablet_manager.cc:1403] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.019s
I20260709 12:18:22.958043 24128 tablet_bootstrap.cc:492] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:22.958202 24133 raft_consensus.cc:493] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.958961 24128 ts_tablet_manager.cc:1403] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.052s	user 0.030s	sys 0.021s
I20260709 12:18:22.958815 24133 raft_consensus.cc:515] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.960846 24129 raft_consensus.cc:359] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.961719 24129 raft_consensus.cc:385] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.962029 24129 raft_consensus.cc:740] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.963032 24129 consensus_queue.cc:260] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.963905 24128 raft_consensus.cc:359] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.965255 24128 raft_consensus.cc:385] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.965740 24129 ts_tablet_manager.cc:1434] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:18:22.965965 24133 leader_election.cc:290] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:22.966913 24129 tablet_bootstrap.cc:492] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:22.965811 24128 raft_consensus.cc:740] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.970870 24128 consensus_queue.cc:260] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.980278 24128 ts_tablet_manager.cc:1434] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 12:18:22.981511 24128 tablet_bootstrap.cc:492] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:22.981946 24129 tablet_bootstrap.cc:654] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.982038 24127 tablet_bootstrap.cc:492] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:22.985941 24127 ts_tablet_manager.cc:1403] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.017s
I20260709 12:18:22.989404 24128 tablet_bootstrap.cc:654] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:22.989238 24127 raft_consensus.cc:359] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.990046 24127 raft_consensus.cc:385] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:22.990387 24127 raft_consensus.cc:740] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:22.991292 24127 consensus_queue.cc:260] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:22.993615 24127 ts_tablet_manager.cc:1434] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:18:22.996991 24127 tablet_bootstrap.cc:492] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
W20260709 12:18:22.997547 24113 tablet.cc:2406] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:18:23.002257 24128 tablet_bootstrap.cc:492] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:23.002959 24128 ts_tablet_manager.cc:1403] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.010s
I20260709 12:18:23.003893 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bad77401a0647049e6f37a773c719f6" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
W20260709 12:18:23.006131 23975 leader_election.cc:343] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.006901 24128 raft_consensus.cc:359] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.007908 24128 raft_consensus.cc:385] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.008256 24128 raft_consensus.cc:740] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.009086 24128 consensus_queue.cc:260] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.009609 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bad77401a0647049e6f37a773c719f6" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.012130 24128 ts_tablet_manager.cc:1434] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
W20260709 12:18:23.013006 23976 leader_election.cc:343] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.013410 24128 tablet_bootstrap.cc:492] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:23.013813 23976 leader_election.cc:304] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.014853 24133 raft_consensus.cc:2749] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:23.021154 24127 tablet_bootstrap.cc:654] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.021196 24128 tablet_bootstrap.cc:654] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.028826 24129 tablet_bootstrap.cc:492] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:23.029558 24129 ts_tablet_manager.cc:1403] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.022s
I20260709 12:18:23.044842 24127 tablet_bootstrap.cc:492] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:23.045728 24127 ts_tablet_manager.cc:1403] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.016s
I20260709 12:18:23.045521 24129 raft_consensus.cc:359] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.044842 24128 tablet_bootstrap.cc:492] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:23.046445 24129 raft_consensus.cc:385] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.047056 24129 raft_consensus.cc:740] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.047062 24128 ts_tablet_manager.cc:1403] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.034s	user 0.005s	sys 0.021s
I20260709 12:18:23.047952 24129 consensus_queue.cc:260] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.048983 24127 raft_consensus.cc:359] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.049767 24127 raft_consensus.cc:385] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.050081 24127 raft_consensus.cc:740] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.050266 24129 ts_tablet_manager.cc:1434] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.003s
I20260709 12:18:23.050647 24128 raft_consensus.cc:359] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.050966 24127 consensus_queue.cc:260] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.051438 24128 raft_consensus.cc:385] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.051642 24129 tablet_bootstrap.cc:492] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:23.051824 24128 raft_consensus.cc:740] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.053158 24135 raft_consensus.cc:493] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.053946 24127 ts_tablet_manager.cc:1434] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.006s
I20260709 12:18:23.053756 24135 raft_consensus.cc:515] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.055294 24127 tablet_bootstrap.cc:492] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:23.058792 24134 raft_consensus.cc:493] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.059746 24129 tablet_bootstrap.cc:654] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.059443 24134 raft_consensus.cc:515] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.063067 24127 tablet_bootstrap.cc:654] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.064996 24135 leader_election.cc:290] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.070844 24134 leader_election.cc:290] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.073201 24129 tablet_bootstrap.cc:492] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:23.073916 24129 ts_tablet_manager.cc:1403] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.005s
I20260709 12:18:23.077086 24128 consensus_queue.cc:260] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.080166 24129 raft_consensus.cc:359] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.081317 24127 tablet_bootstrap.cc:492] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:23.081466 24129 raft_consensus.cc:385] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.082175 24129 raft_consensus.cc:740] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.082191 24127 ts_tablet_manager.cc:1403] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.003s
I20260709 12:18:23.083247 24129 consensus_queue.cc:260] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.086042 24127 raft_consensus.cc:359] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.086625 24129 ts_tablet_manager.cc:1434] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:18:23.086987 24127 raft_consensus.cc:385] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.087450 24127 raft_consensus.cc:740] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.088115 24129 tablet_bootstrap.cc:492] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
W20260709 12:18:23.088723 23882 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:23.088469 24127 consensus_queue.cc:260] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.089320 24128 ts_tablet_manager.cc:1434] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 12:18:23.091778 24127 ts_tablet_manager.cc:1434] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:18:23.093834 24128 tablet_bootstrap.cc:492] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:23.096741 24129 tablet_bootstrap.cc:654] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.105129 24127 tablet_bootstrap.cc:492] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:23.105589 24128 tablet_bootstrap.cc:654] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.113819 24129 tablet_bootstrap.cc:492] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:23.114565 24129 ts_tablet_manager.cc:1403] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.012s
I20260709 12:18:23.117318 24128 tablet_bootstrap.cc:492] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:23.118100 24129 raft_consensus.cc:359] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.119119 24129 raft_consensus.cc:385] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.119462 24129 raft_consensus.cc:740] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.120311 24129 consensus_queue.cc:260] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.118103 24128 ts_tablet_manager.cc:1403] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.003s
I20260709 12:18:23.124352 24128 raft_consensus.cc:359] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.124940 24128 raft_consensus.cc:385] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.125263 24128 raft_consensus.cc:740] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.126045 24128 consensus_queue.cc:260] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.127532 24129 ts_tablet_manager.cc:1434] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:18:23.128427 24128 ts_tablet_manager.cc:1434] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 12:18:23.128834 24129 tablet_bootstrap.cc:492] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:23.129549 24128 tablet_bootstrap.cc:492] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:23.131379 24135 raft_consensus.cc:493] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.131990 24135 raft_consensus.cc:515] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.134684 24135 leader_election.cc:290] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.136831 24129 tablet_bootstrap.cc:654] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.136953 24128 tablet_bootstrap.cc:654] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.150758 24128 tablet_bootstrap.cc:492] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:23.151341 24128 ts_tablet_manager.cc:1403] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.010s
I20260709 12:18:23.152274 24135 raft_consensus.cc:493] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.153115 24129 tablet_bootstrap.cc:492] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:23.152936 24135 raft_consensus.cc:515] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.153743 24129 ts_tablet_manager.cc:1403] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.008s
I20260709 12:18:23.154695 24128 raft_consensus.cc:359] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.155534 24128 raft_consensus.cc:385] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.155608 24135 leader_election.cc:290] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.155998 24128 raft_consensus.cc:740] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.157024 24128 consensus_queue.cc:260] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.159673 24128 ts_tablet_manager.cc:1434] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:18:23.160804 24128 tablet_bootstrap.cc:492] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:23.157095 24129 raft_consensus.cc:359] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.161582 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.162766 24129 raft_consensus.cc:385] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.163143 24129 raft_consensus.cc:740] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.164045 24129 consensus_queue.cc:260] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
W20260709 12:18:23.164655 23900 leader_election.cc:343] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.166951 24129 ts_tablet_manager.cc:1434] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:18:23.167773 24128 tablet_bootstrap.cc:654] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.168323 24129 tablet_bootstrap.cc:492] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:23.176503 24133 raft_consensus.cc:493] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.177418 24133 raft_consensus.cc:515] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.179950 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.180756 24086 raft_consensus.cc:2468] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:23.182571 24133 leader_election.cc:290] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.182904 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8dc14c966494b2f930c926041c32bae" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.183866 23900 leader_election.cc:304] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:23.186312 24134 raft_consensus.cc:2804] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.186878 24134 raft_consensus.cc:493] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.187343 24134 raft_consensus.cc:3060] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.188531 24127 tablet_bootstrap.cc:654] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
W20260709 12:18:23.190410 23975 leader_election.cc:343] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.191555 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8dc14c966494b2f930c926041c32bae" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.193791 24133 raft_consensus.cc:493] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.195976 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.197255 23936 raft_consensus.cc:2468] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:23.197012 24133 raft_consensus.cc:515] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.203106 24129 tablet_bootstrap.cc:654] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.205189 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.206010 23936 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 0.
I20260709 12:18:23.207613 23975 leader_election.cc:304] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
W20260709 12:18:23.208806 23976 leader_election.cc:343] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.210419 23976 leader_election.cc:304] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.213315 24127 tablet_bootstrap.cc:492] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:23.213944 24133 leader_election.cc:290] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.214105 24127 ts_tablet_manager.cc:1403] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.109s	user 0.020s	sys 0.006s
I20260709 12:18:23.214761 24133 raft_consensus.cc:493] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.215420 24133 raft_consensus.cc:515] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.218715 24127 raft_consensus.cc:359] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.219902 24127 raft_consensus.cc:385] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.220520 24133 leader_election.cc:290] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.221367 24133 raft_consensus.cc:2804] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.221848 24133 raft_consensus.cc:493] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.222294 24133 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.222530 24161 raft_consensus.cc:2749] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:23.220294 24127 raft_consensus.cc:740] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.225246 24127 consensus_queue.cc:260] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.227074 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.227902 24127 ts_tablet_manager.cc:1434] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:18:23.228843 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.229048 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.229439 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
W20260709 12:18:23.228555 23975 leader_election.cc:343] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.230213 24011 raft_consensus.cc:2468] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:23.231078 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.231439 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.231650 24049 leader_election.cc:304] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.232687 24135 raft_consensus.cc:2804] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.233053 24135 raft_consensus.cc:493] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.233397 24135 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.234614 24129 tablet_bootstrap.cc:492] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:23.235162 24129 ts_tablet_manager.cc:1403] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.020s
I20260709 12:18:23.237056 24127 tablet_bootstrap.cc:492] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
W20260709 12:18:23.237960 24050 leader_election.cc:343] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.238409 24129 raft_consensus.cc:359] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.239260 24129 raft_consensus.cc:385] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.239558 24129 raft_consensus.cc:740] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.239912 24133 raft_consensus.cc:515] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.241113 24162 raft_consensus.cc:493] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.241726 24162 raft_consensus.cc:515] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.241940 24050 leader_election.cc:304] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:23.243326 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.244026 23935 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.246059 24129 consensus_queue.cc:260] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.246856 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.247648 23934 raft_consensus.cc:2468] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 0.
I20260709 12:18:23.248315 24134 raft_consensus.cc:515] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.248742 24162 leader_election.cc:290] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.249398 23975 leader_election.cc:304] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.251946 24162 raft_consensus.cc:2804] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.252102 23936 raft_consensus.cc:2393] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0cbb7c136dd3451c93deee0d3fd64be6 in current term 1: Already voted for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in this term.
I20260709 12:18:23.252388 24162 raft_consensus.cc:493] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.252837 24162 raft_consensus.cc:3060] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.250171 24135 raft_consensus.cc:515] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.259732 24127 tablet_bootstrap.cc:654] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.261229 23935 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 1.
I20260709 12:18:23.261703 24134 leader_election.cc:290] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
W20260709 12:18:23.262539 24049 leader_election.cc:343] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.263288 24128 tablet_bootstrap.cc:492] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:23.265956 23975 leader_election.cc:304] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.270310 24128 ts_tablet_manager.cc:1403] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.110s	user 0.026s	sys 0.016s
I20260709 12:18:23.266443 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.270884 24161 raft_consensus.cc:2804] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.271051 23936 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.271656 24161 raft_consensus.cc:697] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:23.272790 24161 consensus_queue.cc:237] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.275766 24128 raft_consensus.cc:359] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.276621 24128 raft_consensus.cc:385] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.263211 24163 raft_consensus.cc:2804] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.265690 24162 raft_consensus.cc:515] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.277562 24163 raft_consensus.cc:493] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:23.262897 24050 leader_election.cc:343] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.278041 24163 raft_consensus.cc:3060] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.280356 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.265017 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.282161 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.280861 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.283452 24086 raft_consensus.cc:3060] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.283329 24083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.284132 23936 raft_consensus.cc:2468] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
W20260709 12:18:23.284725 23900 leader_election.cc:343] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.285017 24133 leader_election.cc:290] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.265331 24129 ts_tablet_manager.cc:1434] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.006s
I20260709 12:18:23.281363 24085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.286545 24085 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 0.
I20260709 12:18:23.286969 24129 tablet_bootstrap.cc:492] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:23.266894 24135 leader_election.cc:290] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.289419 24163 raft_consensus.cc:515] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.290050 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.291823 24163 leader_election.cc:290] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.292407 23935 raft_consensus.cc:3060] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.292701 24084 raft_consensus.cc:2393] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8c21fcb800b438b9df58d40c6853250 in current term 1: Already voted for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in this term.
I20260709 12:18:23.294310 24129 tablet_bootstrap.cc:654] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.294992 24086 raft_consensus.cc:2468] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:23.295300 24049 leader_election.cc:304] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6; no voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.297405 24049 leader_election.cc:304] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.298234 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.276957 24128 raft_consensus.cc:740] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.298996 23936 raft_consensus.cc:3060] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.298566 24135 raft_consensus.cc:2749] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:23.300266 24135 raft_consensus.cc:2804] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.300801 24135 raft_consensus.cc:697] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
W20260709 12:18:23.302266 23976 leader_election.cc:343] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:18:23.301905 24050 leader_election.cc:343] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.304996 23976 leader_election.cc:304] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.302978 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.306617 24010 raft_consensus.cc:3060] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.301584 23900 leader_election.cc:304] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:23.305619 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.311015 24086 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.311062 24134 raft_consensus.cc:2804] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.312016 24134 raft_consensus.cc:697] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:23.313032 24134 consensus_queue.cc:237] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.301774 24135 consensus_queue.cc:237] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.303139 24128 consensus_queue.cc:260] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.317917 24162 leader_election.cc:290] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.320226 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.320748 24128 ts_tablet_manager.cc:1434] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.001s
I20260709 12:18:23.310195 24127 tablet_bootstrap.cc:492] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:23.321574 23936 raft_consensus.cc:2468] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:23.323397 23832 catalog_manager.cc:5697] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:23.323089 23976 leader_election.cc:343] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.326522 24163 raft_consensus.cc:493] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.326309 24049 leader_election.cc:304] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.324611 24128 tablet_bootstrap.cc:492] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:23.330956 24163 raft_consensus.cc:515] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.335687 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550bb56a01054b30a9262bb704bd1d2b" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.324004 24127 ts_tablet_manager.cc:1403] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.012s
I20260709 12:18:23.336377 24011 raft_consensus.cc:2468] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:23.337116 24128 tablet_bootstrap.cc:654] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.337918 24050 leader_election.cc:304] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:23.355327 24010 raft_consensus.cc:2468] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:23.355962 24133 raft_consensus.cc:2749] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:23.347185 24135 raft_consensus.cc:2804] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.357766 24086 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 1.
I20260709 12:18:23.348789 23829 catalog_manager.cc:5697] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.351641 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550bb56a01054b30a9262bb704bd1d2b" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.361976 24129 tablet_bootstrap.cc:492] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:23.353613 24128 tablet_bootstrap.cc:492] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:23.340534 24163 leader_election.cc:290] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
W20260709 12:18:23.363163 24049 leader_election.cc:343] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.363262 24163 raft_consensus.cc:2804] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.363354 24128 ts_tablet_manager.cc:1403] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.011s
I20260709 12:18:23.363945 24163 raft_consensus.cc:493] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.364355 24163 raft_consensus.cc:3060] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.367522 24128 raft_consensus.cc:359] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.369349 24128 raft_consensus.cc:385] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.369767 24128 raft_consensus.cc:740] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.364388 24129 ts_tablet_manager.cc:1403] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.078s	user 0.036s	sys 0.031s
I20260709 12:18:23.367635 23833 catalog_manager.cc:5697] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:23.370895 24128 consensus_queue.cc:260] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.337622 23935 raft_consensus.cc:2468] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 1.
I20260709 12:18:23.372992 24129 raft_consensus.cc:359] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.373383 24128 ts_tablet_manager.cc:1434] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:18:23.373790 24129 raft_consensus.cc:385] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.374116 24129 raft_consensus.cc:740] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.374348 23975 leader_election.cc:304] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 0cbb7c136dd3451c93deee0d3fd64be6
I20260709 12:18:23.374816 24129 consensus_queue.cc:260] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.376034 24133 raft_consensus.cc:2804] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.376672 24133 raft_consensus.cc:697] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:23.377269 24129 ts_tablet_manager.cc:1434] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:18:23.378324 24129 tablet_bootstrap.cc:492] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:23.377821 24133 consensus_queue.cc:237] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.374765 24128 tablet_bootstrap.cc:492] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:23.368031 24135 raft_consensus.cc:697] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:23.352006 24127 raft_consensus.cc:359] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.385448 24127 raft_consensus.cc:385] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.385939 24127 raft_consensus.cc:740] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.387068 24127 consensus_queue.cc:260] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.390108 24127 ts_tablet_manager.cc:1434] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.004s
I20260709 12:18:23.391577 24127 tablet_bootstrap.cc:492] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:23.392791 24129 tablet_bootstrap.cc:654] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.394061 24128 tablet_bootstrap.cc:654] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.395022 24135 consensus_queue.cc:237] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.422959 24174 raft_consensus.cc:493] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.423872 24174 raft_consensus.cc:515] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.426690 24134 raft_consensus.cc:493] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.427522 24134 raft_consensus.cc:515] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.428578 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.429718 23935 raft_consensus.cc:2468] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:23.430709 24134 leader_election.cc:290] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.433172 24049 leader_election.cc:304] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.433240 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8dc14c966494b2f930c926041c32bae" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.437978 24010 raft_consensus.cc:2468] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:23.436141 24135 raft_consensus.cc:2804] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.439385 24135 raft_consensus.cc:493] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.439759 24135 raft_consensus.cc:3060] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.440848 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8dc14c966494b2f930c926041c32bae" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.441915 24086 raft_consensus.cc:2468] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:23.443854 23900 leader_election.cc:304] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.445335 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.447189 24010 raft_consensus.cc:2468] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:23.446349 24134 raft_consensus.cc:2804] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.449606 24174 leader_election.cc:290] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.450466 24134 raft_consensus.cc:493] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.451035 24134 raft_consensus.cc:3060] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.451352 23832 catalog_manager.cc:5697] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 1 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.447190 24127 tablet_bootstrap.cc:654] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.450393 24163 raft_consensus.cc:515] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.461365 24134 raft_consensus.cc:515] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.461619 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550bb56a01054b30a9262bb704bd1d2b" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
W20260709 12:18:23.464636 24049 leader_election.cc:343] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:23.465685 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "550bb56a01054b30a9262bb704bd1d2b" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.466660 24010 raft_consensus.cc:3060] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.475379 24134 leader_election.cc:290] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.475425 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8dc14c966494b2f930c926041c32bae" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.476504 24011 raft_consensus.cc:3060] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.477507 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8dc14c966494b2f930c926041c32bae" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.478204 24086 raft_consensus.cc:3060] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.484539 23832 catalog_manager.cc:5697] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:23.492295 24163 leader_election.cc:290] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.494293 24086 raft_consensus.cc:2468] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:23.495949 23900 leader_election.cc:304] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.497028 24135 raft_consensus.cc:515] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.495841 24128 tablet_bootstrap.cc:492] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:23.498617 24128 ts_tablet_manager.cc:1403] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.124s	user 0.019s	sys 0.029s
I20260709 12:18:23.500334 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.500710 24135 leader_election.cc:290] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.501123 23935 raft_consensus.cc:3060] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.502144 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.502825 24012 raft_consensus.cc:3060] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.503032 24128 raft_consensus.cc:359] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.503532 24128 raft_consensus.cc:385] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.503700 24128 raft_consensus.cc:740] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.504287 24128 consensus_queue.cc:260] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.505720 24128 ts_tablet_manager.cc:1434] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:18:23.497473 24134 raft_consensus.cc:2804] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.501159 24011 raft_consensus.cc:2468] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:23.511989 24129 tablet_bootstrap.cc:492] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:23.512614 24129 ts_tablet_manager.cc:1403] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.134s	user 0.028s	sys 0.083s
I20260709 12:18:23.513713 24127 tablet_bootstrap.cc:492] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:23.514281 24127 ts_tablet_manager.cc:1403] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.123s	user 0.010s	sys 0.004s
I20260709 12:18:23.515754 24129 raft_consensus.cc:359] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.516533 24129 raft_consensus.cc:385] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.516803 24129 raft_consensus.cc:740] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.518501 24010 raft_consensus.cc:2468] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:23.517488 24129 consensus_queue.cc:260] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.509708 24134 raft_consensus.cc:697] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:23.520773 24127 raft_consensus.cc:359] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.521216 24129 ts_tablet_manager.cc:1434] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:18:23.521081 24050 leader_election.cc:304] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.521454 24127 raft_consensus.cc:385] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.521905 24127 raft_consensus.cc:740] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.522102 24135 raft_consensus.cc:2804] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.522662 24135 raft_consensus.cc:697] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:23.522805 24127 consensus_queue.cc:260] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.523525 24135 consensus_queue.cc:237] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.524966 24127 ts_tablet_manager.cc:1434] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:18:23.526144 24012 raft_consensus.cc:2468] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:23.528029 23935 raft_consensus.cc:2468] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:23.529722 24049 leader_election.cc:304] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.520773 24134 consensus_queue.cc:237] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.530658 24163 raft_consensus.cc:2804] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.531193 24163 raft_consensus.cc:697] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:23.532068 24163 consensus_queue.cc:237] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.536818 24127 tablet_bootstrap.cc:492] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:23.545392 24127 tablet_bootstrap.cc:654] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:23.547415 23829 catalog_manager.cc:5697] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:23.550225 24127 tablet_bootstrap.cc:492] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:23.549531 23829 catalog_manager.cc:5697] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:23.550801 24127 ts_tablet_manager.cc:1403] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 12:18:23.551172 23832 catalog_manager.cc:5697] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.553082 24127 raft_consensus.cc:359] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.553701 24127 raft_consensus.cc:385] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:23.553968 24127 raft_consensus.cc:740] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:23.554586 24127 consensus_queue.cc:260] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.556423 24127 ts_tablet_manager.cc:1434] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:18:23.570297 24134 raft_consensus.cc:493] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.570874 24134 raft_consensus.cc:515] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.573011 24134 leader_election.cc:290] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.573822 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bad77401a0647049e6f37a773c719f6" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.574080 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bad77401a0647049e6f37a773c719f6" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.574472 24012 raft_consensus.cc:2468] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:23.574626 24086 raft_consensus.cc:2468] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:23.575460 23900 leader_election.cc:304] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.576754 24134 raft_consensus.cc:2804] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:23.577207 24134 raft_consensus.cc:493] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.577571 24134 raft_consensus.cc:3060] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.578927 24133 raft_consensus.cc:493] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.579411 24133 raft_consensus.cc:515] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.581172 24133 leader_election.cc:290] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.582180 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.582396 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.583261 24086 raft_consensus.cc:2393] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8c21fcb800b438b9df58d40c6853250 in current term 1: Already voted for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in this term.
I20260709 12:18:23.584539 23975 leader_election.cc:304] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.585565 24133 raft_consensus.cc:3060] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.585503 24134 raft_consensus.cc:515] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.587343 24134 leader_election.cc:290] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.588238 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bad77401a0647049e6f37a773c719f6" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.588896 24012 raft_consensus.cc:3060] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.588830 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bad77401a0647049e6f37a773c719f6" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.589576 24086 raft_consensus.cc:3060] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.592619 24133 raft_consensus.cc:2749] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:23.597201 24012 raft_consensus.cc:2468] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:23.597203 24086 raft_consensus.cc:2468] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:23.598918 23900 leader_election.cc:304] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.599709 24134 raft_consensus.cc:2804] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:23.600189 24134 raft_consensus.cc:697] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:23.600976 24134 consensus_queue.cc:237] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.608274 23833 catalog_manager.cc:5697] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.634794 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
mode: GRACEFUL
new_leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:38238
I20260709 12:18:23.635449 24012 raft_consensus.cc:606] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Received request to transfer leadership to TS e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.647429 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
mode: GRACEFUL
new_leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:38238
I20260709 12:18:23.648025 24012 raft_consensus.cc:606] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Received request to transfer leadership to TS e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.662859 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:23.663477 24086 raft_consensus.cc:606] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:23.676775 23935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:45020
I20260709 12:18:23.677484 23935 raft_consensus.cc:606] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:23.683374 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:23.683885 24086 raft_consensus.cc:606] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:23.689237 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:23.689926 24086 raft_consensus.cc:606] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:23.743396 24134 consensus_queue.cc:1048] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:23.758553 24134 consensus_queue.cc:1048] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:23.763569 24133 raft_consensus.cc:493] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.764088 24133 raft_consensus.cc:515] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.767750 24133 leader_election.cc:290] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.770530 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.771514 23935 raft_consensus.cc:2393] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a8c21fcb800b438b9df58d40c6853250 in current term 1: Already voted for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in this term.
I20260709 12:18:23.771342 24133 consensus_queue.cc:1048] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.773682 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:23.775561 23976 leader_election.cc:304] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.776970 24133 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.782094 24134 raft_consensus.cc:993] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.782788 24134 raft_consensus.cc:1081] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.784497 24163 consensus_queue.cc:1048] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:23.786563 24133 raft_consensus.cc:2749] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:23.787249 24012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38234
I20260709 12:18:23.788100 24012 raft_consensus.cc:493] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:23.788609 24012 raft_consensus.cc:3060] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.798877 24012 raft_consensus.cc:515] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.800539 24133 consensus_queue.cc:1048] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.802053 24135 consensus_queue.cc:1048] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.803927 24012 leader_election.cc:290] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.807235 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.807905 23934 raft_consensus.cc:3055] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:23.808276 23934 raft_consensus.cc:740] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:23.809120 23934 consensus_queue.cc:260] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.811415 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22599f2e4d04980829f7a39547e0117" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.812114 23934 raft_consensus.cc:3060] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.812500 24084 raft_consensus.cc:3060] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.824023 23934 raft_consensus.cc:2468] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:23.825438 23975 leader_election.cc:304] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.828122 24162 raft_consensus.cc:993] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250: : Instructing follower e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:23.828856 24162 raft_consensus.cc:1081] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Signalling peer e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:23.826843 24133 raft_consensus.cc:2804] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:23.830816 23936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:45004
I20260709 12:18:23.831410 23936 raft_consensus.cc:493] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:23.831800 23936 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.833024 24174 raft_consensus.cc:993] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.833563 24174 raft_consensus.cc:1081] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.835150 24133 raft_consensus.cc:697] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:23.836172 24133 consensus_queue.cc:237] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.836859 24011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38232
I20260709 12:18:23.837539 24011 raft_consensus.cc:493] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:23.837970 24011 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.841251 23936 raft_consensus.cc:515] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.843675 23936 leader_election.cc:290] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.845467 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.845862 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.846199 24010 raft_consensus.cc:3055] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:23.846627 24086 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.846897 24084 raft_consensus.cc:2468] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:23.846626 24010 raft_consensus.cc:740] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:23.848747 24010 consensus_queue.cc:260] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.850281 24010 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.851038 24135 consensus_queue.cc:1048] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.846822 24011 raft_consensus.cc:515] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.856298 24011 leader_election.cc:290] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.857595 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.858639 23936 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.860293 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.861958 24084 raft_consensus.cc:3055] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:23.862979 24084 raft_consensus.cc:740] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:23.864527 24084 consensus_queue.cc:260] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.864619 23829 catalog_manager.cc:5697] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 1 to 2, leader changed from e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.862722 24086 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:23.868335 24084 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.868693 23900 leader_election.cc:304] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.869244 23936 raft_consensus.cc:2468] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:23.870342 24134 raft_consensus.cc:2804] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:23.870833 23975 leader_election.cc:304] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.871948 24133 raft_consensus.cc:2804] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:23.872438 24133 raft_consensus.cc:697] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:23.870913 24134 raft_consensus.cc:697] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:23.873292 24133 consensus_queue.cc:237] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.874552 24134 consensus_queue.cc:237] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.875460 24135 raft_consensus.cc:993] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.875571 24010 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:23.876032 24135 raft_consensus.cc:1081] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.878656 24163 raft_consensus.cc:493] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.879165 24163 raft_consensus.cc:515] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.881407 24163 leader_election.cc:290] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:23.882637 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:23.883665 23936 raft_consensus.cc:2393] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0cbb7c136dd3451c93deee0d3fd64be6 in current term 1: Already voted for candidate a8c21fcb800b438b9df58d40c6853250 in this term.
I20260709 12:18:23.885468 24012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38232
I20260709 12:18:23.886363 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:23.887516 24086 raft_consensus.cc:3060] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.887804 24050 leader_election.cc:304] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6; no voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:23.887830 24135 consensus_queue.cc:1048] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.894429 24084 raft_consensus.cc:2468] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:23.896490 24012 raft_consensus.cc:493] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:23.897055 24012 raft_consensus.cc:3060] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.902484 24169 raft_consensus.cc:2749] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:23.905160 24133 consensus_queue.cc:1048] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.906067 24012 raft_consensus.cc:515] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.912936 24012 leader_election.cc:290] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.914230 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:23.914970 23935 raft_consensus.cc:3060] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.914390 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d3e13d378d74eab8e016be08fe8832f" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.913661 23833 catalog_manager.cc:5697] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 1 to 2, leader changed from a8c21fcb800b438b9df58d40c6853250 (127.9.198.194) to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.915524 24084 raft_consensus.cc:3055] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:23.915953 24084 raft_consensus.cc:740] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:23.916944 24084 consensus_queue.cc:260] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.918434 24084 raft_consensus.cc:3060] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.923435 23935 raft_consensus.cc:2468] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:23.924728 23975 leader_election.cc:304] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:23.926333 24133 raft_consensus.cc:2804] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:23.926874 24133 raft_consensus.cc:697] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:23.927752 24133 consensus_queue.cc:237] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.931281 24162 consensus_queue.cc:1048] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.934227 23832 catalog_manager.cc:5697] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 1 to 2, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.935084 24084 raft_consensus.cc:2468] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:23.940030 24134 consensus_queue.cc:1048] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:23.948957 24162 raft_consensus.cc:993] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250: : Instructing follower e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:23.949533 24162 raft_consensus.cc:1081] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Signalling peer e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:23.952481 23935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:45004
I20260709 12:18:23.953110 23935 raft_consensus.cc:493] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:23.953522 23935 raft_consensus.cc:3060] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.958984 24169 consensus_queue.cc:1048] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.959389 24135 consensus_queue.cc:1048] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:23.960116 23833 catalog_manager.cc:5697] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 1 to 2, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:23.963220 23935 raft_consensus.cc:515] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.968336 23935 leader_election.cc:290] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:23.970804 23935 raft_consensus.cc:3060] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:23.971818 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:23.972772 24009 raft_consensus.cc:3055] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:23.973165 24009 raft_consensus.cc:740] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:23.975212 24164 consensus_queue.cc:1048] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:23.976042 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14b7bfd5aa9844a685f6a3384f7f1d65" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:23.976830 24084 raft_consensus.cc:3060] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.982151 24009 consensus_queue.cc:260] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:23.984197 24009 raft_consensus.cc:3060] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:23.985989 24169 consensus_queue.cc:1048] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:23.989133 24163 raft_consensus.cc:993] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.989881 24163 raft_consensus.cc:1081] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:23.989130 24084 raft_consensus.cc:2468] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:23.998090 24012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38232
I20260709 12:18:23.998812 24012 raft_consensus.cc:493] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:23.999306 24012 raft_consensus.cc:3060] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:24.003815 23900 leader_election.cc:304] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:24.005501 24190 raft_consensus.cc:2804] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:24.006338 24190 raft_consensus.cc:697] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:24.007711 24190 consensus_queue.cc:237] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:24.009562 24169 consensus_queue.cc:1048] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:24.009935 24134 consensus_queue.cc:1048] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:24.027943 24009 raft_consensus.cc:2468] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:24.028354 24012 raft_consensus.cc:515] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:24.031810 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:24.032155 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b60d1559cb4f40a2463a3d2ef322d4" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:24.032868 24086 raft_consensus.cc:3055] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:24.033202 24086 raft_consensus.cc:740] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:24.034070 24086 consensus_queue.cc:260] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:24.035044 24086 raft_consensus.cc:3060] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:24.038867 24190 consensus_queue.cc:1048] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:24.048914 24012 leader_election.cc:290] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:24.060688 24086 raft_consensus.cc:2468] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:24.061998 23976 leader_election.cc:304] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:24.063321 24161 raft_consensus.cc:2804] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:24.063849 24161 raft_consensus.cc:697] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:24.064704 24161 consensus_queue.cc:237] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:24.069911 23833 catalog_manager.cc:5697] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 1 to 2, leader changed from a8c21fcb800b438b9df58d40c6853250 (127.9.198.194) to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:24.082631 23832 catalog_manager.cc:5697] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 1 to 2, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:24.305598 24086 raft_consensus.cc:1275] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.307220 24161 consensus_queue.cc:1048] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.309832 24086 raft_consensus.cc:1275] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.311975 24190 consensus_queue.cc:1048] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.317232 23935 raft_consensus.cc:1275] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.319046 24201 consensus_queue.cc:1048] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.325356 24012 raft_consensus.cc:1275] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.327119 24164 consensus_queue.cc:1048] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.369504 23935 raft_consensus.cc:1275] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.371097 24201 consensus_queue.cc:1048] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.371747 24086 raft_consensus.cc:1275] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.373330 24201 consensus_queue.cc:1048] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.441408 23935 raft_consensus.cc:1275] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.443009 24086 raft_consensus.cc:1275] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.444203 24162 consensus_queue.cc:1048] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:24.445086 24201 consensus_queue.cc:1048] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.484730 24012 raft_consensus.cc:1275] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.486253 24164 consensus_queue.cc:1048] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.494948 24086 raft_consensus.cc:1275] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.497306 24190 consensus_queue.cc:1048] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:24.683691 24086 raft_consensus.cc:1275] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.685243 24133 consensus_queue.cc:1048] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:24.686025 23935 raft_consensus.cc:3060] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:24.694156 23935 raft_consensus.cc:1275] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:24.695853 24162 consensus_queue.cc:1048] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:26.735536 23832 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49114:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:18:26.738620 23832 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:18:26.800056 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 4ab0e9f23ac94f85b5cc283501aa7625 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:26.801473 23997 tablet_service.cc:1511] Processing CreateTablet for tablet 4ab0e9f23ac94f85b5cc283501aa7625 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:26.802801 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ab0e9f23ac94f85b5cc283501aa7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.803691 23997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ab0e9f23ac94f85b5cc283501aa7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.805124 24073 tablet_service.cc:1511] Processing CreateTablet for tablet 4ab0e9f23ac94f85b5cc283501aa7625 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:26.806845 24073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ab0e9f23ac94f85b5cc283501aa7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.812000 23925 tablet_service.cc:1511] Processing CreateTablet for tablet 4724a9f868554ae18d7057a7a81827b0 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:26.814177 23925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4724a9f868554ae18d7057a7a81827b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.814502 23994 tablet_service.cc:1511] Processing CreateTablet for tablet 4724a9f868554ae18d7057a7a81827b0 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:26.816233 23994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4724a9f868554ae18d7057a7a81827b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.818019 23919 tablet_service.cc:1511] Processing CreateTablet for tablet bfed3369f649424298588422d695690a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:26.818485 23996 tablet_service.cc:1511] Processing CreateTablet for tablet bfed3369f649424298588422d695690a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:26.819730 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfed3369f649424298588422d695690a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.820024 23996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfed3369f649424298588422d695690a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.823815 24075 tablet_service.cc:1511] Processing CreateTablet for tablet 4724a9f868554ae18d7057a7a81827b0 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:26.835402 24074 tablet_service.cc:1511] Processing CreateTablet for tablet bfed3369f649424298588422d695690a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:26.836261 24070 tablet_service.cc:1511] Processing CreateTablet for tablet 310753142d224fbd95e46c25cc565491 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:26.838120 24068 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac01989752e43fab92f98ee7a0a3d51 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:26.838855 24076 tablet_service.cc:1511] Processing CreateTablet for tablet ab7390595cfa43c2a2f088f0cb502840 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:26.843235 24001 tablet_service.cc:1511] Processing CreateTablet for tablet 310753142d224fbd95e46c25cc565491 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:26.825537 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4724a9f868554ae18d7057a7a81827b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.846835 23998 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac01989752e43fab92f98ee7a0a3d51 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:26.849701 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 310753142d224fbd95e46c25cc565491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.848567 24072 tablet_service.cc:1511] Processing CreateTablet for tablet 88df43e9a51548b2be863cad3662f67a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:26.851475 24072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88df43e9a51548b2be863cad3662f67a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.854039 24070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 310753142d224fbd95e46c25cc565491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.854907 24068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac01989752e43fab92f98ee7a0a3d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.855437 23998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac01989752e43fab92f98ee7a0a3d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.857035 24069 tablet_service.cc:1511] Processing CreateTablet for tablet dfde0376f8d84d329ec80c542e7c6e7a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:26.857381 24071 tablet_service.cc:1511] Processing CreateTablet for tablet fb92272c273649dd942dc7b1843b9e5c (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:26.858775 24069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfde0376f8d84d329ec80c542e7c6e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.860973 24071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb92272c273649dd942dc7b1843b9e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.862720 24002 tablet_service.cc:1511] Processing CreateTablet for tablet 88df43e9a51548b2be863cad3662f67a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:26.864456 24002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88df43e9a51548b2be863cad3662f67a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.845877 23920 tablet_service.cc:1511] Processing CreateTablet for tablet 6ac01989752e43fab92f98ee7a0a3d51 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:26.866119 23920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ac01989752e43fab92f98ee7a0a3d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.866737 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7390595cfa43c2a2f088f0cb502840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.870023 24074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfed3369f649424298588422d695690a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.843427 23995 tablet_service.cc:1511] Processing CreateTablet for tablet ab7390595cfa43c2a2f088f0cb502840 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:26.874616 23999 tablet_service.cc:1511] Processing CreateTablet for tablet dfde0376f8d84d329ec80c542e7c6e7a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:26.878079 24000 tablet_service.cc:1511] Processing CreateTablet for tablet fb92272c273649dd942dc7b1843b9e5c (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:26.881069 23995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7390595cfa43c2a2f088f0cb502840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.881290 23922 tablet_service.cc:1511] Processing CreateTablet for tablet 310753142d224fbd95e46c25cc565491 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:26.882717 23922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 310753142d224fbd95e46c25cc565491. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.883160 23999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfde0376f8d84d329ec80c542e7c6e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.886561 23924 tablet_service.cc:1511] Processing CreateTablet for tablet ab7390595cfa43c2a2f088f0cb502840 (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:26.888377 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7390595cfa43c2a2f088f0cb502840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.892014 23926 tablet_service.cc:1511] Processing CreateTablet for tablet 88df43e9a51548b2be863cad3662f67a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:26.894359 24000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb92272c273649dd942dc7b1843b9e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.896503 23926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88df43e9a51548b2be863cad3662f67a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.971509 23923 tablet_service.cc:1511] Processing CreateTablet for tablet dfde0376f8d84d329ec80c542e7c6e7a (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:26.973099 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfde0376f8d84d329ec80c542e7c6e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:26.977820 24222 tablet_bootstrap.cc:492] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:26.979466 23918 tablet_service.cc:1511] Processing CreateTablet for tablet fb92272c273649dd942dc7b1843b9e5c (DEFAULT_TABLE table=soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:26.980830 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb92272c273649dd942dc7b1843b9e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:27.024734 24224 tablet_bootstrap.cc:492] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.042421 24222 tablet_bootstrap.cc:654] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.081619 24224 tablet_bootstrap.cc:654] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.086063 24225 tablet_bootstrap.cc:492] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.093330 24225 tablet_bootstrap.cc:654] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.096441 24222 tablet_bootstrap.cc:492] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.098066 24222 ts_tablet_manager.cc:1403] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.121s	user 0.016s	sys 0.000s
I20260709 12:18:27.100157 24224 tablet_bootstrap.cc:492] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.100754 24224 ts_tablet_manager.cc:1403] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.000s
I20260709 12:18:27.104369 24224 raft_consensus.cc:359] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.105325 24224 raft_consensus.cc:385] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.105690 24224 raft_consensus.cc:740] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.107101 24224 consensus_queue.cc:260] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.105674 24222 raft_consensus.cc:359] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.110040 24224 ts_tablet_manager.cc:1434] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:18:27.112470 24222 raft_consensus.cc:385] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.112982 24222 raft_consensus.cc:740] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.113601 24224 tablet_bootstrap.cc:492] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.114044 24222 consensus_queue.cc:260] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.117887 24222 ts_tablet_manager.cc:1434] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.000s
I20260709 12:18:27.123401 24222 tablet_bootstrap.cc:492] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
W20260709 12:18:27.129549 23882 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:27.133723 24224 tablet_bootstrap.cc:654] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.136338 24222 tablet_bootstrap.cc:654] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.138095 24225 tablet_bootstrap.cc:492] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.138777 24225 ts_tablet_manager.cc:1403] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.032s
I20260709 12:18:27.140655 24224 tablet_bootstrap.cc:492] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.141336 24224 ts_tablet_manager.cc:1403] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.004s
I20260709 12:18:27.142318 24225 raft_consensus.cc:359] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.142935 24222 tablet_bootstrap.cc:492] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.143172 24225 raft_consensus.cc:385] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.143478 24222 ts_tablet_manager.cc:1403] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.000s
I20260709 12:18:27.143476 24225 raft_consensus.cc:740] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.144554 24224 raft_consensus.cc:359] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.145216 24224 raft_consensus.cc:385] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.145551 24224 raft_consensus.cc:740] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.145987 24225 consensus_queue.cc:260] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.146467 24224 consensus_queue.cc:260] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.147023 24222 raft_consensus.cc:359] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.149657 24224 ts_tablet_manager.cc:1434] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:18:27.153067 24222 raft_consensus.cc:385] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.153503 24222 raft_consensus.cc:740] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.154224 24224 tablet_bootstrap.cc:492] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.154407 24222 consensus_queue.cc:260] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.149654 24225 ts_tablet_manager.cc:1434] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:18:27.156729 24225 tablet_bootstrap.cc:492] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.157029 24222 ts_tablet_manager.cc:1434] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:18:27.158154 24222 tablet_bootstrap.cc:492] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:27.163317 24224 tablet_bootstrap.cc:654] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.164449 24225 tablet_bootstrap.cc:654] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.166893 24222 tablet_bootstrap.cc:654] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.177582 24224 tablet_bootstrap.cc:492] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.178473 24224 ts_tablet_manager.cc:1403] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 12:18:27.183636 24224 raft_consensus.cc:359] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.184854 24224 raft_consensus.cc:385] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.185472 24224 raft_consensus.cc:740] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.186831 24224 consensus_queue.cc:260] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.191008 24224 ts_tablet_manager.cc:1434] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 12:18:27.192651 24224 tablet_bootstrap.cc:492] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.203502 24224 tablet_bootstrap.cc:654] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.218163 24224 tablet_bootstrap.cc:492] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.219003 24224 ts_tablet_manager.cc:1403] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.008s
I20260709 12:18:27.223529 24224 raft_consensus.cc:359] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.224356 24224 raft_consensus.cc:385] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.224737 24224 raft_consensus.cc:740] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.225723 24224 consensus_queue.cc:260] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.226569 24225 tablet_bootstrap.cc:492] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.227308 24225 ts_tablet_manager.cc:1403] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.071s	user 0.031s	sys 0.038s
I20260709 12:18:27.229162 24224 ts_tablet_manager.cc:1434] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:18:27.230746 24224 tablet_bootstrap.cc:492] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.231462 24222 tablet_bootstrap.cc:492] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.232121 24222 ts_tablet_manager.cc:1403] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.074s	user 0.017s	sys 0.037s
I20260709 12:18:27.231796 24225 raft_consensus.cc:359] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.232741 24225 raft_consensus.cc:385] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.233115 24225 raft_consensus.cc:740] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.235806 24222 raft_consensus.cc:359] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.239320 24224 tablet_bootstrap.cc:654] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.239434 24222 raft_consensus.cc:385] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.239006 24225 consensus_queue.cc:260] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.240048 24222 raft_consensus.cc:740] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.241020 24222 consensus_queue.cc:260] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.243024 24225 ts_tablet_manager.cc:1434] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 12:18:27.243598 24222 ts_tablet_manager.cc:1434] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:18:27.244648 24225 tablet_bootstrap.cc:492] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.244817 24222 tablet_bootstrap.cc:492] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:27.252986 24222 tablet_bootstrap.cc:654] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.255612 24225 tablet_bootstrap.cc:654] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.256624 24224 tablet_bootstrap.cc:492] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.257357 24224 ts_tablet_manager.cc:1403] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.008s
I20260709 12:18:27.261677 24224 raft_consensus.cc:359] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.262943 24224 raft_consensus.cc:385] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.263410 24224 raft_consensus.cc:740] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.264714 24224 consensus_queue.cc:260] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.268705 24224 ts_tablet_manager.cc:1434] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:18:27.271693 24224 tablet_bootstrap.cc:492] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.277907 24225 tablet_bootstrap.cc:492] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.278939 24225 ts_tablet_manager.cc:1403] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.013s
I20260709 12:18:27.280889 24224 tablet_bootstrap.cc:654] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.283234 24225 raft_consensus.cc:359] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.284705 24225 raft_consensus.cc:385] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.285279 24225 raft_consensus.cc:740] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.286540 24225 consensus_queue.cc:260] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.287091 24134 raft_consensus.cc:493] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.288619 24134 raft_consensus.cc:515] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.290652 24225 ts_tablet_manager.cc:1434] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.006s
I20260709 12:18:27.291975 24134 leader_election.cc:290] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.294799 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfed3369f649424298588422d695690a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.295305 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfed3369f649424298588422d695690a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:27.296047 24012 raft_consensus.cc:2468] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.296080 24084 raft_consensus.cc:2468] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.301883 23900 leader_election.cc:304] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.297103 24225 tablet_bootstrap.cc:492] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.307081 24134 raft_consensus.cc:2804] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.307646 24134 raft_consensus.cc:493] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.308097 24134 raft_consensus.cc:3060] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.315629 24225 tablet_bootstrap.cc:654] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.328711 24225 tablet_bootstrap.cc:492] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.329273 24225 ts_tablet_manager.cc:1403] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.013s
I20260709 12:18:27.332309 24134 raft_consensus.cc:515] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.332477 24225 raft_consensus.cc:359] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.333963 24225 raft_consensus.cc:385] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.334394 24225 raft_consensus.cc:740] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.335455 24225 consensus_queue.cc:260] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.339488 24225 ts_tablet_manager.cc:1434] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:18:27.341050 24225 tablet_bootstrap.cc:492] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.350881 24225 tablet_bootstrap.cc:654] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.366408 24224 tablet_bootstrap.cc:492] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.367074 24224 ts_tablet_manager.cc:1403] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.096s	user 0.030s	sys 0.046s
I20260709 12:18:27.369959 24222 tablet_bootstrap.cc:492] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.370501 24224 raft_consensus.cc:359] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.370899 24222 ts_tablet_manager.cc:1403] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.126s	user 0.050s	sys 0.047s
I20260709 12:18:27.371356 24224 raft_consensus.cc:385] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.371744 24224 raft_consensus.cc:740] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.372613 24224 consensus_queue.cc:260] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.374817 24134 leader_election.cc:290] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.375213 24224 ts_tablet_manager.cc:1434] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:18:27.380690 24224 tablet_bootstrap.cc:492] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.381134 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfed3369f649424298588422d695690a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:27.381856 24084 raft_consensus.cc:3060] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.383049 24135 raft_consensus.cc:493] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.383620 24135 raft_consensus.cc:515] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.384321 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfed3369f649424298588422d695690a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.385020 24011 raft_consensus.cc:3060] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.388506 24135 leader_election.cc:290] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.389257 24224 tablet_bootstrap.cc:654] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.391462 24225 tablet_bootstrap.cc:492] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.392110 24225 ts_tablet_manager.cc:1403] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.007s
I20260709 12:18:27.391477 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.391534 24222 raft_consensus.cc:359] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.393548 24222 raft_consensus.cc:385] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.393900 24222 raft_consensus.cc:740] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.394719 24222 consensus_queue.cc:260] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.395849 24011 raft_consensus.cc:2468] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.397504 24225 raft_consensus.cc:359] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.398700 24225 raft_consensus.cc:385] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.399070 24225 raft_consensus.cc:740] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.399960 24225 consensus_queue.cc:260] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.400995 24009 raft_consensus.cc:2468] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.403609 24225 ts_tablet_manager.cc:1434] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 12:18:27.404392 24050 leader_election.cc:304] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:27.410323 24225 tablet_bootstrap.cc:492] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.412307 24135 raft_consensus.cc:2804] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.412846 24135 raft_consensus.cc:493] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.413206 24135 raft_consensus.cc:3060] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.416791 24222 ts_tablet_manager.cc:1434] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.045s	user 0.003s	sys 0.003s
I20260709 12:18:27.418069 24222 tablet_bootstrap.cc:492] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:27.422873 24135 raft_consensus.cc:515] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.423885 24224 tablet_bootstrap.cc:492] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.424259 24225 tablet_bootstrap.cc:654] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.424492 24224 ts_tablet_manager.cc:1403] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.005s
I20260709 12:18:27.428046 24224 raft_consensus.cc:359] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.428846 24224 raft_consensus.cc:385] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.429169 24224 raft_consensus.cc:740] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.433995 24224 consensus_queue.cc:260] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.439445 24222 tablet_bootstrap.cc:654] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.407739 24084 raft_consensus.cc:2468] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.441913 24225 tablet_bootstrap.cc:492] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.442656 23900 leader_election.cc:304] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.443683 24241 raft_consensus.cc:2804] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.444185 24241 raft_consensus.cc:697] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:27.444877 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:27.445696 23936 raft_consensus.cc:2468] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.445212 24241 consensus_queue.cc:237] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.447321 24225 ts_tablet_manager.cc:1403] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.010s
I20260709 12:18:27.447350 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:27.448194 23936 raft_consensus.cc:3060] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.449375 24135 leader_election.cc:290] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.450606 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.451383 24009 raft_consensus.cc:3060] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.453490 24224 ts_tablet_manager.cc:1434] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.004s
I20260709 12:18:27.454187 24225 raft_consensus.cc:359] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.455154 24225 raft_consensus.cc:385] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.455538 24225 raft_consensus.cc:740] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.456509 24225 consensus_queue.cc:260] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.454802 24224 tablet_bootstrap.cc:492] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.457660 23936 raft_consensus.cc:2468] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:27.460693 24049 leader_election.cc:304] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.462482 24246 raft_consensus.cc:2804] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.463142 24246 raft_consensus.cc:697] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:27.464162 24246 consensus_queue.cc:237] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.472491 24225 ts_tablet_manager.cc:1434] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.004s
I20260709 12:18:27.474222 24225 tablet_bootstrap.cc:492] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.474117 24009 raft_consensus.cc:2468] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:27.481803 24135 raft_consensus.cc:493] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.482434 24135 raft_consensus.cc:515] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.484841 24135 leader_election.cc:290] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.487752 24225 tablet_bootstrap.cc:654] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.482877 24224 tablet_bootstrap.cc:654] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.497507 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:27.497553 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
W20260709 12:18:27.500537 24050 leader_election.cc:343] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:27.504447 24222 tablet_bootstrap.cc:492] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.505424 24222 ts_tablet_manager.cc:1403] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.023s
W20260709 12:18:27.506001 24049 leader_election.cc:343] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:27.506742 24049 leader_election.cc:304] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6; no voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:27.508623 24241 raft_consensus.cc:493] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.509294 24135 raft_consensus.cc:2749] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:27.509215 24241 raft_consensus.cc:515] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.514400 24224 tablet_bootstrap.cc:492] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.515120 24224 ts_tablet_manager.cc:1403] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.009s
I20260709 12:18:27.513573 23832 catalog_manager.cc:5697] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:27.516151 24241 leader_election.cc:290] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.510955 24242 raft_consensus.cc:493] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.524296 24242 raft_consensus.cc:515] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.526487 24242 leader_election.cc:290] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.513216 24222 raft_consensus.cc:359] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.519006 24224 raft_consensus.cc:359] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.519968 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ab0e9f23ac94f85b5cc283501aa7625" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:27.527195 24222 raft_consensus.cc:385] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.527318 24224 raft_consensus.cc:385] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.527577 24084 raft_consensus.cc:2468] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.527845 24224 raft_consensus.cc:740] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.528030 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:27.521929 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ab0e9f23ac94f85b5cc283501aa7625" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.528438 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.528931 24009 raft_consensus.cc:2468] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.528717 24224 consensus_queue.cc:260] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.529155 24011 raft_consensus.cc:2468] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.529608 23900 leader_election.cc:304] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.530455 24242 raft_consensus.cc:2804] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.530937 24242 raft_consensus.cc:493] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.531296 24242 raft_consensus.cc:3060] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.531227 23900 leader_election.cc:304] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.531611 24224 ts_tablet_manager.cc:1434] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:18:27.532298 24241 raft_consensus.cc:2804] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.532681 24241 raft_consensus.cc:493] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.532964 24224 tablet_bootstrap.cc:492] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: Bootstrap starting.
I20260709 12:18:27.533038 24241 raft_consensus.cc:3060] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.528820 24086 raft_consensus.cc:2468] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.540128 24225 tablet_bootstrap.cc:492] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.540323 24242 raft_consensus.cc:515] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.541261 24225 ts_tablet_manager.cc:1403] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.014s
I20260709 12:18:27.527623 24222 raft_consensus.cc:740] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.542511 24222 consensus_queue.cc:260] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.543731 24241 raft_consensus.cc:515] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.545035 24222 ts_tablet_manager.cc:1434] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 12:18:27.546558 24222 tablet_bootstrap.cc:492] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:27.545991 24225 raft_consensus.cc:359] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.546886 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.547191 24225 raft_consensus.cc:385] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.547529 24011 raft_consensus.cc:3060] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.547741 24225 raft_consensus.cc:740] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.553190 24224 tablet_bootstrap.cc:654] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.554364 24222 tablet_bootstrap.cc:654] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.548992 24225 consensus_queue.cc:260] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.557610 24246 raft_consensus.cc:493] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.557457 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:27.558465 24086 raft_consensus.cc:3060] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.558198 24246 raft_consensus.cc:515] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.559265 24225 ts_tablet_manager.cc:1434] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:18:27.558800 23829 catalog_manager.cc:5697] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.560604 24225 tablet_bootstrap.cc:492] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.560732 24246 leader_election.cc:290] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.562474 24242 leader_election.cc:290] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.566942 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ab0e9f23ac94f85b5cc283501aa7625" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.567946 24009 raft_consensus.cc:3060] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.568714 24224 tablet_bootstrap.cc:492] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: No bootstrap required, opened a new log
I20260709 12:18:27.569476 24224 ts_tablet_manager.cc:1403] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.001s
I20260709 12:18:27.572235 24222 tablet_bootstrap.cc:492] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.572882 24222 ts_tablet_manager.cc:1403] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.005s
I20260709 12:18:27.574146 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ab0e9f23ac94f85b5cc283501aa7625" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:27.574421 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.574916 24084 raft_consensus.cc:3060] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.575208 24012 raft_consensus.cc:2468] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.576542 24225 tablet_bootstrap.cc:654] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.576828 24050 leader_election.cc:304] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:27.576637 24222 raft_consensus.cc:359] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.572965 24224 raft_consensus.cc:359] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.577796 24246 raft_consensus.cc:2804] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.577945 24224 raft_consensus.cc:385] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.578301 24246 raft_consensus.cc:493] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.578348 24224 raft_consensus.cc:740] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.578755 24246 raft_consensus.cc:3060] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.579488 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:27.579355 24224 consensus_queue.cc:260] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.580237 23936 raft_consensus.cc:2468] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.581776 24224 ts_tablet_manager.cc:1434] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:18:27.591506 24222 raft_consensus.cc:385] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.597649 24222 raft_consensus.cc:740] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.595001 24241 leader_election.cc:290] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.597151 24011 raft_consensus.cc:2468] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.599594 24086 raft_consensus.cc:2468] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.599068 24222 consensus_queue.cc:260] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.601085 23900 leader_election.cc:304] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.602618 24009 raft_consensus.cc:2468] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.602967 24246 raft_consensus.cc:515] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.605906 24246 leader_election.cc:290] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.607141 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.607897 24009 raft_consensus.cc:3060] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.608464 24225 tablet_bootstrap.cc:492] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.608703 24222 ts_tablet_manager.cc:1434] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:18:27.609300 24225 ts_tablet_manager.cc:1403] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.014s
I20260709 12:18:27.609807 24084 raft_consensus.cc:2468] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.609392 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:27.610612 23936 raft_consensus.cc:3060] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.614131 24225 raft_consensus.cc:359] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.614997 24225 raft_consensus.cc:385] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.615280 24225 raft_consensus.cc:740] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.616019 24225 consensus_queue.cc:260] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.616569 24009 raft_consensus.cc:2468] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:27.617482 24246 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.618046 24246 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.618853 24225 ts_tablet_manager.cc:1434] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:18:27.619169 24050 leader_election.cc:304] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:27.619968 24225 tablet_bootstrap.cc:492] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250: Bootstrap starting.
I20260709 12:18:27.620101 24135 raft_consensus.cc:2804] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.620335 24246 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.621789 24135 raft_consensus.cc:697] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:27.622714 24135 consensus_queue.cc:237] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.608417 23900 leader_election.cc:304] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.627064 24223 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.628729 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.629513 24223 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.616896 24222 tablet_bootstrap.cc:492] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:27.631029 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
W20260709 12:18:27.632530 24049 leader_election.cc:343] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:27.628685 24242 raft_consensus.cc:2804] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.635382 24242 raft_consensus.cc:697] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:27.636335 24242 consensus_queue.cc:237] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.639484 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:27.626520 24134 raft_consensus.cc:2804] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.632071 24223 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.641414 24222 tablet_bootstrap.cc:654] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.643616 24009 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.652288 24134 raft_consensus.cc:697] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:27.627465 24225 tablet_bootstrap.cc:654] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.650535 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:27.653575 24134 consensus_queue.cc:237] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.656226 23936 raft_consensus.cc:2468] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:27.657425 24259 raft_consensus.cc:493] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.658601 24050 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: e1961a8ed25e49c0a2a4c95a8ec043c1
W20260709 12:18:27.654659 23975 leader_election.cc:343] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:27.657995 24259 raft_consensus.cc:515] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.652905 24084 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 0.
I20260709 12:18:27.663295 23976 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:27.665020 24262 raft_consensus.cc:2804] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.665410 24259 leader_election.cc:290] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.665534 24262 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.674467 24262 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.675778 24222 tablet_bootstrap.cc:492] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.670146 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:27.670184 24246 raft_consensus.cc:2804] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.670231 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.685555 24009 raft_consensus.cc:2468] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.686435 24262 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.687009 24050 leader_election.cc:304] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:27.687831 24246 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.688346 24246 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.689121 24225 tablet_bootstrap.cc:492] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250: No bootstrap required, opened a new log
I20260709 12:18:27.689754 24225 ts_tablet_manager.cc:1403] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.012s
I20260709 12:18:27.689550 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
W20260709 12:18:27.691258 23975 leader_election.cc:343] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:27.693667 24262 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.694316 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:27.695037 24259 raft_consensus.cc:2804] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.694180 23830 catalog_manager.cc:5697] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.695405 24259 raft_consensus.cc:493] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.695905 24259 raft_consensus.cc:3060] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.692873 24225 raft_consensus.cc:359] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.697956 24225 raft_consensus.cc:385] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.698320 24225 raft_consensus.cc:740] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.699254 24225 consensus_queue.cc:260] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.699347 24246 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
W20260709 12:18:27.699426 24049 leader_election.cc:343] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:27.700798 24086 raft_consensus.cc:2393] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a8c21fcb800b438b9df58d40c6853250 in current term 1: Already voted for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in this term.
I20260709 12:18:27.683478 24222 ts_tablet_manager.cc:1403] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.003s
I20260709 12:18:27.704033 24259 raft_consensus.cc:515] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.705225 24222 raft_consensus.cc:359] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.705852 23976 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:27.706218 24222 raft_consensus.cc:385] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.706642 24222 raft_consensus.cc:740] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.707511 24262 raft_consensus.cc:2749] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:18:27.708459 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.709102 24009 raft_consensus.cc:3060] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.709884 24225 ts_tablet_manager.cc:1434] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 12:18:27.707549 24222 consensus_queue.cc:260] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.710691 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:27.711782 24246 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.714120 24222 ts_tablet_manager.cc:1434] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:18:27.714032 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:27.715222 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.715581 24222 tablet_bootstrap.cc:492] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:27.716061 24011 raft_consensus.cc:2393] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0cbb7c136dd3451c93deee0d3fd64be6 in current term 1: Already voted for candidate a8c21fcb800b438b9df58d40c6853250 in this term.
W20260709 12:18:27.716979 24049 leader_election.cc:343] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:18:27.720952 24049 leader_election.cc:343] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:27.721529 24049 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6; no voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:27.723415 24135 raft_consensus.cc:2749] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:18:27.724129 24222 tablet_bootstrap.cc:654] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.724792 23832 catalog_manager.cc:5697] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:27.727484 24009 raft_consensus.cc:2468] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:27.729748 24050 leader_election.cc:304] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:27.730996 24135 raft_consensus.cc:2804] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.731653 24135 raft_consensus.cc:697] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:27.732738 24135 consensus_queue.cc:237] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.734875 24222 tablet_bootstrap.cc:492] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.735437 24222 ts_tablet_manager.cc:1403] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 12:18:27.738382 24222 raft_consensus.cc:359] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.739063 24222 raft_consensus.cc:385] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.739320 24222 raft_consensus.cc:740] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.739991 24222 consensus_queue.cc:260] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.725558 24259 leader_election.cc:290] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.742390 24222 ts_tablet_manager.cc:1434] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:18:27.743402 24222 tablet_bootstrap.cc:492] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1: Bootstrap starting.
I20260709 12:18:27.742889 23832 catalog_manager.cc:5697] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.745134 23830 catalog_manager.cc:5697] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:27.750481 24222 tablet_bootstrap.cc:654] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:27.759508 24222 tablet_bootstrap.cc:492] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1: No bootstrap required, opened a new log
I20260709 12:18:27.760219 24222 ts_tablet_manager.cc:1403] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.002s
I20260709 12:18:27.763231 24222 raft_consensus.cc:359] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.763835 24222 raft_consensus.cc:385] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:27.764063 24222 raft_consensus.cc:740] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Initialized, Role: FOLLOWER
I20260709 12:18:27.764683 24222 consensus_queue.cc:260] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.766667 24222 ts_tablet_manager.cc:1434] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:18:27.811746 24242 raft_consensus.cc:493] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.812285 24242 raft_consensus.cc:515] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.814281 24242 leader_election.cc:290] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.815110 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.815307 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:27.815750 24009 raft_consensus.cc:2468] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.816116 24084 raft_consensus.cc:2468] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 0.
I20260709 12:18:27.816833 23900 leader_election.cc:304] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.817826 24242 raft_consensus.cc:2804] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.818257 24242 raft_consensus.cc:493] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.818598 24242 raft_consensus.cc:3060] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.825537 24242 raft_consensus.cc:515] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.827605 24242 leader_election.cc:290] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.828255 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.828547 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:27.828941 24009 raft_consensus.cc:3060] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.829154 24084 raft_consensus.cc:3060] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.836555 24084 raft_consensus.cc:2468] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.836555 24009 raft_consensus.cc:2468] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 1.
I20260709 12:18:27.836609 24259 raft_consensus.cc:493] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.837761 24259 raft_consensus.cc:515] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.838100 23900 leader_election.cc:304] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.838971 24242 raft_consensus.cc:2804] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.839457 24242 raft_consensus.cc:697] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:27.839670 24259 leader_election.cc:290] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.840377 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310753142d224fbd95e46c25cc565491" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:27.840363 24242 consensus_queue.cc:237] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.841002 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310753142d224fbd95e46c25cc565491" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.841177 23936 raft_consensus.cc:2468] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.841789 24009 raft_consensus.cc:2468] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 0.
I20260709 12:18:27.842428 24049 leader_election.cc:304] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:27.843318 24259 raft_consensus.cc:2804] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:27.843765 24259 raft_consensus.cc:493] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.844218 24259 raft_consensus.cc:3060] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.850418 23832 catalog_manager.cc:5697] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 1 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:27.852478 24259 raft_consensus.cc:515] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.855540 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310753142d224fbd95e46c25cc565491" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:27.856259 23936 raft_consensus.cc:3060] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.856257 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310753142d224fbd95e46c25cc565491" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:27.856947 24259 leader_election.cc:290] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:27.857259 24009 raft_consensus.cc:3060] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.863767 24009 raft_consensus.cc:2468] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:27.863758 23936 raft_consensus.cc:2468] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:27.864979 24050 leader_election.cc:304] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:27.865772 24259 raft_consensus.cc:2804] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:27.866302 24259 raft_consensus.cc:697] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:27.867046 24259 consensus_queue.cc:237] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.875562 23832 catalog_manager.cc:5697] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 1 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.916854 24259 consensus_queue.cc:1048] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:27.931706 24259 consensus_queue.cc:1048] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:27.939975 24242 consensus_queue.cc:1048] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:27.947743 24242 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.948354 24242 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:27.950716 24242 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:27.952291 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:27.953317 24084 raft_consensus.cc:2393] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in current term 1: Already voted for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in this term.
I20260709 12:18:27.955688 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:27.956553 24011 raft_consensus.cc:2393] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in current term 1: Already voted for candidate a8c21fcb800b438b9df58d40c6853250 in this term.
I20260709 12:18:27.957557 24279 consensus_queue.cc:1048] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:27.959729 23900 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:27.961822 24279 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:27.971729 24279 raft_consensus.cc:2749] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:28.045045 24259 consensus_queue.cc:1048] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.050648 24279 raft_consensus.cc:493] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.051529 24279 raft_consensus.cc:515] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:28.053747 24279 leader_election.cc:290] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:28.055840 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:28.056844 24011 raft_consensus.cc:2393] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in current term 1: Already voted for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in this term.
I20260709 12:18:28.058481 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" is_pre_election: true
I20260709 12:18:28.060124 24135 consensus_queue.cc:1048] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:28.060118 23900 leader_election.cc:304] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:28.061460 24279 raft_consensus.cc:3060] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:28.069805 24279 raft_consensus.cc:2749] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:18:28.135984 23882 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:28.148262 24242 consensus_queue.cc:1048] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.153760 24279 consensus_queue.cc:1048] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:28.165500 24242 consensus_queue.cc:1048] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.176271 24279 consensus_queue.cc:1048] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.185497 24135 consensus_queue.cc:1048] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.194649 24135 consensus_queue.cc:1048] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.244477 24279 consensus_queue.cc:1048] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.255173 24242 consensus_queue.cc:1048] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.376785 24135 consensus_queue.cc:1048] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:28.387979 24135 consensus_queue.cc:1048] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:18:29.143496 23882 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:29.846513 24282 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.847283 24282 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:29.850597 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" is_pre_election: true
I20260709 12:18:29.850031 24282 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:29.851084 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250" is_pre_election: true
I20260709 12:18:29.851432 23936 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:29.851763 24011 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 1.
I20260709 12:18:29.852852 24049 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:29.853813 24282 raft_consensus.cc:2804] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:18:29.854173 24282 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.854429 24282 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:29.860410 24282 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:29.862154 24282 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:29.863116 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:29.863116 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:29.864069 24011 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:29.864068 23936 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:29.871778 23936 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 2.
I20260709 12:18:29.871781 24011 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 2.
I20260709 12:18:29.873193 24050 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:29.874024 24282 raft_consensus.cc:2804] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:29.874482 24282 raft_consensus.cc:697] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:29.875190 24282 consensus_queue.cc:237] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:29.882629 23832 catalog_manager.cc:5697] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 0 to 2, leader changed from <none> to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:30.236135 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:30.236667 24086 raft_consensus.cc:606] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Received request to transfer leadership to TS e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:30.245023 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:30.245545 24086 raft_consensus.cc:606] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Received request to transfer leadership to TS e1961a8ed25e49c0a2a4c95a8ec043c1
I20260709 12:18:30.250976 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "310753142d224fbd95e46c25cc565491"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:30.251484 24086 raft_consensus.cc:606] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:30.256574 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:30.257081 24086 raft_consensus.cc:606] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:30.262509 23935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:45020
I20260709 12:18:30.263118 23935 raft_consensus.cc:606] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:30.268231 24086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
mode: GRACEFUL
new_leader_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:40352
I20260709 12:18:30.268787 24086 raft_consensus.cc:606] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 LEADER]: Received request to transfer leadership to TS a8c21fcb800b438b9df58d40c6853250
I20260709 12:18:30.273994 23935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
mode: GRACEFUL
new_leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
 from {username='slave'} at 127.0.0.1:45020
I20260709 12:18:30.274600 23935 raft_consensus.cc:606] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Received request to transfer leadership to TS 0cbb7c136dd3451c93deee0d3fd64be6
I20260709 12:18:30.304113 24301 raft_consensus.cc:993] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.304788 24301 raft_consensus.cc:1081] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.305995 23935 raft_consensus.cc:1275] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:18:30.306497 24009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38234
I20260709 12:18:30.307163 24009 raft_consensus.cc:493] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:30.307591 24009 raft_consensus.cc:3060] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.307391 24282 consensus_queue.cc:1048] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:30.315692 24009 raft_consensus.cc:515] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.318768 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:30.319446 23935 raft_consensus.cc:3055] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:30.319788 23935 raft_consensus.cc:740] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:30.320534 24009 leader_election.cc:290] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:30.320781 23935 consensus_queue.cc:260] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.322152 23935 raft_consensus.cc:3060] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.321511 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88df43e9a51548b2be863cad3662f67a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:30.324193 24086 raft_consensus.cc:3060] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.334590 24086 raft_consensus.cc:2468] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:30.336210 23976 leader_election.cc:304] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:30.337030 24223 raft_consensus.cc:2804] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:30.337669 24223 raft_consensus.cc:697] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:30.338524 24223 consensus_queue.cc:237] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.336609 23935 raft_consensus.cc:2468] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:30.342335 24306 raft_consensus.cc:993] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.342931 24306 raft_consensus.cc:1081] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.345100 24009 raft_consensus.cc:1275] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:18:30.345146 24011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38232
I20260709 12:18:30.346181 24011 raft_consensus.cc:493] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:30.346705 24011 raft_consensus.cc:3060] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.347302 24306 consensus_queue.cc:1048] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:30.353264 23832 catalog_manager.cc:5697] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 1 to 2, leader changed from e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:30.355872 24011 raft_consensus.cc:515] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.358968 24011 leader_election.cc:290] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:30.359146 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:30.359462 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7390595cfa43c2a2f088f0cb502840" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:30.359833 23935 raft_consensus.cc:3060] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.360107 24086 raft_consensus.cc:3055] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:30.360460 24086 raft_consensus.cc:740] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:30.361260 24086 consensus_queue.cc:260] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.361876 24306 raft_consensus.cc:993] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.362387 24306 raft_consensus.cc:1081] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.362516 24086 raft_consensus.cc:3060] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.367427 23935 raft_consensus.cc:2468] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:30.364827 24009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38232
I20260709 12:18:30.368356 24009 raft_consensus.cc:493] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:30.368755 24009 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:30.368714 23975 leader_election.cc:304] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:30.370076 24223 raft_consensus.cc:2804] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:30.370548 24223 raft_consensus.cc:697] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:30.371387 24223 consensus_queue.cc:237] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.375669 24086 raft_consensus.cc:2468] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:30.377192 24009 raft_consensus.cc:515] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.378446 24295 raft_consensus.cc:993] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.378950 24302 raft_consensus.cc:993] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: : Instructing follower 0cbb7c136dd3451c93deee0d3fd64be6 to start an election
I20260709 12:18:30.379521 24134 raft_consensus.cc:1081] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Signalling peer 0cbb7c136dd3451c93deee0d3fd64be6 to start an election
I20260709 12:18:30.378984 24295 raft_consensus.cc:1081] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Signalling peer a8c21fcb800b438b9df58d40c6853250 to start an election
I20260709 12:18:30.381333 24086 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
 from {username='slave'} at 127.0.0.1:40344
I20260709 12:18:30.381274 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:30.381908 24012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "310753142d224fbd95e46c25cc565491"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
 from {username='slave'} at 127.0.0.1:38232
I20260709 12:18:30.382356 23935 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:30.382508 24012 raft_consensus.cc:493] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:30.383020 24012 raft_consensus.cc:3060] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.384063 24009 leader_election.cc:290] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 3 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:30.384269 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfde0376f8d84d329ec80c542e7c6e7a" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:30.385098 24084 raft_consensus.cc:3055] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:30.385493 24084 raft_consensus.cc:740] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:30.386595 24084 consensus_queue.cc:260] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.388312 24084 raft_consensus.cc:3060] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:30.390383 23935 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 3.
I20260709 12:18:30.391040 24086 raft_consensus.cc:493] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:30.392241 24086 raft_consensus.cc:3060] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.392155 23975 leader_election.cc:304] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:30.394044 24012 raft_consensus.cc:515] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.397161 23935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310753142d224fbd95e46c25cc565491" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:30.397761 23935 raft_consensus.cc:3060] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.398248 24223 raft_consensus.cc:2804] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:30.397549 23832 catalog_manager.cc:5697] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 1 to 2, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:30.398870 24223 raft_consensus.cc:697] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 3 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:30.399734 24223 consensus_queue.cc:237] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.400264 24012 leader_election.cc:290] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:30.400801 24086 raft_consensus.cc:515] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.397835 24084 raft_consensus.cc:2468] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 3.
I20260709 12:18:30.402006 24085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310753142d224fbd95e46c25cc565491" candidate_uuid: "a8c21fcb800b438b9df58d40c6853250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:30.402696 24085 raft_consensus.cc:3055] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:30.403017 24085 raft_consensus.cc:740] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:30.404039 24086 leader_election.cc:290] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 2 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:30.403825 24085 consensus_queue.cc:260] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.404757 23936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:30.405259 24085 raft_consensus.cc:3060] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.405367 23936 raft_consensus.cc:3055] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:30.406898 23936 raft_consensus.cc:740] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:30.407079 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb92272c273649dd942dc7b1843b9e5c" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:30.407878 24009 raft_consensus.cc:3060] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.408102 23936 consensus_queue.cc:260] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.406888 23935 raft_consensus.cc:2468] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:30.409430 23936 raft_consensus.cc:3060] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.410789 23975 leader_election.cc:304] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:30.412606 24294 raft_consensus.cc:2804] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:30.413115 24294 raft_consensus.cc:697] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Leader. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:30.413976 24294 consensus_queue.cc:237] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.416803 24009 raft_consensus.cc:2468] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 2.
I20260709 12:18:30.418267 24050 leader_election.cc:304] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:30.418421 24085 raft_consensus.cc:2468] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c21fcb800b438b9df58d40c6853250 in term 2.
I20260709 12:18:30.419220 24295 raft_consensus.cc:2804] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:30.419718 24295 raft_consensus.cc:697] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:30.420569 24295 consensus_queue.cc:237] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.428383 23936 raft_consensus.cc:2468] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 2.
I20260709 12:18:30.433399 23832 catalog_manager.cc:5697] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 2 to 3, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 3 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:30.434280 23830 catalog_manager.cc:5697] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 1 to 2, leader changed from e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193) to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 2 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:30.434791 23832 catalog_manager.cc:5697] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 reported cstate change: term changed from 1 to 2, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to a8c21fcb800b438b9df58d40c6853250 (127.9.198.194). New cstate: current_term: 2 leader_uuid: "a8c21fcb800b438b9df58d40c6853250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:30.525493 24306 raft_consensus.cc:993] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:30.526018 24295 raft_consensus.cc:1081] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Signalling peer e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:30.527729 23936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:45016
I20260709 12:18:30.528342 23936 raft_consensus.cc:493] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:30.528656 23936 raft_consensus.cc:3060] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.533728 23936 raft_consensus.cc:515] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.536244 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:30.535545 23936 leader_election.cc:290] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:30.536514 24085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ac01989752e43fab92f98ee7a0a3d51" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:30.536908 24009 raft_consensus.cc:3060] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.537212 24085 raft_consensus.cc:3055] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:30.537617 24085 raft_consensus.cc:740] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:30.538501 24085 consensus_queue.cc:260] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.539733 24085 raft_consensus.cc:3060] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.542903 24009 raft_consensus.cc:2468] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:30.544006 23900 leader_election.cc:304] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:30.544723 24134 raft_consensus.cc:2804] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:30.545190 24134 raft_consensus.cc:697] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:30.545946 24085 raft_consensus.cc:2468] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:30.545987 24134 consensus_queue.cc:237] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.554193 23832 catalog_manager.cc:5697] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 1 to 2, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:30.678227 24306 raft_consensus.cc:993] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: : Instructing follower e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:30.678740 24295 raft_consensus.cc:1081] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Signalling peer e1961a8ed25e49c0a2a4c95a8ec043c1 to start an election
I20260709 12:18:30.680274 23936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
 from {username='slave'} at 127.0.0.1:45016
I20260709 12:18:30.680708 23936 raft_consensus.cc:493] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:30.680943 23936 raft_consensus.cc:3060] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.686825 23936 raft_consensus.cc:515] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.688422 23936 leader_election.cc:290] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195:43695)
I20260709 12:18:30.689828 24085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
I20260709 12:18:30.689826 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4724a9f868554ae18d7057a7a81827b0" candidate_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:30.690600 24085 raft_consensus.cc:3055] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:30.690658 24009 raft_consensus.cc:3060] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.690958 24085 raft_consensus.cc:740] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:30.691885 24085 consensus_queue.cc:260] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.692963 24085 raft_consensus.cc:3060] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:30.696595 24009 raft_consensus.cc:2468] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:30.697983 23900 leader_election.cc:304] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8c21fcb800b438b9df58d40c6853250, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:30.698751 24301 raft_consensus.cc:2804] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:30.699179 24301 raft_consensus.cc:697] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Becoming Leader. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:30.699381 24085 raft_consensus.cc:2468] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1961a8ed25e49c0a2a4c95a8ec043c1 in term 2.
I20260709 12:18:30.700004 24301 consensus_queue.cc:237] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:30.707758 23832 catalog_manager.cc:5697] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 reported cstate change: term changed from 1 to 2, leader changed from 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195) to e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193). New cstate: current_term: 2 leader_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:30.799119 24084 raft_consensus.cc:1275] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.800698 24294 consensus_queue.cc:1048] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.810765 23936 raft_consensus.cc:1275] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.812695 24317 consensus_queue.cc:1048] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:30.815963 24084 raft_consensus.cc:1275] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.821597 24223 consensus_queue.cc:1048] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:30.837855 23936 raft_consensus.cc:1275] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.839389 24223 consensus_queue.cc:1048] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.858062 23936 raft_consensus.cc:1275] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.859604 24294 consensus_queue.cc:1048] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.869113 24084 raft_consensus.cc:1275] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.871097 24317 consensus_queue.cc:1048] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.920847 23936 raft_consensus.cc:1275] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:18:30.922453 24317 consensus_queue.cc:1048] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.927420 24084 raft_consensus.cc:1275] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.929319 24301 consensus_queue.cc:1048] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.938519 24084 raft_consensus.cc:1275] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Refusing update from remote peer a8c21fcb800b438b9df58d40c6853250: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:18:30.940378 24317 consensus_queue.cc:1048] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.945917 24009 raft_consensus.cc:1275] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.948891 24301 consensus_queue.cc:1048] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.950115 24009 raft_consensus.cc:1275] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.952919 24295 consensus_queue.cc:1048] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:30.969897 23936 raft_consensus.cc:1275] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:30.972802 24306 consensus_queue.cc:1048] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:31.891886 24081 raft_consensus.cc:1275] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:31.907186 24330 consensus_queue.cc:1048] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:31.909574 24012 raft_consensus.cc:1275] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Refusing update from remote peer e1961a8ed25e49c0a2a4c95a8ec043c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:31.925817 24337 consensus_queue.cc:1048] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:18:32.909757 24108 debug-util.cc:398] Leaking SignalData structure 0x7b080009eda0 after lost signal to thread 10014
W20260709 12:18:32.910910 24108 debug-util.cc:398] Leaking SignalData structure 0x7b0800144ce0 after lost signal to thread 23867
W20260709 12:18:32.912199 24108 debug-util.cc:398] Leaking SignalData structure 0x7b0800170880 after lost signal to thread 23960
W20260709 12:18:32.913362 24108 debug-util.cc:398] Leaking SignalData structure 0x7b080020cc40 after lost signal to thread 24036
W20260709 12:18:32.914542 24108 debug-util.cc:398] Leaking SignalData structure 0x7b0800281280 after lost signal to thread 24111
I20260709 12:18:33.980947 23832 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49114:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:18:34.007431 23832 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49114:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:18:34.067670 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:34.119004 24379 tablet.cc:1724] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:34.277061 24376 log.cc:927] Time spent T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:18:34.287972 24377 log.cc:927] Time spent T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:18:34.310411 24379 tablet.cc:1724] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.283393 24386 tablet.cc:1724] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:34.369725 24388 log.cc:927] Time spent T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:18:34.416487 24394 log.cc:927] Time spent T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
W20260709 12:18:34.477871 24384 log.cc:927] Time spent T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: Append to log took a long time: real 0.227s	user 0.003s	sys 0.000s
I20260709 12:18:34.486723 24414 tablet.cc:1724] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.466243 24409 tablet.cc:1724] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.499357 24418 tablet.cc:1724] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:34.524214 24050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:46607, user_credentials={real_user=slave}} blocked reactor thread for 57772.4us
W20260709 12:18:34.518306 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:43695, user_credentials={real_user=slave}} blocked reactor thread for 76400us
W20260709 12:18:34.536465 24395 log.cc:927] Time spent T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: Append to log took a long time: real 0.212s	user 0.003s	sys 0.000s
I20260709 12:18:34.540725 24417 tablet.cc:1724] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.544646 24415 tablet.cc:1724] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:34.549265 24402 log.cc:927] Time spent T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 12:18:34.549593 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22"
dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
mode: GRACEFUL
new_leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
 from {username='slave'} at 127.0.0.1:46198
I20260709 12:18:34.550411 23934 raft_consensus.cc:606] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Received request to transfer leadership to TS 0cbb7c136dd3451c93deee0d3fd64be6
I20260709 12:18:34.552002 24386 tablet.cc:1724] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.587291 24421 tablet.cc:1724] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.599272 24423 tablet.cc:1724] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.605787 24430 tablet.cc:1724] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.617868 24416 tablet.cc:1724] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:34.668399 24050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.194:46607, user_credentials={real_user=slave}} blocked reactor thread for 53099.3us
W20260709 12:18:34.669539 23975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.193:41953, user_credentials={real_user=slave}} blocked reactor thread for 43493.4us
I20260709 12:18:34.610680 24418 tablet.cc:1724] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:34.674422 23976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.198.195:43695, user_credentials={real_user=slave}} blocked reactor thread for 57816.2us
I20260709 12:18:34.678603 24414 tablet.cc:1724] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.674278 24428 tablet.cc:1724] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.683157 24386 tablet.cc:1724] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.697239 24417 tablet.cc:1724] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.702142 24415 tablet.cc:1724] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.701565 24409 tablet.cc:1724] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.732278 24421 tablet.cc:1724] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.728641 24379 tablet.cc:1724] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.738246 24414 tablet.cc:1724] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.747786 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50"
dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
mode: GRACEFUL
new_leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
 from {username='slave'} at 127.0.0.1:50530
I20260709 12:18:34.748601 24012 raft_consensus.cc:606] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Received request to transfer leadership to TS 0cbb7c136dd3451c93deee0d3fd64be6
I20260709 12:18:34.750521 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:18:34.751396 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:34.723155 24430 tablet.cc:1724] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.769861 24405 raft_consensus.cc:993] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1: : Instructing follower 0cbb7c136dd3451c93deee0d3fd64be6 to start an election
I20260709 12:18:34.770476 24405 raft_consensus.cc:1081] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Signalling peer 0cbb7c136dd3451c93deee0d3fd64be6 to start an election
I20260709 12:18:34.772598 24081 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
 from {username='slave'} at 127.0.0.1:40344
I20260709 12:18:34.773265 24081 raft_consensus.cc:493] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:34.773721 24081 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:34.772784 24417 tablet.cc:1724] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.740976 24423 tablet.cc:1724] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.786319 24428 tablet.cc:1724] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:34.800370 24081 raft_consensus.cc:515] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:34.804311 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:34.804939 24012 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:34.806674 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f33ba99e41400b8d16dae902c15b22" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:34.807415 23932 raft_consensus.cc:3055] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:34.807727 23932 raft_consensus.cc:740] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1961a8ed25e49c0a2a4c95a8ec043c1, State: Running, Role: LEADER
I20260709 12:18:34.810248 23932 consensus_queue.cc:260] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:34.811646 23932 raft_consensus.cc:3060] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:34.814872 24012 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 3.
I20260709 12:18:34.816357 24050 leader_election.cc:304] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, a8c21fcb800b438b9df58d40c6853250; no voters: 
I20260709 12:18:34.817912 24404 raft_consensus.cc:2804] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:34.818399 24404 raft_consensus.cc:697] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:34.819218 24404 consensus_queue.cc:237] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:34.808697 24081 leader_election.cc:290] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 3 election: Requested vote from peers a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607), e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953)
I20260709 12:18:34.841313 23932 raft_consensus.cc:2468] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 3.
I20260709 12:18:34.856093 23833 catalog_manager.cc:6242] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf: soft_deleted_table [id=096cab4bd97b439f9e8dbfdecc6908ac] alter complete (version 1)
I20260709 12:18:34.874109 23832 catalog_manager.cc:5697] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 2 to 3, leader changed from e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193) to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:34.999831 24397 raft_consensus.cc:993] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250: : Instructing follower 0cbb7c136dd3451c93deee0d3fd64be6 to start an election
I20260709 12:18:35.000487 24390 raft_consensus.cc:1081] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Signalling peer 0cbb7c136dd3451c93deee0d3fd64be6 to start an election
I20260709 12:18:35.002283 24081 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50"
dest_uuid: "0cbb7c136dd3451c93deee0d3fd64be6"
 from {username='slave'} at 127.0.0.1:40334
I20260709 12:18:35.002815 24081 raft_consensus.cc:493] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:35.003145 24081 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:35.008884 24081 raft_consensus.cc:515] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:35.010916 24081 leader_election.cc:290] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 3 election: Requested vote from peers e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953), a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607)
I20260709 12:18:35.012509 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a8c21fcb800b438b9df58d40c6853250"
I20260709 12:18:35.012629 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be9fe9035a814d269d247bde6ab34d50" candidate_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1"
I20260709 12:18:35.014303 24012 raft_consensus.cc:3055] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:35.014499 23932 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:35.014770 24012 raft_consensus.cc:740] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a8c21fcb800b438b9df58d40c6853250, State: Running, Role: LEADER
I20260709 12:18:35.016045 24012 consensus_queue.cc:260] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:35.017501 24012 raft_consensus.cc:3060] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:35.021198 23932 raft_consensus.cc:2468] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 3.
I20260709 12:18:35.022578 24049 leader_election.cc:304] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cbb7c136dd3451c93deee0d3fd64be6, e1961a8ed25e49c0a2a4c95a8ec043c1; no voters: 
I20260709 12:18:35.023416 24404 raft_consensus.cc:2804] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:35.023917 24404 raft_consensus.cc:697] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 LEADER]: Becoming Leader. State: Replica: 0cbb7c136dd3451c93deee0d3fd64be6, State: Running, Role: LEADER
I20260709 12:18:35.024755 24404 consensus_queue.cc:237] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } }
I20260709 12:18:35.026582 24012 raft_consensus.cc:2468] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cbb7c136dd3451c93deee0d3fd64be6 in term 3.
I20260709 12:18:35.037724 23832 catalog_manager.cc:5697] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 reported cstate change: term changed from 2 to 3, leader changed from a8c21fcb800b438b9df58d40c6853250 (127.9.198.194) to 0cbb7c136dd3451c93deee0d3fd64be6 (127.9.198.195). New cstate: current_term: 3 leader_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cbb7c136dd3451c93deee0d3fd64be6" member_type: VOTER last_known_addr { host: "127.9.198.195" port: 43695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:35.303005 24012 raft_consensus.cc:1275] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:35.304773 24404 consensus_queue.cc:1048] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:35.321535 23932 raft_consensus.cc:1275] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:35.325942 24404 consensus_queue.cc:1048] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:35.571321 24012 raft_consensus.cc:1275] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:35.572964 24404 consensus_queue.cc:1048] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c21fcb800b438b9df58d40c6853250" member_type: VOTER last_known_addr { host: "127.9.198.194" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:35.586294 23932 raft_consensus.cc:1275] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Refusing update from remote peer 0cbb7c136dd3451c93deee0d3fd64be6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:35.588896 24404 consensus_queue.cc:1048] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1961a8ed25e49c0a2a4c95a8ec043c1" member_type: VOTER last_known_addr { host: "127.9.198.193" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:35.752753 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:35.760293 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:35.760877 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:36.762883 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:36.773051 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:36.773604 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:37.775130 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:37.782523 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:37.783164 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:38.784354 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:38.789939 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:38.790494 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:39.791523 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:39.796788 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:39.797325 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:40.798738 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:40.805853 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:40.806399 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:41.807662 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:41.813256 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:41.813791 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:42.814883 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:42.820154 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:42.820582 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:43.821470 10011 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:43.827103 10011 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:43.827569 10011 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:44.899641 10011 tablet_server.cc:179] TabletServer@127.9.198.193:0 shutting down...
I20260709 12:18:44.960059 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:18:44.960806 24049 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
I20260709 12:18:44.962039 10011 tablet_replica.cc:333] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:44.963953 10011 raft_consensus.cc:2243] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:44.965113 10011 raft_consensus.cc:2272] T dfde0376f8d84d329ec80c542e7c6e7a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:44.965683 24049 consensus_peers.cc:597] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 -> Peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Couldn't send request to peer e1961a8ed25e49c0a2a4c95a8ec043c1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:44.970150 10011 tablet_replica.cc:333] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
W20260709 12:18:44.971980 23975 consensus_peers.cc:597] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 -> Peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Couldn't send request to peer e1961a8ed25e49c0a2a4c95a8ec043c1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:44.972843 10011 raft_consensus.cc:2243] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:44.973804 10011 raft_consensus.cc:2272] T fb92272c273649dd942dc7b1843b9e5c P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:44.976876 10011 tablet_replica.cc:333] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:44.977825 10011 raft_consensus.cc:2243] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:44.978538 10011 raft_consensus.cc:2272] T 310753142d224fbd95e46c25cc565491 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:44.981383 10011 tablet_replica.cc:333] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:44.982101 10011 raft_consensus.cc:2243] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:44.983237 10011 raft_consensus.cc:2272] T 4724a9f868554ae18d7057a7a81827b0 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:44.986248 10011 tablet_replica.cc:333] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:44.987037 10011 raft_consensus.cc:2243] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:44.987749 10011 raft_consensus.cc:2272] T 88df43e9a51548b2be863cad3662f67a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:44.990759 10011 tablet_replica.cc:333] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:44.991519 10011 raft_consensus.cc:2243] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:44.992197 10011 raft_consensus.cc:2272] T 6d3e13d378d74eab8e016be08fe8832f P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:44.995069 10011 tablet_replica.cc:333] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:44.995849 10011 raft_consensus.cc:2243] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:44.997001 10011 raft_consensus.cc:2272] T bfed3369f649424298588422d695690a P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:44.999951 10011 tablet_replica.cc:333] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.000682 10011 raft_consensus.cc:2243] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.001271 10011 raft_consensus.cc:2272] T 550bb56a01054b30a9262bb704bd1d2b P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.004464 10011 tablet_replica.cc:333] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.005192 10011 raft_consensus.cc:2243] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.006101 10011 raft_consensus.cc:2272] T be9fe9035a814d269d247bde6ab34d50 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.009092 10011 tablet_replica.cc:333] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.009786 10011 raft_consensus.cc:2243] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.010762 10011 raft_consensus.cc:2272] T 4ab0e9f23ac94f85b5cc283501aa7625 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.013406 10011 tablet_replica.cc:333] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.014173 10011 raft_consensus.cc:2243] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.015187 10011 raft_consensus.cc:2272] T f8dc14c966494b2f930c926041c32bae P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.017786 10011 tablet_replica.cc:333] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.018373 10011 raft_consensus.cc:2243] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.019302 10011 raft_consensus.cc:2272] T 6ac01989752e43fab92f98ee7a0a3d51 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.021991 10011 tablet_replica.cc:333] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.022671 10011 raft_consensus.cc:2243] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.023211 10011 raft_consensus.cc:2272] T 15b60d1559cb4f40a2463a3d2ef322d4 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.025995 10011 tablet_replica.cc:333] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.026618 10011 raft_consensus.cc:2243] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.027594 10011 raft_consensus.cc:2272] T 4bad77401a0647049e6f37a773c719f6 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.030141 10011 tablet_replica.cc:333] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.030748 10011 raft_consensus.cc:2243] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.031733 10011 raft_consensus.cc:2272] T 14b7bfd5aa9844a685f6a3384f7f1d65 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.034211 10011 tablet_replica.cc:333] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.034909 10011 raft_consensus.cc:2243] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.035465 10011 raft_consensus.cc:2272] T d22599f2e4d04980829f7a39547e0117 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.038141 10011 tablet_replica.cc:333] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.038743 10011 raft_consensus.cc:2243] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.039382 10011 raft_consensus.cc:2272] T ab7390595cfa43c2a2f088f0cb502840 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.042943 10011 tablet_replica.cc:333] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1: stopping tablet replica
I20260709 12:18:45.043653 10011 raft_consensus.cc:2243] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.044312 10011 raft_consensus.cc:2272] T 64f33ba99e41400b8d16dae902c15b22 P e1961a8ed25e49c0a2a4c95a8ec043c1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:45.064332 23975 consensus_peers.cc:597] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 -> Peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Couldn't send request to peer e1961a8ed25e49c0a2a4c95a8ec043c1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:45.098868 10011 tablet_server.cc:196] TabletServer@127.9.198.193:0 shutdown complete.
I20260709 12:18:45.152897 10011 tablet_server.cc:179] TabletServer@127.9.198.194:0 shutting down...
W20260709 12:18:45.158768 24049 consensus_peers.cc:597] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 -> Peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Couldn't send request to peer e1961a8ed25e49c0a2a4c95a8ec043c1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:45.171864 24050 consensus_peers.cc:597] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 -> Peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Couldn't send request to peer a8c21fcb800b438b9df58d40c6853250. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:45.191447 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:45.192520 10011 tablet_replica.cc:333] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.193378 10011 raft_consensus.cc:2243] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.194523 10011 raft_consensus.cc:2272] T 310753142d224fbd95e46c25cc565491 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.197310 10011 tablet_replica.cc:333] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.198160 10011 raft_consensus.cc:2243] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.198833 10011 raft_consensus.cc:2272] T fb92272c273649dd942dc7b1843b9e5c P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.201292 10011 tablet_replica.cc:333] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.202276 10011 raft_consensus.cc:2243] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.203555 10011 raft_consensus.cc:2272] T dfde0376f8d84d329ec80c542e7c6e7a P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:45.205976 24050 consensus_peers.cc:597] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 -> Peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Couldn't send request to peer a8c21fcb800b438b9df58d40c6853250. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:45.206975 10011 tablet_replica.cc:333] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.207720 10011 raft_consensus.cc:2243] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.208696 10011 raft_consensus.cc:2272] T 88df43e9a51548b2be863cad3662f67a P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.211637 10011 tablet_replica.cc:333] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.212337 10011 raft_consensus.cc:2243] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.213300 10011 raft_consensus.cc:2272] T 6d3e13d378d74eab8e016be08fe8832f P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.216022 10011 tablet_replica.cc:333] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.216713 10011 raft_consensus.cc:2243] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.217269 10011 raft_consensus.cc:2272] T bfed3369f649424298588422d695690a P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.219882 10011 tablet_replica.cc:333] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.220616 10011 raft_consensus.cc:2243] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.221617 10011 raft_consensus.cc:2272] T d22599f2e4d04980829f7a39547e0117 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.224572 10011 tablet_replica.cc:333] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.225332 10011 raft_consensus.cc:2243] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.226306 10011 raft_consensus.cc:2272] T 4724a9f868554ae18d7057a7a81827b0 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.231693 10011 tablet_replica.cc:333] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.232564 10011 raft_consensus.cc:2243] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.233361 10011 raft_consensus.cc:2272] T be9fe9035a814d269d247bde6ab34d50 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:45.235183 24050 consensus_peers.cc:597] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 -> Peer a8c21fcb800b438b9df58d40c6853250 (127.9.198.194:46607): Couldn't send request to peer a8c21fcb800b438b9df58d40c6853250. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:45.236883 10011 tablet_replica.cc:333] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.237741 10011 raft_consensus.cc:2243] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.238260 10011 raft_consensus.cc:2272] T 550bb56a01054b30a9262bb704bd1d2b P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.241057 10011 tablet_replica.cc:333] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.241720 10011 raft_consensus.cc:2243] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.242790 10011 raft_consensus.cc:2272] T ab7390595cfa43c2a2f088f0cb502840 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.246115 10011 tablet_replica.cc:333] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.247030 10011 raft_consensus.cc:2243] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.247776 10011 raft_consensus.cc:2272] T 64f33ba99e41400b8d16dae902c15b22 P a8c21fcb800b438b9df58d40c6853250 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.251415 10011 tablet_replica.cc:333] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.252250 10011 raft_consensus.cc:2243] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.253018 10011 raft_consensus.cc:2272] T 6ac01989752e43fab92f98ee7a0a3d51 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.256865 10011 tablet_replica.cc:333] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.257876 10011 raft_consensus.cc:2243] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.259253 10011 raft_consensus.cc:2272] T 15b60d1559cb4f40a2463a3d2ef322d4 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.262883 10011 tablet_replica.cc:333] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.263790 10011 raft_consensus.cc:2243] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.264644 10011 raft_consensus.cc:2272] T 4bad77401a0647049e6f37a773c719f6 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:45.266345 24049 consensus_peers.cc:597] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 -> Peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Couldn't send request to peer e1961a8ed25e49c0a2a4c95a8ec043c1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:45.268513 10011 tablet_replica.cc:333] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.269511 10011 raft_consensus.cc:2243] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.270301 10011 raft_consensus.cc:2272] T 4ab0e9f23ac94f85b5cc283501aa7625 P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.274145 10011 tablet_replica.cc:333] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.274813 10011 raft_consensus.cc:2243] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.275346 10011 raft_consensus.cc:2272] T f8dc14c966494b2f930c926041c32bae P a8c21fcb800b438b9df58d40c6853250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.277894 10011 tablet_replica.cc:333] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250: stopping tablet replica
I20260709 12:18:45.278466 10011 raft_consensus.cc:2243] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.279006 10011 raft_consensus.cc:2272] T 14b7bfd5aa9844a685f6a3384f7f1d65 P a8c21fcb800b438b9df58d40c6853250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.325768 10011 tablet_server.cc:196] TabletServer@127.9.198.194:0 shutdown complete.
W20260709 12:18:45.365020 24049 consensus_peers.cc:597] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 -> Peer e1961a8ed25e49c0a2a4c95a8ec043c1 (127.9.198.193:41953): Couldn't send request to peer e1961a8ed25e49c0a2a4c95a8ec043c1. Status: Network error: Client connection negotiation failed: client connection to 127.9.198.193:41953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:45.370535 10011 tablet_server.cc:179] TabletServer@127.9.198.195:0 shutting down...
I20260709 12:18:45.404584 10011 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:45.405582 10011 tablet_replica.cc:333] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.406555 10011 raft_consensus.cc:2243] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.407284 10011 raft_consensus.cc:2272] T 310753142d224fbd95e46c25cc565491 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.409817 10011 tablet_replica.cc:333] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.410465 10011 raft_consensus.cc:2243] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.411003 10011 raft_consensus.cc:2272] T 88df43e9a51548b2be863cad3662f67a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.413280 10011 tablet_replica.cc:333] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.413980 10011 raft_consensus.cc:2243] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.414527 10011 raft_consensus.cc:2272] T dfde0376f8d84d329ec80c542e7c6e7a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.416734 10011 tablet_replica.cc:333] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.417313 10011 raft_consensus.cc:2243] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.418300 10011 raft_consensus.cc:2272] T fb92272c273649dd942dc7b1843b9e5c P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.420701 10011 tablet_replica.cc:333] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.421351 10011 raft_consensus.cc:2243] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.421896 10011 raft_consensus.cc:2272] T 6d3e13d378d74eab8e016be08fe8832f P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.424232 10011 tablet_replica.cc:333] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.425009 10011 raft_consensus.cc:2243] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.425524 10011 raft_consensus.cc:2272] T bfed3369f649424298588422d695690a P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.427942 10011 tablet_replica.cc:333] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.428581 10011 raft_consensus.cc:2243] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.429133 10011 raft_consensus.cc:2272] T 14b7bfd5aa9844a685f6a3384f7f1d65 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.431440 10011 tablet_replica.cc:333] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.432107 10011 raft_consensus.cc:2243] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.433051 10011 raft_consensus.cc:2272] T 550bb56a01054b30a9262bb704bd1d2b P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.435900 10011 tablet_replica.cc:333] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.436784 10011 raft_consensus.cc:2243] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.437711 10011 raft_consensus.cc:2272] T 4ab0e9f23ac94f85b5cc283501aa7625 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.440840 10011 tablet_replica.cc:333] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.441491 10011 raft_consensus.cc:2243] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.442018 10011 raft_consensus.cc:2272] T f8dc14c966494b2f930c926041c32bae P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.444262 10011 tablet_replica.cc:333] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.444794 10011 raft_consensus.cc:2243] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.445243 10011 raft_consensus.cc:2272] T 4bad77401a0647049e6f37a773c719f6 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.447604 10011 tablet_replica.cc:333] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.448100 10011 raft_consensus.cc:2243] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.448544 10011 raft_consensus.cc:2272] T 6ac01989752e43fab92f98ee7a0a3d51 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.450845 10011 tablet_replica.cc:333] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.451360 10011 raft_consensus.cc:2243] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.451813 10011 raft_consensus.cc:2272] T 15b60d1559cb4f40a2463a3d2ef322d4 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.454008 10011 tablet_replica.cc:333] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.454551 10011 raft_consensus.cc:2243] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.455108 10011 raft_consensus.cc:2272] T ab7390595cfa43c2a2f088f0cb502840 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.457342 10011 tablet_replica.cc:333] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.457942 10011 raft_consensus.cc:2243] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.458781 10011 raft_consensus.cc:2272] T 64f33ba99e41400b8d16dae902c15b22 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.460988 10011 tablet_replica.cc:333] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.461505 10011 raft_consensus.cc:2243] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.462069 10011 raft_consensus.cc:2272] T d22599f2e4d04980829f7a39547e0117 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.464552 10011 tablet_replica.cc:333] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.465094 10011 raft_consensus.cc:2243] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:45.465620 10011 raft_consensus.cc:2272] T 4724a9f868554ae18d7057a7a81827b0 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.468006 10011 tablet_replica.cc:333] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6: stopping tablet replica
I20260709 12:18:45.468495 10011 raft_consensus.cc:2243] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.469432 10011 raft_consensus.cc:2272] T be9fe9035a814d269d247bde6ab34d50 P 0cbb7c136dd3451c93deee0d3fd64be6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.538513 10011 tablet_server.cc:196] TabletServer@127.9.198.195:0 shutdown complete.
I20260709 12:18:45.578811 10011 master.cc:562] Master@127.9.198.254:46803 shutting down...
I20260709 12:18:45.608132 10011 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:45.608793 10011 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:45.609174 10011 tablet_replica.cc:333] T 00000000000000000000000000000000 P ee3d396e148244428354d966418022cf: stopping tablet replica
I20260709 12:18:45.631063 10011 master.cc:584] Master@127.9.198.254:46803 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (23749 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (44963 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (475548 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 12:18:45.694687 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2232 messages since previous log ~34 seconds ago
I20260709 12:18:45.694953 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2232 messages since previous log ~35 seconds ago
I20260709 12:18:45.695154 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2231 messages since previous log ~35 seconds ago
I20260709 12:18:45.695372 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:18:45.695580 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:18:45.695825 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 5 messages since previous log ~184 seconds ago
I20260709 12:18:45.696089 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 9 messages since previous log ~0 seconds ago
I20260709 12:18:45.696401 10011 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~22 seconds ago
